llama.com.s3 package

Tools for uploading to/downloading from AWS S3 APIs (including DigitalOcean’s S3-interface clone for DigitalOcean Spaces).

If using DigitalOcean spaces, for example, You will need to configure an access token for DigitalOcean spaces and set the key and secret as environmental variables DIGITALOCEAN_SPACES_KEY and DIGITALOCEAN_SPACES_SECRET, respectively (this should work for AWS S3 as well, though of course the link for generating the tokens will be different).

class llama.com.s3.PrivateFileCacher

Bases: llama.com.s3.PrivateFileCacherTuple

Like llama.utils.RemoteFileCacher but for private files stored behind an AWS S3 interface. If the file is not present locally, it will be automatically downloaded to the pathlib.Path returned by get() (provided that you have API credentials with access permissions for that file).

Parameters

  • key (str) – The key of the remote file object.

  • bucket (str, optional) – The S3 bucket in which the file is stored.

  • localpath (str, optional) – The (optional) local path at which to cache this resource. By default, will just be /root/.cache/llama/objects/filename where filename is actually taken from the remote URL filename.

get()

If the file is not available locally, download it and store it at localpath (do nothing if present). Return localpath.

class llama.com.s3.PrivateFileCacherTuple(key, bucket, localpath)

Bases: tuple

property bucket

Alias for field number 1

property key

Alias for field number 0

property localpath

Alias for field number 2

llama.com.s3.get_client(region_name='nyc3', endpoint_url='https://nyc3.digitaloceanspaces.com', aws_access_key_id='JQWCJFVDROLQKSMJTZW2', aws_secret_access_key='gx1pUAlw6WOAy6y3jtQ0AYt/0Q4EBkMAxItmcFDnW1c', **kwargs)

Get a boto3 client connecting to the given DigitalOcean Spaces/AWS S3 region and endpoint.

Parameters

  • region_name (str, optional) – The server region. This is the geographical region in which your servers reside. Check your DigitalOcean or AWS account to find this.

  • endpoint_url (str, optional) – The endpoint URL for your specific Spaces/S3 instance. Again, check your account to find this.

  • aws_access_key_id (str, optional) – Your access key, generated on your account website. You can only view this when you create it, so if you lost track of an old version, just delete it and make new credentials. For DigitalOcean, you can do this here. If not provided, will default to the value of the DIGITALOCEAN_SPACES_KEY environmental variable, or None if it doesn’t exist (which will result in an authentication error).

  • aws_secret_access_key (str, optional) – The secret corresponding to your aws_access_key_id. Create this at the same time you create your aws_access_key_id (see notes above). If not provided, will default to the value of the DIGITALOCEAN_SPACES_SECRET environmental variable, or None if it doesn’t exist (which will result in an authentication error).

  • **kwargs – Extra keyword arguments to pass to boto3.session.Session.client.

Returns

client – A client for interacting with the specified Spaces/S3 instance using the specified credentials. You can use this client to interact with the S3 API for file storage, retrieval, permissions modifications, etc. See boto3.session.Session.client for more details on the interface.

Return type

boto3.session.Session.Client

llama.com.s3.upload_file(filename, key, bucket='llama', public=False, tries=5, **kwargs)

Upload file to a DigitalOcean Spaces/AWS S3 bucket.

Parameters

  • filename (str) – Local path to the file you wish to upload.

  • key (str) – The object key, analogous to a remote file path; the remote file will be available at /<bucket>/<key>. You can put slashes in the key, which will be treated as subdirectories on the DigitalOcean web file browser.

  • bucket (str, optional) – Name of the target bucket. For DigitalOcean Spaces, this is the naem of the directory in the root Spaces directory, e.g. bucket=llama will put everything under /llama/ remotely.

  • public (bool, optional) – is True, the file will be publicly-accessible.

  • tries (int, optional) – How many times to try the upload before giving up due to errors.

  • **kwargs – Keyword arguments will be passed to get_client to initialize it, overriding its defaults. Use this to specify access credentials and upload target.

Returns

url – If public is True, the remote URL at which the resource can be publicly accessed; otherwise, None.

Return type

str or None