llama dev upload

Tools for uploading manifests of data files to a DigitalOcean Spaces/Amazon S3 object storage solution (for later user installation using llama install).

Prints out a manifest as a JSON dictionary mapping local filenames to tuples of corresponding remote URLs and file hashes.

usage: llama dev upload [-h] [-l LOGFILE]
                        [-v {debug,info,warning,error,critical,none}]
                        [-r ROOT] [-g GLOB] [-p PREFIX] [-b BUCKET] [-P] [-R]
                        [--dry-run]

Named Arguments

-r, --root

The root directory to upload to remote storage. (default: current directory). Paths in the printed manifest will be relative to this.

Default: “.”

-g, --glob

A glob matching files in --root that should be uploaded and thrown in the manifest. (default: all files in all subdirectories).

Default: “**/*”

-p, --prefix

A prefix to the key used on the remote storage instance. This can be something pathlike with / indicating a directory, but note that it is just a prefix prepended to the front of the relative paths found in --root matching --glob, so you’ll have to add a trailing slash if you want this to be a containing directory. (default: llama/objects/)

Default: “llama/objects/”

-b, --bucket

The storage bucket to upload to. For DigitalOcean Spaces, this is just the name of the directory in that space’s top-level directory. (default: llama)

Default: “llama”

-P, --private

Upload this file privately, leaving no public link. You probably don’t want to do this unless you’re just dumping data onto a storage space to deal with later.

Default: False

-R, --relpath

If specified, then the relative path of files from --root will become part of their key on the remote object store, sandwiched between the --prefix and the file’s sha256sum. Use this if you are manually uploading something to the object store that you want to keep track of outside of source-code, e.g. one-off build artifacts and the like. You should AVOID using this flag if you’re just interested in throwing files in an object store and organizing them with the returned manifest in a version-controlled way, since objects are by default only named after their hash. This lets you use the manifest to specify where files go on the filesystem without duplicating files on the remote object store.

Default: False

--dry-run

If specified, just print the manifest that would result from an upload and quit. Use this to see where your files will be uploaded before actually doing it.

Default: False

logging settings

-l, --logfile

File where logs should be written. By default, all logging produced by llama run goes to both an archival logfile shared by all instances of the process as well as STDERR. The archival logfile can be overridden with this argument. If you specify /dev/null or a path that resolves to the same, logfile output will be suppressed automatically. Logs written to the logfile are always at maximum verbosity, i.e. DEBUG. (default: /dev/null)

Default: “/dev/null”

-v, --verbosity

Possible choices: debug, info, warning, error, critical, none

Set the verbosity level at which to log to STDOUT; the --logfile will ALWAYS receive maximum verbosity logs (unless it is completely supressed by writing to /dev/null). Available choices correspond to logging severity levels from the logging library, with the addition of none if you want to completely suppress logging to standard out. (default: info)

Default: “info”