llama.io.classes module¶

Base classes for input/output operations.

class llama.io.classes.CoolDownIO¶

Bases: llama.io.classes.ManifestIOComponent

Base class for input/output operations on a cooldown object.

class llama.io.classes.EventIO¶

Bases: llama.io.classes.EventIOComponent

Base class for input/output operations on llama.Events classes.

class llama.io.classes.EventIOComponent¶

Bases: tuple

Base class for an IO component of IO that needs to know the active run and the eventid.

property eventid¶: Alias for field number 1

property rundir¶: Alias for field number 0

class llama.io.classes.EventLock¶

Bases: llama.io.classes.EventIOComponent

Base class for input/output operations on an EventLock object.

class llama.io.classes.FileHandlerIO¶

Bases: llama.io.classes.ManifestIOComponent

Base class for input/output operations on llama.FileHandler classes.

class llama.io.classes.FlagsIO¶

Bases: llama.io.classes.EventIOComponent

Base class for input/output operations on llama.Event flags.

class llama.io.classes.GraphExecutor¶

Bases: abc.ABC

An implementation-agnostic interface for submitting iterables of FileHandler instances that need to be updated.

abstract classmethod submit(graph) → Iterable[Tuple]¶: Submit a FileGraph instance graph for (potentially parallel) generation of its files. Returns an iterable of Tuple[FileHandler, Future] instances matching the FileHandler instance that is being generated to a Future that will either return the same successfully-generated FileHandler instance or raise any exceptions occuring during generation when its result method is called. An attempt will be made to generate all files in the graph, so downselect accordingly.

class llama.io.classes.IO¶

Bases: tuple

An abstract interface for safe input/output primitives used by LLAMA. Allows for multiple underlying file storage and computational implementations. Partial implementations are allowed by omitting arguments for certain components.

property cooldown¶: Alias for field number 8

property event¶: Alias for field number 3

property eventlock¶: Alias for field number 6

property fh¶: Alias for field number 4

property flags¶: Alias for field number 9

property generate¶: Alias for field number 1

property intent¶: Alias for field number 7

property lock¶: Alias for field number 11

property meta¶: Alias for field number 12

property run¶: Alias for field number 2

property schemes¶: Alias for field number 0

property ver¶: Alias for field number 5

property veto¶: Alias for field number 10

class llama.io.classes.IOComponent¶

Bases: abc.ABC

Base class providing utility methods for all IO components.

class llama.io.classes.IOFetcher¶

Bases: tuple

Class for fetching instantiated

property cooldown¶: Base class for input/output operations on a cooldown object. This property returns an instance of <class 'llama.io.classes.CoolDownIO'> with specific implementation set by IO class and rundir, eventid, and manifest properties. Will raise a TypeError if any of those argument are required by the IOComponent but unavailable for this object type.

property event¶: Base class for input/output operations on llama.Events classes. This property returns an instance of <class 'llama.io.classes.EventIO'> with specific implementation set by IO class and rundir, eventid, and manifest properties. Will raise a TypeError if any of those argument are required by the IOComponent but unavailable for this object type.

property eventid¶: Alias for field number 2

property eventlock¶: Base class for input/output operations on an EventLock object. This property returns an instance of <class 'llama.io.classes.EventLock'> with specific implementation set by IO class and rundir, eventid, and manifest properties. Will raise a TypeError if any of those argument are required by the IOComponent but unavailable for this object type.

property fh¶: Base class for input/output operations on llama.FileHandler classes. This property returns an instance of <class 'llama.io.classes.FileHandlerIO'> with specific implementation set by IO class and rundir, eventid, and manifest properties. Will raise a TypeError if any of those argument are required by the IOComponent but unavailable for this object type.

property flags¶: Base class for input/output operations on llama.Event flags. This property returns an instance of <class 'llama.io.classes.FlagsIO'> with specific implementation set by IO class and rundir, eventid, and manifest properties. Will raise a TypeError if any of those argument are required by the IOComponent but unavailable for this object type.

property intent¶: Base class for input/output operations on an Intent object. This property returns an instance of <class 'llama.io.classes.IntentIO'> with specific implementation set by IO class and rundir, eventid, and manifest properties. Will raise a TypeError if any of those argument are required by the IOComponent but unavailable for this object type.

property io¶: Alias for field number 0

property lock¶: Base class for input/output operations on llama.FileHandler version locks.

This property returns an instance of <class 'llama.io.classes.LockIO'> with specific implementation set by IO class and rundir, eventid, and manifest properties. Will raise a TypeError if any of those argument are required by the IOComponent but unavailable for this object type.

property manifest¶: Alias for field number 3

property meta¶: Base class for input/output operations on llama.FileHandler metadata. This property returns an instance of <class 'llama.io.classes.MetaDataIO'> with specific implementation set by IO class and rundir, eventid, and manifest properties. Will raise a TypeError if any of those argument are required by the IOComponent but unavailable for this object type.

property run¶: Base class for input/output operations on llama.Run classes. This property returns an instance of <class 'llama.io.classes.RunIO'> with specific implementation set by IO class and rundir, eventid, and manifest properties. Will raise a TypeError if any of those argument are required by the IOComponent but unavailable for this object type.

property rundir¶: Alias for field number 1

property ver¶: Base class for input/output operations on llama.Event versioning classes.

This property returns an instance of <class 'llama.io.classes.VersioningIO'> with specific implementation set by IO class and rundir, eventid, and manifest properties. Will raise a TypeError if any of those argument are required by the IOComponent but unavailable for this object type.

property veto¶: Base class for input/output operations on llama.FileHandler vetoes. This property returns an instance of <class 'llama.io.classes.VetoIO'> with specific implementation set by IO class and rundir, eventid, and manifest properties. Will raise a TypeError if any of those argument are required by the IOComponent but unavailable for this object type.

class llama.io.classes.IOMixin¶

Bases: object

A mixin providing an io attribute returning an IOFetcher for this instance with the correct IO implementation and attributes.

property io¶: An implementation-agnostic interface for input/output primitives. The specific implementation will be chosen based on the URL scheme of the rundir attribute of this class; see llama.io.get_schemes for a list of registered IO schemes.

class llama.io.classes.IntentIO¶

Bases: llama.io.classes.ManifestIOComponent

Base class for input/output operations on an Intent object.

class llama.io.classes.LockIO¶

Bases: llama.io.classes.ManifestIOComponent

Base class for input/output operations on llama.FileHandler version locks.

class llama.io.classes.ManifestIOComponent¶

Bases: tuple

Base class for an IO component of IO that needs to know the active run, eventid, and a manifest of filenames related to file generation. All filenames in the manifest share a rundir and eventid.

property eventid¶: Alias for field number 1

property manifest¶: Alias for field number 2

property rundir¶: Alias for field number 0

class llama.io.classes.MetaDataIO¶

Bases: llama.io.classes.ManifestIOComponent

Base class for input/output operations on llama.FileHandler metadata.

class llama.io.classes.NamedTupleABCMeta¶

Bases: typing.NamedTupleMeta, abc.ABCMeta

Metaclass resolver for NamedTuple and ABC. See:

https://stackoverflow.com/questions/11276037/resolving-metaclass-conflicts

class llama.io.classes.RunIO¶

Bases: llama.io.classes.RunIOComponent

Base class for input/output operations on llama.Run classes.

class llama.io.classes.RunIOComponent¶

Bases: tuple

Base class for an IO component of IO that needs to know the active run.

property rundir¶: Alias for field number 0

class llama.io.classes.VersioningIO¶

Bases: llama.io.classes.ManifestIOComponent

Base class for input/output operations on llama.Event versioning classes.

abstract commit_changes(message)¶: Commit the current snapshot of the event and record a descriptive message of the last round of changes.

property current_hash¶: Get a unique ID for the latest version of this event. The ID can vary by storage engine, but it must be unique to the file configuration specified. Equivalent to hashes with max_results=1 and filenames=['--'].

abstract filename_for_download(filename, last_hash=None)¶

abstract hashes(*filenames, pretty='', last_hash=None, max_results=None)¶: Return a list of unique IDs corresponding to event snapshots that include updates to the specified filenames. The IDs can bary by storage engine, but they must be unique to the file configuration specified by each snapshot. If specified, return at most max_results results. Returns a list of unique IDs, with optional pretty formatting following git pretty-print format string conventions (note that not all pretty print features might be implemented). If specified, return results up-to and including last_hash.

abstract init()¶: Perform any initialization tasks required to record versions of an event.

abstract is_repo()¶: Whether snapshots of this event exist in the storage engine. This can be distinct from the question of whether files for this event exist in storage.

abstract remove(*filehandlers)¶: Create a new snapshot with filehandlers and any auxilliary data removed.

abstract reset_hard(ref=None)¶: Reset the state of the pipeline to the specified ref (as returned by current_hash or hashes), cleaning messy state if it exists (for transactional storage engines, such state does not exist). This does not delete history; instead, it labels ref as the official current snapshot from which future history will be generated. To clean state and reset to the last good commit, leave ref=None.

abstract show_log(ref)¶: Show the commit message for the specified ref, which can be a UID of the sort returned by hashes or else a storage-engine-specific unambiguous identifier for a snapshot.

abstract text_graph(*filenames, style='html')¶

class llama.io.classes.VetoIO¶

Bases: llama.io.classes.ManifestIOComponent

Base class for input/output operations on llama.FileHandler vetoes.

llama.io.classes.io_fetcher_helper(func: Callable)¶: Wrapper to implement IO instantiation.