llama.vetoes module¶

A mechanism for vetoing steps in an analysis.

exception llama.vetoes.VetoException¶

Bases: OSError

Indicates that a FileHandler’s generation was vetoed and should proceed no futher at this time. This is a benign exception that should be caught at file generation time.

class llama.vetoes.VetoHandler(eventdir, manifest, vetoes)¶

Bases: llama.vetoes.VetoHandlerTuple, llama.flags.FlagsMixin

A class that performs veto checks to see whether a FileHandler for a specific trigger should be vetoed. Instances can be called like functions to mark a FileHandler as vetoed.

check()¶

No return value. Raises a VetoException if this file has somehow been vetoed, which should be interpreted as meaning that file generation should be aborted.

First checks if self.permanently_vetoed() returns True (this is to be interpreted as meaning that the file should never be generated); if so, VetoException is raised.

Next, runs all veto functions in self.vetoes to see if these files have been vetoed in any way. If and when the first veto fails (returns True), its corresponding action (see the vetoes argument in __init__) is run, further vetoes are not checked, and VetoException is raised.

If these procedures are completed without a VetoException being raised, the method returns None.

permanently_vetoed()¶: Check whether the files in self.manifest should be generated based on whether their vetofiles exist (see vetofilepaths and vetofilenames); as long as this is the case, the files will be considered permanently vetoed and should not be generated.

read_json()¶: Read in the JSON dump of this vetofile. Returns None if the file is not vetoed.

unveto()¶: Remove PERMANENT veto status by deleting vetofiles for the file names specified in self.manifest and event directory specified by eventdir. This does not affect the checks contained in self.vetoes that are called by self.check, so file generation will still get vetoed if those automated checks fail. This method is more useful for removing permanent vetoes from files that were manually vetoed or vetoed under older pipeline versions.

property vetofilenames¶: A list of filenames whose existence indicates that these files should never be generated (at least for as long as these files exist). The contents of these files can optionally contain text descriptions of why the veto happened.

property vetofilepaths¶: Absolute paths to vetofiles listed in self.vetofilenames (created by joining self.eventdir to those filenames).

class llama.vetoes.VetoHandlerTuple(eventdir, manifest, vetoes)¶

Bases: tuple

property eventdir¶: Alias for field number 0

property manifest¶: Alias for field number 1

property vetoes¶: Alias for field number 2

class llama.vetoes.VetoMixin¶

Bases: object

A feature for FileHandler-like objects (producing a manifest and an eventdir) that defines a list of vetoes in a way that inherits additively from superclasses. Notably, provides a veto property returning a VetoHandler for this instance. New vetoes can be added to classes by defining them in class_vetoes (vetoes from superclasses class_vetoes attributes are dynamically included in the VetoHandler returned by veto. See VetoMixin.vetoes docstring for details.

class_vetoes = ()¶

static decorate_checkin(func)¶: If a VetoException was raised during generation, copy vetoes from the temporary event directory back to the parent directory and commit them.

static decorate_checkout(func)¶: Copy any vetoes for this filehandler over to the generation directory.

static decorate_generate(func)¶: Check whether this filehandler has been vetoed before proceeding with generation.

property veto¶: Returns a VetoHandler instance for this FileHandler. See VetoHandler docstring for more info. Combines vetoes from all superclasses.

classmethod vetoes()¶

The list of veto checks and their corresponding actions called by

self.veto.checks(). Returns the vetoes defined in cls.class_vetoes, where vetoes specific to this class are defined, along with the vetoes returned by cls.class_vetoes in all superclasses; for example, the order of vetoes pulled from class_vetoes (in a class and and its superclasses) for a class F with inheritance tree

A   B   C
 \ /   /
  D   E
   \ /
    F

would be: F, D, A, B, E, C (with duplicates after the first instance of a veto check removed, where the ordering of superclasses is listed left-to-right in the above graph).

By default, no vetoes are applied to a base FileHandler. Vetoes must be added to FileHandler subclasses. If you really, truly need to get rid of vetoes from superclasses, consider not inheriting from those superclasses or factoring the veto functionality out of those superclasses as a mixin before overriding this property to only return a reduced veto list.

A description of the form of this data structure from the VetoHandler.__init__ docstring:

An iterable of N length-2 iterables of the form:

((veto1, action1), …, (vetoN, actionN))

matching veto functions to action functions.

veto functions are functions that take eventdir as arguments and return True if that particular veto criterion has been met (for example, a veto could return True if a trigger has somehow been marked as a test trigger).

action functions are callables that indicate some action that should be taken to handle the veto. action can also be None for the default behavior: to mark the set of filenames (specified by manifest) for this trigger as vetoed, which would prevent those files from being generated.

Alternatively, you can also use an actual function for action to do something more subtle than just vetoing the file (e.g. delaying file generation, advanced error logging, dummy testing, etc; this could even involve an alternative file generation method for test cases or edge cases). The veto functions will be run in order and the first to return True will have its action executed; the rest will be ignored.

NOTE that if you specify an action, the file will NOT be vetoed by default (since your veto-handling might involve something like delaying file generation or generating it through some other method, both of which are cases where you are not actually trying to prevent the file from being generated). If you want to fully veto file generation, you will have to manually call this VetoHandler instance at the end of your action to mark these files as PERMANENTLY vetoed and prevent file generation.

A VetoException WILL always be raised, however, since the called action is supposed to cancel any IMMEDIATE file generation by the default method (even if later attempts are still allowed).

llama.vetoes.eventdir_path_contains_string_injection(eventdir)¶: If the path to the trigger directory contains the substring ‘INJECTION’ (case-insensitive), this veto is triggered (returns True).

llama.vetoes.eventdir_path_contains_string_manual(eventdir)¶: If the path to the trigger directory contains the substring ‘MANUAL’ (case-insensitive), this veto is triggered (returns True).

llama.vetoes.eventdir_path_contains_string_scratch(eventdir)¶: If the path to the trigger directory contains the substring ‘SCRATCH’ (case-insensitive), this veto is triggered (returns True).

llama.vetoes.eventdir_path_contains_string_test(eventdir)¶: If the path to the trigger directory contains the substring ‘TEST’ (case-insensitive), this veto is triggered (returns True).

llama.vetoes.generation_dir_copy_vetoes(vetoed, tmpdir)¶: Copy any vetofiles for the vetoed filehandler from the temporary directory tmpdir back to the eventdir. Vital for cleaning up after a failed generation attempt (in which the filehandler manifest would NOT be copied because the files should never have been generated; see _generation_dir_copy_manifest).

llama.vetoes.utcnow()¶: Return a new datetime representing UTC day and time.