llama.intent module¶

Classes for tracking whether files are cooling down after a failed generation attempt or are currently being generated.

class llama.intent.CoolDown(eventdir, manifest_filehandlers)¶

Bases: llama.classes.RiderFile, llama.classes.JsonRiderMixin

An object for keeping track of whether a particular file failed to generate recently and determining how long the pipeline should wait before trying again to generate it. Instances of this class are meant to be calculated as properties from FileHandlers and similar classes.

Parameters

manifest (list) – List of names of the files that need to be generated (and which might fail, necessitating a cooldown countdown).
eventdir (str) – Path to the directory containing that file.

in_progress()¶: Determine whether a cooldown is still in progress for generating this file, i.e. whether this file generation routine failed too recently for another generation attempt.

rider_fmt = '.{}.cooldown.json'¶

write()¶

Write a json file with the name .<FILENAME>.cooldown.json for each file in the manifest containing the time of the last attempt at generating the file as well as the number of consecutive failed attempts at generating the file. Format is

{: “num_attempts”: 3, “last_attempt”: “2016-09-12T17:16:20.337867”

}

Where last_attempt in particular is an ISO formatted time string.

class llama.intent.CoolDownMixin¶

Bases: object

Add a cooldown property to FileHandler class.

property cooldown¶: Set and get information about whether an attempt to generate this file failed, and, if so, whether enough time has passed to warrant retrying that file’s generation.

static decorate_checkin(func)¶: If file generation failed, mark the file as cooling down when the generation attempt is being checked back in.

static decorate_checkout(func)¶: Make sure the file is not cooling down before trying to generate it. If it is cooling down, raise a GenerationError. This will wrap a FileHandler.generate method, so it needs to have the same signature.

class llama.intent.CoolDownParams(base, increment, maximum)¶

Bases: tuple

property base¶: Alias for field number 0

property increment¶: Alias for field number 1

property maximum¶: Alias for field number 2

class llama.intent.EventLock(eventdir)¶

Bases: object

A class for getting a cooperative lock on an Event directory.

acquire(wait=True, steal=True)¶

Try to get a cooperative lock on the current directory. Lock expires after LOCK_TIMEOUT seconds. Lock directory will be cleaned up at exit of program.

Parameters

wait (bool, optional) – If True, and the lock cannot be acquired, keep waiting until the lock goes away (note that this will not time out unless steal is also True).
steal (bool, optional) – If this and wait are True, if the lock times out without being deleted, delete it and acquire lock on the assumption that a previous transaction failed. Note that you can still fail to acquire lock after the timeout if another thread/process manages to acquire it after the expiration but before you try to acquire lock.

Returns

success – Whether lock was acquired.

Return type

bool

expiration()¶: Get the UNIX timestamp for when the lock expires. If not locked, returns 0. After this time, it’s likely that the locking process crashed and the lock can be safely freed.

release()¶: Try removing lock. Returns True if an existing lock was successfully removed and False if the lock did not previously exist.

renew()¶: Set the modification time of the lock directory to the current time, effectively extending the timeout. If the lockdir does not exist, FileNotFound will be raised. It’s up to you to make sure to keep renewing your lock well before LOCK_TIMEOUT to make sure you don’t lose lock.

class llama.intent.Intent(eventdir, manifest_filehandlers)¶

Bases: llama.classes.RiderFile, llama.classes.JsonRiderMixin

An object tracking intent to make a file. Tracks whether another process should start making this file. Saves the timestamp at which intent expires to a rider file for each file in a FileHandler’s manifest.

in_progress()¶: Check whether this file is already being generated and has not timed out yet.

rider_fmt = '.{}.intent.json'¶

write(timeout)¶: Write the timeout (as a UNIX timestamp) at which this file has timed out and another file can attempt to generate it.

class llama.intent.IntentMixin¶

Bases: object

static decorate_checkin(func)¶: Make sure the file’s intent is still set, and clear it up regardless of whether checkin succeeds. This will wrap FileHandler.checkin, so it needs to have the same signature.

static decorate_checkout(func)¶: Make sure the file is not currently being generated before trying to generate it. If it is currently being generated, raise a GenerationError. This will wrap FileHandler.checkout, so it needs to have the same signature.

property eventlock¶: Get a cooperative lock on the event directory.

property intent¶: Set and get information about whether this file is being generated.

llama.intent.utcnow()¶: Return a new datetime representing UTC day and time.