llama.test.test_filehandler module

Test FileHandler and other related basic LLAMA classes found in llama.filehandler. Also provides useful methods for testing FileHandler subclasses in general.

class llama.test.test_filehandler.AbstractTestObsolescence

Bases: abc.ABC

Test our ability to mark files as obsolete in the expected ways (with the expected effects on file generation). These test are run with a mock Pipeline with topology (directed left-to-right):

   ___________
  /   _____   \
 /   /     \   \
1---2---3---4---5
 \_____/

This allows for tests of a few different scenarios in which file generation ordering can be tested to have the correct ordering. As a trivial example, changing 1 will obsolete all remaining FileHandler instances, but because they depend on one another in sequence, the regeneration order must be in increasing order of label value. This lets us test methods related to checking file obsolescence and reacting by updating output files.

EVENTID = 'obsolescence'

property event

The event for this test.

abstract main()

Run the main part of the test (after the temporary test directory has been set up and the files in the MockPipeline have been generated). This is where you should put obsolescence-related test logic.

property rundir

The rundir for this test (if it is set). Raises an AttributeError if none has yet been specified.

sorted_by_generation_time()

Get a list of the generated FileHandlers in temporal order of file generation.

test()

Run the main test after having set everything up in a temporary test directory.

class llama.test.test_filehandler.MockFh1

Bases: llama.filehandler.FileHandler

llama.test.test_filehandler.MockFh1 is created from external triggers. It therefore has no LLAMA-representable input dependencies, but instead acts as initial input for other FileHandler classes.

DEPENDENCIES = ()

FILENAME = 'MockFh1.txt'

MANIFEST_TYPES = (<class 'llama.test.test_filehandler.MockFh1'>,)

UR_DEPENDENCIES = ()

UR_DEPENDENCY_TREE = frozenset({})

class llama.test.test_filehandler.MockFh2

Bases: llama.filehandler.FileHandler

Required input files for llama.test.test_filehandler.MockFh2 (located on the far right of the graph). For a typical trigger, the leftmost files will be generated as triggers become available. They will be used as input for files to their right, eventually allowing llama.test.test_filehandler.MockFh2 to be generated.

DEPENDENCIES = (<class 'llama.test.test_filehandler.MockFh1'>,)

FILENAME = 'MockFh2.txt'

MANIFEST_TYPES = (<class 'llama.test.test_filehandler.MockFh2'>,)

UR_DEPENDENCIES = (<class 'llama.test.test_filehandler.MockFh1'>,)

UR_DEPENDENCY_TREE = frozenset({<class 'llama.test.test_filehandler.MockFh1'>})

class llama.test.test_filehandler.MockFh3

Bases: llama.filehandler.FileHandler

Required input files for llama.test.test_filehandler.MockFh3 (located on the far right of the graph). For a typical trigger, the leftmost files will be generated as triggers become available. They will be used as input for files to their right, eventually allowing llama.test.test_filehandler.MockFh3 to be generated.

DEPENDENCIES = (<class 'llama.test.test_filehandler.MockFh1'>, <class 'llama.test.test_filehandler.MockFh2'>)

FILENAME = 'MockFh3.txt'

MANIFEST_TYPES = (<class 'llama.test.test_filehandler.MockFh3'>,)

UR_DEPENDENCIES = (<class 'llama.test.test_filehandler.MockFh1'>, <class 'llama.test.test_filehandler.MockFh2'>)

UR_DEPENDENCY_TREE = ImmutableDict({<class 'llama.test.test_filehandler.MockFh2'>: ImmutableDict({<class 'llama.test.test_filehandler.MockFh1'>: ImmutableDict({})}), <class 'llama.test.test_filehandler.MockFh1'>: ImmutableDict({})})

class llama.test.test_filehandler.MockFh4

Bases: llama.filehandler.FileHandler

Required input files for llama.test.test_filehandler.MockFh4 (located on the far right of the graph). For a typical trigger, the leftmost files will be generated as triggers become available. They will be used as input for files to their right, eventually allowing llama.test.test_filehandler.MockFh4 to be generated.

DEPENDENCIES = (<class 'llama.test.test_filehandler.MockFh2'>, <class 'llama.test.test_filehandler.MockFh3'>)

FILENAME = 'MockFh4.txt'

MANIFEST_TYPES = (<class 'llama.test.test_filehandler.MockFh4'>,)

UR_DEPENDENCIES = (<class 'llama.test.test_filehandler.MockFh1'>, <class 'llama.test.test_filehandler.MockFh2'>, <class 'llama.test.test_filehandler.MockFh3'>)

UR_DEPENDENCY_TREE = ImmutableDict({<class 'llama.test.test_filehandler.MockFh2'>: ImmutableDict({<class 'llama.test.test_filehandler.MockFh1'>: ImmutableDict({})}), <class 'llama.test.test_filehandler.MockFh3'>: ImmutableDict({<class 'llama.test.test_filehandler.MockFh2'>: ImmutableDict({<class 'llama.test.test_filehandler.MockFh1'>: ImmutableDict({})}), <class 'llama.test.test_filehandler.MockFh1'>: ImmutableDict({})})})

class llama.test.test_filehandler.MockFh5

Bases: llama.filehandler.FileHandler

Required input files for llama.test.test_filehandler.MockFh5 (located on the far right of the graph). For a typical trigger, the leftmost files will be generated as triggers become available. They will be used as input for files to their right, eventually allowing llama.test.test_filehandler.MockFh5 to be generated.

DEPENDENCIES = (<class 'llama.test.test_filehandler.MockFh1'>, <class 'llama.test.test_filehandler.MockFh4'>)

FILENAME = 'MockFh5.txt'

MANIFEST_TYPES = (<class 'llama.test.test_filehandler.MockFh5'>,)

UR_DEPENDENCIES = (<class 'llama.test.test_filehandler.MockFh1'>, <class 'llama.test.test_filehandler.MockFh2'>, <class 'llama.test.test_filehandler.MockFh3'>, <class 'llama.test.test_filehandler.MockFh4'>)

UR_DEPENDENCY_TREE = ImmutableDict({<class 'llama.test.test_filehandler.MockFh1'>: ImmutableDict({}), <class 'llama.test.test_filehandler.MockFh4'>: ImmutableDict({<class 'llama.test.test_filehandler.MockFh2'>: ImmutableDict({<class 'llama.test.test_filehandler.MockFh1'>: ImmutableDict({})}), <class 'llama.test.test_filehandler.MockFh3'>: ImmutableDict({<class 'llama.test.test_filehandler.MockFh2'>: ImmutableDict({<class 'llama.test.test_filehandler.MockFh1'>: ImmutableDict({})}), <class 'llama.test.test_filehandler.MockFh1'>: ImmutableDict({})})})})

class llama.test.test_filehandler.TestGenerationOrder

Bases: llama.test.test_filehandler.AbstractTestObsolescence

Make sure that, by obsoleting a file, we end up regenerating its descendants in the correct order.

main()

Make sure we regenerate files in the correct order.

class llama.test.test_filehandler.TestObsolescenceAndLocking

Bases: llama.test.test_filehandler.AbstractTestObsolescence

Test FileHandler’s ability to mark files as obsolete, then lock those files, marking them as non-obsolete in perpetuity, then unlock them again, returning to the original state.

assert_obsolete()

Check that everything else is obsolete after changing MockFh1.

main()

Run the main part of the test (after the temporary test directory has been set up and the files in the MockPipeline have been generated). This is where you should put obsolescence-related test logic.

llama.test.test_filehandler.check_filehandler_definition_consistency(filehandler: llama.filehandler.FileHandler)

Check whether a FileHandler has been consistently defined, raising an AssertionError if not.

llama.test.test_filehandler.check_required_attributes(filehandler: llama.filehandler.FileHandler, err: bool = False)

Return False if filehandler does not have all of its required class constants (in filehandler.required_attributes()) defined. If err is True, raise an assertion error instead of returning False. Return True if all required attributes are set (meaning that this FileHandler class is valid and ready to use).

llama.test.test_filehandler.implemented_filehandler(item)

Check whether an object is an implemented FileHandler (not abstract, all required attributes set).

llama.test.test_filehandler.make_mock_filehandler(name, deps)

Make a mock filehandler for simple mock pipeline tests. Give it the requested DEPENDENCIES as deps. Will simply write the current timestamp to file.

llama.test.test_filehandler.test_filehandler_definition_consistency()

Run check_filehandler_definition_consistency on all FileHandler classes that can be found defined in llama and its submodules that have their required_attributes set.