1# (c) Stefan Countryman, 2019 

2 

3""" 

4Take the output files made by ``llama dev background table`` for each 

5population you are interested in and collate them into a single HDF5 file for 

6p-value calculations. All files are expected to be in the current directory, 

7and the output will be in this directory as well. 

8 

9Both input and output files correspond to background significance values for 

10various source populations. The HDF5 file simply contains a dictionary of 

11ascending-order arrays whose values are the test statistics of background 

12simulations and whose keys are the name of the background population in a 

13specific heirarchical order. These keys are automatically selected when 

14`populations-hdf5-collater` is run in this directory by splitting 

15the source `.txt` table filenames on hyphens (`-`). For example, if the 

16populations folder containg `bns-design.txt` `bns-o2.txt`, the output HDF5 

17populations file will have the following dictionary in it: 

18 

19.. code:: json 

20 

21 { 

22 "bns": { 

23 "design": array([...]), 

24 "o2": array([...]) 

25 } 

26 } 

27 

28The output HDF5 file is saved to the current directory by default, 

29but if you want to use it for LLAMA p-value calculations, make sure to upload 

30both the tarballed populations directory and the compressed HDF5 file to cloud 

31storage using ``llama dev upload`` and then store the new URLs in the 

32``RemoteFileCacher`` instances used for 

33``llama.files.coinc_significance.NEUTRINO_POPULATIONS`` and 

34``llama.files.coinc_significance.NEUTRINO_POPULATIONS_TARBALL``. 

35 

36To summarize: you want to download the existing population tarball, decompress 

37it with ``tar -xzf populations.tar.gz`` (assuming it's named 

38``populations.tar.gz``, of course), move your ``llama dev background table`` 

39output text files to that directory, name the new tables in the expected format 

40of ``{SEARCH_TYPE}-{POPULATION}-{LVCRUN}-{GWDETECTORS}.txt`` (e.g. 

41``opa-bbh-o3-H1V1.txt``), run this script in that directory, put the text 

42tables in a compressed tarball called e.g. ``populations.tar.gz`` by running 

43``tar -czf populations.tar.gz populations`` in the containing directory, upload 

44both that and the HDF5 file to the cloud with ``llama dev upload``, and then 

45use the URLs in the manifest printed by ``llama dev upload`` to update the 

46values of ``NEUTRINO_POPULATIONS`` and ``NEUTRINO_POPULATIONS_TARBALL`` in the 

47``llama.files.coinc_significance`` source code. 

48"""