tunnell / siovd Goto Github PK

We can use a pandas DataFrame to store global conditions / configuration / correction information for an experiment. Have the data for the DataFrame be stored as CSV or better yet Google doc so widely accessible:

https://stackoverflow.com/questions/24063629/loading-a-generic-google-spreadsheet-in-pandas

In the dataframe:
Row: date
Column: global version

(Bonus points: multindex of date, correction name?)

Global version means a unique set of correction maps and variables (electron lifetime, liquid level height, etc) when running e.g. strax. Therefore, analysts can say "I used strax v5 and corrections v4.3".

Furthermore, global versions will be split so:

• Online (corrections used at initial processing)
• Offline
  • v1.0: uses v2 of correction Y, v0 of map X, etc
  • v1.0: uses v2 of correction Y, v2 of map X, etc

Where each of the list above is a column in dataframe.

Each element in the DataFrame will be a reference to something that can be downloaded. For example, "v2 of correction Y" would be something unique ID "correction_Y_v2". This string is stored in table. This string can be given to another function to fetch the files corresponding to this unique ID. These files can live on Github, MongoDB GridFS, HTTP, or S3.