humancellatlas / data-bundle-examples Goto Github PK

So that we don't fail the data store integration test by attempting to load garbage.

In the last sprint we only were able to load 22 sample bundles. We're waiting a blue box fix. But after this fix is in place, deploy a new system and load all of Jim's samples.

@TimothyTickle

Some comments the analysis.json example

https://github.com/HumanCellAtlas/data-bundle-examples/blob/develop/10X_v2/analysis/analysis.json

1 - Names in the inputs array e.g "name": "left_sample",

sample feels like an overloaded term here, we have a core entity called sample, if sample here doesn't mean biological sample, can you chose another word? read looks like it might work or sequence

2 - reference bundle

For this, I assume this is the bundle which contains the reference genome and geneset or any other standard files which are used as part of an analysis, it would be good to pick a more specific term like reference_genome or analysis_reference as reference is a term which can mean many different things to many different people so using reference along could be confused

"reference_bundle" : "bluebox://03gh7eb0-23a3-4898-b043-f7e982de281b/v1"

3 - Attributes about analyses for portal developers/users

It would be good to understand what the portal writers use case is for analysis bundles and if it is desirable to have any of the information from the work flow file pointed to be computational_method pulled out as directly readable attributes like program names, genome version etc rather than having to parse the computational method file or the reference bundle in order to discover those things

4 - Structure

@tburdett it would be good for you to review this too and comment from a structure/format perspective and see if it breaks any of your current principles on how these files should be formatted

https://github.com/HumanCellAtlas/data-bundle-examples/blob/e89afc2fa9cfcdb8f8d3b642c04245311839cd1a/bin/bundle_tools/s3.py
https://github.com/HumanCellAtlas/data-bundle-examples/blob/5e3849874300f4461f0001a479a449cc9241cf21/bin/stage_and_store.py

I was looking in the repo readme for instructions on how to load an individual bundle for testing, however I couldn't find any. Can we add such a section?

P.S. By "load a bundle", I mean download the data files for it and upload the result to the DSS.

Write a quick script to load the collection of sample data bundles here into a running blue box instance.