templateflow / python-manager

A toolbox to upload and update the TemplateFlow repository

TemplateFlow Archive

This repository corresponds to the DataLad super-dataset of the TemplateFlow infrastructure. Therefore, this repository indexes actual template datasets (Git repositories), that are linked as Git submodules. This repository is the right place to send issues affecting the whole infrastructure. It is automatically managed via GitHub Actions.

About

Reference anatomies of the brain and corresponding atlases play a central role in experimental neuroimaging workflows and are the foundation for reporting standardized results. The choice of such references —i.e., templates— and atlases is one relevant source of methodological variability across studies, which has recently been brought to attention as an important challenge to reproducibility in neuroscience. TemplateFlow is a publicly available framework for human and nonhuman brain models. The framework combines an open database with software for access, management, and vetting, allowing scientists to distribute their resources under FAIR —findable, accessible, interoperable, reusable— principles. TemplateFlow supports a multifaceted insight into brains across species, and enables multiverse analyses testing whether results generalize across standard references, scales, and in the long term, species, thereby contributing to increasing the reliability of neuroimaging results.

Please visit www.templateflow.org for a more comprehensive description of this project. News and some discussions take place at the Nipy discourse platform.

Vision

The rationale behind TemplateFlow and how we envision it as a fundamental instrument to neuroimaging studies is presented in our preprint:

Ciric R. et al., 2021. doi:10.1101/2021.02.10.430678

Acknowledgments

This work is steered and maintained by the NiPreps Community. The development of this framework is supported the NIMH (RF1MH121867, RAP, OE).

Thanks to the DataLad developers, as we rely on their wonderful tool for the management of the TemplateFlow Archive.

python-manager's Issues

Create CLI tool

With click and two subcommands: create and update.

Example create

tfmgr create --template-id MNI152NLin2050Asym ./tpl-MNI152NLin2050Asym

or

tfmgr create --template-id MNI152NLin2050Asym ./tpl-MNI152NLin2050Asym --osf-project '<some_osf_id>' --osf-username user --osf-password <secret>

Example update

tfmgr update --template-id MNI152NLin2009cAsym ./tpl-MNI152NLin2009cAsym

Create package structure

• including docs, CI/CD and python modules

OSF upload error with personal access token

tfmgr add --osf-project r6duz tpl-RESILIENT
gives the error:

ERROR:
usage: osf upload [-h] [-f] [-U] [-r] source destination
osf upload: error: You are not authorized to access this project.

$OSF_USERNAME and $OSF_PASSWORD are passed from .zshrc, and r6duz is public and I have full permissions.

$OSF_PASSWORD is a personal access token, so I think that $OSF_PASSWORD is not using this argument as intended.

GIN as alternative to OSF?

Quick note, since OSF is apparently tricky: http://gin.g-node.org/

credential error during `dl.create_sibling_github`?

My uploading process crashed during this dl.create_sibling_github command:
https://github.com/templateflow/python-manager/blob/master/tfmanager/cli.py#L260-L267

For some reason the traditional github_login option keeps causing credential error, and I need to change it to credential and use a token to fix it. I think it might be because the traditional ssh option is obsolete for github and/or datalad?

Create nightly manager

Write some code to run on circle or travis daily:

• Maintains S3 buckets in sync with the repo.
• Updates the root directory if there are changes in the subdatasets
• Updates the client's skeleton (and removes such responsibility from the client's CircleCI workflow)

Additionally, when a new release of the manager is cut:

• Cut a new release of the TF repository (https://github.com/templateflow/templateflow) by pushing a new annotated tag.
• Perhaps hit some endpoint to notify @yarikoptic's crawlers that a new release of ///templateflow is ready?

tfmgr add has no `--osf-project` parameter

The TemplateFlow contributing instructions say to use the following command to add a new template, per Step 3:

tfmgr add tpl-Name --osf-project '<project_id>'

However, in version 20.0.0a2.dev23+gf4c596a (installed from master today), tfmgr add doesn't have an --osf-project parameter.

Here's the current help message:

Usage: tfmgr add [OPTIONS] TEMPLATE_ID

  Add a new template.

Options:
  --osf-user TEXT
  --osf-password TEXT
  --osf-overwrite
  --gh-user TEXT
  --gh-token TEXT
  --path PATH
  -j, --nprocs INTEGER RANGE  [x>=1]
  --help                      Show this message and exit.

Drop six (and any leftovers of Python 2 compatibility)