kdestasio / bidsqc

Scripts to batch convert dicoms to BIDS in parallel on an HPC environment, followed by a file check

BidsQC

Overview

The scripts in this repo are a combination of wrappers and quality checking scripts that take neuroimaging files from DICOMS to Niftis named and structured as per BIDS. They add functionality to the Dcm2Bids scripts that allow the user to:

1. Convert dicoms for all or only a subset of participants in the dicom directory (/conversion)
2. Run the conversion on Talapas, the University of Oregon's high performance cluster (/conversion)
3. Check that the expected files exist for each participant and notify the user if there is a discrepency (/qualityCheck)
4. Rename task runs, retaining those that occured latest in time (/qualityCheck)

The purpose of these scripts is to allow the user to run batches of specific subjects on a high performance cluster -- or -- to run the subjects serially on a local machine. They also provide some functionality to check that the resultant data contains the expected runs.

bidsQC Table of Contents

• Using bidsQC: Overview
  • Dependencies
• Repository Contents
  • Conversion Folder Overview
    • Running dcm2bids
    • On a Linux Cluster
    • Locally
  • Bids Quality Check Folder
    • Running qualityCheck
• Final BIDS Steps
• Detailed tutorial: University of Oregon specific

Using BidsQC

1. Using either git clone or the download button, copy this repo to the machine you will use for dicom conversion. This can be your local machine or a high performance cluster such as the UO's Talapas.
2. Convert your DICOMS to Nifti files that adhere to BIDS. To do so, you can:
   1. Convert all DICOMS in a specified directory. Simply follow the instructions on the Dcm2Bids tutorial. The tutorial is comprehensive and should have all of the information you need.
   2. Or you can convert only a subset of the participant data, or run the conversion in a cluster environment such as Talapas. To do so, see the instructions and scripts in the conversion folder of this repo.
3. Check that the results of the conversion and renaming are as expected. You can use the files in the qualityCheck folder of this repo to help you do so.
4. Verify your data are in BIDS using the BIDS Validator

Dependencies

• Python 3 with the future module (pip install of cbedetti's dcm2bids will install the future module)
• dcm2niix conversion tool. If it is not already available on your machine or cluster, install it.
  • Get via the rordenlab github
• dcm2Bids If it is not already available on your machine or cluster, install it.
  • bidsQC is confirmed compatible with dcm2bids v2.1.5.
  • Available from UNFmontreal for your local machine.
  • Here are instructions for getting a singularity image for use on a cluster. You will need to module load singularity prior to running the singularity command.

Note: to check if Python 3, dcm2niix, or other packages are available on a Linux cluster, type module avail [packageName] at the command line, e.g. module avail Python3

Repo Contents

Conversion Folder

The scripts located in the conversion folder allow the user to convert dicoms for only a subset of participants in the dicom directory and to run the conversion on Talapas, the University of Oregon's high performance cluster. They build off of Dcm2Bids, which on its own will allow you to convert the DICOM files to Nifti files en masse. See the Dcm2Bids tutorial to learn how to do the basic conversion and for detailed instructions on how to create the configuration file specific to your study's dicoms.

For instructions on how to use the conversion scripts, see the conversion folder README.

qualityCheck Scripts

Once Niftis are in BIDS, these scripts can be used to check whether each sequence has the correct number of runs. Files in the target directories are checked against values in the configuration file where the user specifies how many runs of each sequence are expected at each time point. Optional: append the run-## key-value string to file names based on sequence order (option specifed in config file).

For step-by-step instructions and a description of the naming rules, see the qualityCheck README.

Final BIDS Steps

Manually Create Metadata Files

As per: http://bids.neuroimaging.io/bids_spec1.0.0-rc2.pdf

Place these files in the top level bids directory.

• dataset_description.json
• phasediff.json
• A JSON for each functional task with TaskName and RepetitionTime
• README (optional, but strongly recommended)
• CHANGES (optional, but strongly recommended)

Check the BIDS Conversion

BIDS validator: http://incf.github.io/bids-validator

bidsqc's Issues

Suggested feature: run bidsQC in parallel

To avoid the script breaking on any one subject, it'd be great if the script ran in parallel via batch submission.

Link to cbedetti's tutorial is dead

New link should be https://unfmontreal.github.io/Dcm2Bids/tutorial/.

Field maps assigned incorrect run number

May be a project specific issue. Will examine further.

Suggested feature: print series number

To facilitate quality control, it'd be great to print the series number along and the renamed run number to the output log for the bidsQC.py script.

Improve readability of qualitycheck output

The current output of the qualitycheck.py script is lengthy and difficult to use. It is split into output and error logs, but the content of each is not intuitive. Refine the output so it is more user friendly.

add series name info to output files

would it be possible to add info in the bids conversion output files to reflect the series name of the files being copied over? for example, if there are 3 functional runs, and only two make it into pretty bids format, the output might say "error: wrong number of files; largest file copied", but it would be nice to know the original name of the file too.

Add option to omit ses- tag in file names for single session data

The dcm2bids_batch.py script currently requires a session number in the third column of the subject_list.txt file. That number is used to create the ses-label portion of file names when converted into BIDS. However, the BIDS spec does not expect a ses- tag in the file name for data collected within a single session. It would make sense to have an option that can select for single session data or multiple session data and that names the files accordingly with or without the ses- tag.

Here is the error if a user tries to omit session information from the subject_list.txt file

Traceback (most recent call last):
  File "dcm2bids_batch.py", line 105, in <module>
    main()
  File "dcm2bids_batch.py", line 17, in main
    batch_jobs(cfg.subjectlist, cfg.path_dicoms, cfg.path_config, cfg.path_bidsdata)
  File "dcm2bids_batch.py", line 27, in batch_jobs
    wave = entry.split(",")[2]
IndexError: list index out of range

Make moving files an explicit option

Currently, the qualitycheck.py script will move files that do not match the template established in the file qualitycheck_config.py to the temporary directory specified in the configuration file. Moving files should be made an explicit option in the config file just as renaming files is an explicit option.

The default behavior of the qualitycheck.py script should be set to produce a log that describes the dataset, but does nothing to the files themselves. Explicit options can then be set to move or rename files based on whether they adhere to the config file specifications.

File names incorrect if bidsQC is run multiple times

If bidsQC.py is run on the same subjects multiple times, the run-## string is duplicated each time.