edickie / ciftify Goto Github PK

add a flag into ciftify_subject_fmri and cifti_vis_fmri to drop n number of dummy scans from the front of the image...

add potential "group" mode (that runs the whole group and creates the index) into future releases

cifti vis requires workbench 1.2.3 and image magik to function properly..
When these are not loaded, it produces user-unfriendly errors.

Solution, add extra dependancy checks directly into the program..and user-friendly error reporting

Might consider doing this for all of the functions..

The current plan, which we think works,

The problem: fmriprep and ciftify space-T1w nifti images are beautifully aligned (when you open them in more viewers) but they have different underlying sforms. This means that you can't just reslice one to the other using FLIRT (ident.mat) OR concatenate the ident.mat with the ciftify MNI transform to atlas transform the functionals from FMRIPREP.

Solution, (I think) is to use nilearn's nilearn.image.resample_to_image to do the reslicing (this seems to work/not be affected by the underlying different sforms). To get a FLIRT .mat file that also does this, I register the FMRIPREP func file to the nilearn resliced func file...

Will build this as a default behaviour of ciftitify_subject_fmri...also will switch the reg target to be the first image rather than the mean image.

File "${CODEPATH}/ciftify/bin/ciftify_subject_fmri.py", line 522, in define_func_3D
cifify.meants.verify_nifti_dimensions_match(settings.ref_vol.path,
NameError: name 'cifify' is not defined

After correcting the typo, running into below issue

ciftify.meants.verify_nifti_dimensions_match(settings.ref_vol.path,
AttributeError: 'Settings' object has no attribute 'ref_vol'

Here is the command that is used

ciftify_subject_fmri --surf-reg FS --ciftify-work-dir ${PROCESSDIR} --FLIRT-to-T1w --func-ref ${DATAPATH}/fmri_map1.nii.gz fmri_mapAll.nii.gz SUBJECT mapname

Please assist if you see any issue in the usage

Thanks

Example:
"~/Documents/abide_PINT/20160908_PINT_fixa_UPSM_Long_Yeo7-80v_rad6-6-12/50007_followup_2/50007_followup_2_Yeo7-80v_rad6-6-12_ivertex_meants.csv"

I tried to convert Nifti a 4D volume to surface and I got this error:

ciftify_subject_fmri MYBASEDIR/fmri_s0025447_session3_rest.nii.gz 0025447 out
Subject HCPfolder does not exit

Any insights related to this please ?

soo I used the check_output_writable function that I wrote for checking if a file could be written. it needs to be updated to work for directory inputs as well..

./ciftify_seed_corr/sub-50005_tvertex_func_ciftitocifti_unmasked_meants.csv (should it be there?)
./ciftify_seed_corr/sub-50005_tvertex7_giftitocifti_masked_cortex.csv (correctly named??)
./ciftify_seed_corr/sub-50005_tvertex7_giftitocifti_masked_subcortical.csv
(correctly named?)

I'm thinking
fs2hcp.py -> ciftify_recon_all
func2hcp.py -> ciftify_fmri_nifti

let's just drop all the links that don't have .py at the end?

Things to add
[X ] volume calculation (i.e. cluster size)
[ ] atlas roi reading
[ X] subcortical component..

right now MSMSulc functions for both, but they don't have similar user arguments.

We are adding MSM as a dependancy.

Note this from T. Coalson:

Just a reminder: the MSM code/binary is forbidden from commercial use (similar to FSL), because of the terms of the ELC and HOCR discrete optimization libraries. Commercial interests would need to make arrangements with Waseda University (and maybe oxford, as it uses some FSL code, and I am unsure of the details of the FSL commercial restrictions).

I similar situation exists for Freesurfer and FSL.

Will need to add something to the docs about this.

Hi, when I run ciftify_recon_all script in the newest version, some errors appear. By checking the path, I find two MNINonLinear folds generated.

/root/projects/ciftify/ciftify/output/sub-50005/T1w/Native
/tmp/tmpzrV5el/T1w/native
/root/projects/ciftify/ciftify/output/sub-50005/MNINonLinear/Native
/tmp/tmpzrV5el/MNINonLinear/native
_/root/projects/ciftify/ciftify/output/sub-50005/MNINonLinear_

Traceback (most recent call last):
File "/usr/local/bin/ciftify_recon_all", line 11, in
load_entry_point('ciftify==2.0.2', 'console_scripts', 'ciftify_recon_all')()
File "/usr/local/lib/python2.7/site-packages/ciftify-2.0.2-py2.7.egg/ciftify/bin/ciftify_recon_all.py", line 1436, in main
run_ciftify_recon_all(tmpdir, settings)
File "/usr/local/lib/python2.7/site-packages/ciftify-2.0.2-py2.7.egg/ciftify/bin/ciftify_recon_all.py", line 106, in run_ciftify_recon_all
os.path.join(subject.atlas_space_dir, 'Results'))
File "/usr/local/lib/python2.7/site-packages/ciftify-2.0.2-py2.7.egg/ciftify/bin/ciftify_recon_all.py", line 400, in create_output_directories
ciftify.utils.make_dir(mesh['Folder'], DRYRUN, suppress_exists_error = True)
File "/usr/local/lib/python2.7/site-packages/ciftify-2.0.2-py2.7.egg/ciftify/utils.py", line 73, in make_dir
os.makedirs(dir_name)
File "/usr/local/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 17] File exists: '/root/projects/ciftify/ciftify/output/sub-50005/MNINonLinear'

=================================================================================

I find some clues in ciftify_recon_all.py in line 398, create_output_directories function. The meshes.values include the following information:
{'ROI': 'roi', 'meshname': 'native', 'T2wImage': '/root/projects/ciftify/ciftify/output/sub-50005/T1w/T2w.nii.gz', 'DenseMapsFolder': '/root/projects/ciftify/ciftify/output/sub-50005/MNINonLinear/Native', 'T1wImage': '/root/projects/ciftify/ciftify/output/sub-50005/T1w/T1w.nii.gz', 'Folder': '/root/projects/ciftify/ciftify/output/sub-50005/T1w/Native', 'tmpdir': '/tmp/tmps9u7P2/T1w/native'}

{'ROI': 'roi', 'meshname': 'native', 'T2wImage': '/root/projects/ciftify/ciftify/output/sub-50005/MNINonLinear/T2w.nii.gz', 'T1wImage': '/root/projects/ciftify/ciftify/output/sub-50005/MNINonLinear/T1w.nii.gz', 'Folder': '/root/projects/ciftify/ciftify/output/sub-50005/MNINonLinear/Native', 'tmpdir': '/tmp/tmps9u7P2/MNINonLinear/native'}

{'ROI': 'atlasroi', 'meshname': '164k_fs_LR', 'T2wImage': '/root/projects/ciftify/ciftify/output/sub-50005/MNINonLinear/T2w.nii.gz', 'T1wImage': '/root/projects/ciftify/ciftify/output/sub-50005/MNINonLinear/T1w.nii.gz', 'Folder': '/root/projects/ciftify/ciftify/output/sub-50005/MNINonLinear', 'tmpdir': '/tmp/tmps9u7P2/164k_fs_LR'}

No more than 4mm. Maybe change the way that rois are defined (use bigger one's) to achieve the same effects..

the index page views for cifti_vis_recon_all and cifti_vis_fmri don't have any images..even though the main index is still linking to all the individual subject pages..

This probably needs to happen before we do the validation experiments in corr

I might be that when we tune the parameters..the number of needed iterations might change..

This really just mean moving all the print statements in cifitfy-PINT-veritices into a logger and logging to a file pint.log?

Please specify that the argument should contain the path of the output folder and the subject ID. E.g. /path/to/output/SubjID

I preprocessed my data using fmriprep. I'd like to use the data that's transformed to the MNI Asymmetric template and been denoised with AROMA (e.g., *bold_space-MNI152NLin2009cAsym_variant-smoothAROMAnonaggr_preproc.nii.gz). For ciftify_subject_fmri I used the --already-in-MNI flag, but would I need to do anything differently in terms of processing the T1w? I'm assuming not since the ciftify_recon_all step just uses the freesurfer outputs, but I wanted to check.

Thanks!

We got some worry that the --hcp-data-dir flag is not well named (it's not acutally data from the HCP) so we will rename it (and the enviroment variable) and add deprecation warnings to the current version

-deprecation warning if the flag is used
-deprecation warning in the help docs
-help docs will switch to using new flag

New flag is:
--ciftify-working-dir?
--ciftify-data-dir?

So ...when I refactored the filenames.py ness (about a year ago) I screwed this up (it's using the wrong input mesh.

Need to fix

In ciftify_clean_img

logger.error('Indicated colBIDS_ZHH/sub-10186/ses-02/func/sub-10186_ses-02_task-rest_bold.nii.gzumn {} not in confounds'.format(colname_arg))

Don't imaging all that ZHH bit is supposed to be coded in there : )

line 169 says : run(['fslmaths', inputfMRI, '-Tmean', func3D])
the i in inputfMRI needs to be capitalized

In reponse to valid concerns that mapping to the S900 average surfaces is, most of the time, not the best decision. We are doing to change the usage so that this is not an easy thing to do..

also ciffity_a_nifti will get renamed ciffity_nifti_result to emplasize to the user that the intended usage is for the presentation of result from a previous volume smoothed analysis..

Adopt the bootstrap template used in vis-Pint to all projects

Is it available to map non-HCP DTI data to CIFTI ?

Some variables in the tsv from fMRIPREP have a n/a/ value, and clean won't accept them. Change to zero first : )

add functinality to use nilearn cleaning for filtering and confound regression of cifti files (and signals)

Note: we need to build in functionality that it smarter for memory usage (i.e. doesn't try to clean the while image as once if the image is too big..)

There is some weird (memory leak?) thing happening with wb_command -cifti-smoothing where everything freezes if you try to smooth more than one image in parallel on the same machine... Will switch to separating and smoothing the three parts (instead of the cifti file) to see if this goes away.

looks like I forgot to update that code to use pvertex instead...urg

I currently makes a bunch of python error madness.. not very nice for the user

From Tim Coalson

The func2hcp.py script uses the -volume-parcel-resampling-generic command that we took out of our pipelines, in order to instead do cubic spline resampling, resolving an issue of it creating a grid pattern in the noise variance (since this command uses a form of linear resampling). We now use -cifti-resample after making a subject-specific subcortical cifti file at the acquired resolution. See here:

https://github.com/Washington-University/Pipelines/blob/master/fMRISurface/scripts/SubcorticalProcessing.sh

This is a QC scene for the HCP Pipelines

• integrate the views (especially MSMSulc & Myelin related ones)
• retest that the scene export works..as a func and other??
  + default location for the scene should be within the subjects hcp folder? i.e. same level as T1w and MNINonlinear folders..

Print a friendly error message indicating that the user might have forgotten to indicate the '--weighted' or '-roi-label' file

Hi dear friend, I cannot find the newest version of ciftify on the homepage of https://github.com/edickie/ciftify/wiki

the available pip setup code is still for v1.0.1. Please let me know

Thank u!

The ciftify_recon_all.log ciftify_subject_fmri.log files aren't don't contain the "Running: cmd" sections..

They do contains the section headers, which are also at the info level...and error messages... so maybe the run function is internally resetting the log level??

maybe throw x,y,z mapping into an internal function so that it can be used for forward inference later on..

citify_subject_fmri with --FLIRT-to-T1w as below is giving usage error

ciftify_subject_fmri --hcp-data-dir ${HCP_DIR} --FLIRT-to-T1w func.nii.gz subjid mapname

Error: Usage:
ciftify_subject_fmri [options] <func.nii.gz>

Its running fine without --FLIRT-to-T1w option as below,

ciftify_subject_fmri --hcp-data-dir ${HCP_DIR} func.nii.gz subjid mapname

Can you please suggest if you see any issues

First of all, thank a lot to share your work.
I wish to run ciftify in a docker so I cloned from github the project ciftify, build the Dockerfile and ran it....
But I do not understand how that can work ....because It seems to me that the dockerfile open only a jupyter notebook and thats it ...
I think I have missed something....
Could you please say a word on that ?

I think there might be a bug in the latest update. I tried installed the update with pip but there was an error. (I replicated the problem on another computer with Python 3.6.5)

Successfully built ciftify
Installing collected packages: ciftify
Exception:
Traceback (most recent call last):
File "/apps/psyc/anaconda3/python35_520/lib/python3.6/site-packages/pip/_internal/basecommand.py", line 228, in main
status = self.run(options, args)
File "/apps/psyc/anaconda3/python35_520/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 335, in run
use_user_site=options.use_user_site,
File "/apps/psyc/anaconda3/python35_520/lib/python3.6/site-packages/pip/_internal/req/init.py", line 49, in install_given_reqs
**kwargs
File "/apps/psyc/anaconda3/python35_520/lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 748, in install
use_user_site=use_user_site, pycompile=pycompile,
File "/apps/psyc/anaconda3/python35_520/lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 961, in move_wheel_files
warn_script_location=warn_script_location,
File "/apps/psyc/anaconda3/python35_520/lib/python3.6/site-packages/pip/_internal/wheel.py", line 327, in move_wheel_files
console, gui = get_entrypoints(ep_file)
File "/apps/psyc/anaconda3/python35_520/lib/python3.6/site-packages/pip/_internal/wheel.py", line 143, in get_entrypoints
console = dict(_split_ep(v) for v in console.values())
ValueError: dictionary update sequence element #4 has length 3; 2 is required

So I did a manual install and still have an issue running commands:

[joseph.orr@gfx bin$ ciftify_subject_fmri --SmoothingFWHM 2 --ciftify-work-dir $outdir --already-in-MNI $func $subject $label
Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
File "/apps/psyc/ciftify/2.0.3/ciftify/io.py", line 9, in
File "/apps/psyc/anaconda3/python35_520/lib/python3.6/logging/init.py", line 26, in
File "/apps/psyc/anaconda3/python35_520/lib/python3.6/traceback.py", line 5, in
File "/apps/psyc/anaconda3/python35_520/lib/python3.6/linecache.py", line 11, in
File "/apps/psyc/anaconda3/python35_520/lib/python3.6/tokenize.py", line 27, in
ImportError: cannot import name 'open'
Aborted (core dumped)

Some systems can't work with the sym-links, so we need user options that are ok with getting rid of these