FileNotFoundErrors: debug_resampled.tif & annotation_25_1-246Ld.nrrd about clearmap2

[SaibotMagd]

I just saw this strange problem in the cellmap pipeline. When you use the command ws.debug = True, all future ws.filename are augmented by an "debug_"+X. But you didn't calc the resample.tif or resample_autofluroescence.tif or all the alignment parameters using the "debug_" filenames. On the other hand you need the whole parameter set to do a proper alignment.

I solved this issue this way:

• use the whole dataset for all the alignment, stitching parts
• create debug subset (...slicing = (slice(...) ... ws.create_debug('stitched', slicing=slicing))
• rename subset "debug_stitched" to only "stitched.npy" (don't overwrite the full stitched.npy; i saved it as old_stitched.npy)
• deactivate debug modus by calling ws.debug = False
  ...and go on with the pipeline

pretty annoying but possible.

[saskra]

Tanks a lot! Still, now I get a new error message:

transformix has finished at Wed Aug  5 15:24:17 2020.
Total time elapsed: 12.0s.
Traceback (most recent call last):
  File "/snap/pycharm-professional/211/plugins/python/helpers/pydev/pydevd.py", line 1448, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/snap/pycharm-professional/211/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/Scripts/TubeMap.py", line 507, in <module>
    edge_geometry_properties = {'coordinates_atlas' : 'annotation'});
  File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/Analysis/Graphs/GraphGt.py", line 1413, in annotate_properties
    edge_geometry_properties=edge_geometry_properties);
  File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/Analysis/Graphs/GraphGt.py", line 1366, in transform_properties
    values = transformation(values);
  File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/Scripts/TubeMap.py", line 502, in annotation
    label = ano.label_points(coordinates, key='order');
  File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/Alignment/Annotation.py", line 445, in label_points
    atlas = io.read(annotation_file);   
  File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/IO/IO.py", line 409, in read
    return mod.read(source, *args, **kwargs);
  File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/IO/NRRD.py", line 297, in read
    return source.array
  File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/IO/NRRD.py", line 70, in array
    return _array(self.location);
  File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/IO/NRRD.py", line 600, in _array
    with open(filename,'rb') as filehandle:
FileNotFoundError: [Errno 2] Datei oder Verzeichnis nicht gefunden: '/home/nicolas.renier/Documents/ClearMap_Ressources/annotation_25_1-246Ld.nrrd'
python-BaseException 

While it is obvious that this hard-coded path does not exist on my machine, I do not know with which one to replace. Is this a file I should have downloaded or created?

[SaibotMagd]

It's always better to open a new issue than to modify an old one, because your solution can help others so they can find it.

@question:
I don't work on the tubemap pipeline, but I found a zip archive containing additional files: https://idisco.info/clearmap-2/ ... the File you need should be: "annotation_25_full_nrrd" and then you have to cut it like you did with the reference file.

It's just a guess I didn't try this at all.

[saskra]

I found a zip archive containing additional files: https://idisco.info/clearmap-2/ ... the File you need should be: "annotation_25_full_nrrd" and then you have to cut it like you did with the reference file.

Thanks again, I was not aware of this additional source of files! Unfortunately, this does not look like the brain I have downloaded from here: https://osf.io/sa3x8/files/?view_only=4427a838cbd0468c9fbad9cab465d866

[SaibotMagd]

Thanks again, I was not aware of this additional source of files! Unfortunately, this does not look like the brain I have downloaded from here: https://osf.io/sa3x8/files/?view_only=4427a838cbd0468c9fbad9cab465d866

I don't get it. The annotation file is a part of the atlas (it's based on MRI). The osf.io files are actually LSM Data, so it shouldn't be the same.

[saskra]

Then I do not really understand what this file is, why I need it and how those two should go together - how shall I crop the MRI like the LSM when they have completely different resolution, orientation etc.? And I could not use this pipeline on our own data, if I needed MRI images of the same (?) brain as well. Or am I getting something completely wrong here?

[ChristophKirst]

The reference file is needed if you want to align the data to an atlas (in this case the Allan btrain Atlas).
If you only like to process your own data without atlas annotation you can skip all the steps that involved this file and the alignment.

If you work on graphs you should also skip the steps that transform the coordinates to the refernece etc.

Does this make sense?

In general, I see that some of the questions here arise by assuming the pipeline will work out of the box.
In fact, we provide the most complete pipeline that needs to be modified for your specific purposes, including removing certain steps that are not needed.

If you working on small data sets that are part of a larger one and like to use the atlas annotation, the pipeline also might needs tweaking to make this work with alignment of the full data.

Hope this helps ?

Feel free to reopen if this does not solve the issue