Running ImageExtractor.py is not finding .dcm files in target directory about niffler

[Nitesh639]

@ceiag Go with this link.
I have a strong feeling you will get some output files. Don't change the config.json file.

[pradeeban]

The log means, the folder does not have any valid DICOM files. I know you have a file with the .dcm extension in that folder. But the file must be invalid (i.e., not a DICOM file).

Can you please try to reproduce the issue with a publicly available DICOM file such as the ones in https://www.cancerimagingarchive.net/collections/?

Then if you can still reproduce it, please point me to one such DICOM file so that I can further debug this issue.

I tried to reproduce with the below configuration with the same Niffler version as yours.

{
"DICOMHome": "/Users/Pradeeban/Desktop/op",
"OutputDirectory": "png",
"Depth": 0,
"SplitIntoChunks": 1,
"PrintImages": true,
"CommonHeadersOnly": false,
"PublicHeadersOnly": true,
"SpecificHeadersOnly": false,
"UseProcesses": 0,
"FlattenedToLevel": "patient",
"is16Bit":false,
"SendEmail": true,
"YourEmail": "[email protected]"
}

I cannot reproduce the issue. Even when a DICOM file that has some issues were used, this is the log that is thrown:

$ cat ImageExtractor.out
INFO:root:------- Values Initialization DONE -------
INFO:root:Number of dicom files: 1
DEBUG:root:Loaded the first file successfully
WARNING:pydicom:Expected sequence item with tag (fffe, e000) at file position 0x14
WARNING:pydicom:Expected sequence item with tag (fffe, e000) at file position 0x14
WARNING:pydicom:Expected sequence item with tag (fffe, e000) at file position 0x14
WARNING:pydicom:Expected sequence item with tag (fffe, e000) at file position 0x14
INFO:root:Chunk 0 Number of fields per file : 48
INFO:root:Start processing Images
ERROR:root:Unable to convert the pixel data: one of Pixel Data, Float Pixel Data or Double Float Pixel Data must be present in the dataset
ERROR:root:1 out of 1 dicom images have failed extraction
INFO:root:Chunk run time: 0.11768984794616699 seconds!
INFO:root:Generating final metadata file
INFO:root:Generating final mapping file
INFO:root:Total run time: 0.15414905548095703 seconds!
Pradeebans-MacBook-Pro:png pradeeban$ tree
.
├── ImageExtractor.out
├── ImageExtractor.pickle
├── extracted-images
├── failed-dicom
│   ├── 1
│   │   └── 1.2.276.0.28.3.194080963160286.12.3612.2018102319423250000.dcm
│   ├── 2
│   ├── 3
│   ├── 4
│   └── 5
├── mapping.csv
├── maps
│   └── mapping_0.csv
├── meta
│   └── metadata_0.csv
└── metadata.csv

9 directories, 7 files

On the other hand, if I instead use a fake DICOM file (for example, create a text file and store it as test.dcm under the folder), I get the message that you get (which simply states there is no real DICOM files present).

[Nitesh639]

ImageExtractor.py is working properly. @ceiag make sure you are use .dcm file format not .DCM format.
This thing also cause error because we are only detect .dcm files.
https://github.com/Emory-HITI/Niffler/blob/master/modules/png-extraction/ImageExtractor.py#L334

[Nitesh639]

@pradeeban, I think we should be add some examples of .dicom(.dcm format) images.

[pradeeban]

@Nitesh639 I will include the links to TCIA to download sample files. We shouldn't duplicate DICOM files in the repository. Git repositories should avoid binary files.

Also, in practice, whoever uses the Niffler PNG Extraction should have access to DICOM files (as Niffler is a domain-specific radiology software). I understand sample DICOM files will be helpful for situations such as Google Summer of Code contributors.

[Nitesh639]

OK.

[ceiag]

Thanks All for the replies.

Yep looks like something isn't right my end, even after downloading a publicly available dicom dataset from https://www.cancerimagingarchive.net/collections/ it's still not detecting .dcm files in the target directory.

[pradeeban]

Closing this issue since @Nitesh639 has given sample files to run without even changing the configurations. This should help you get started, when you are not familiar with Niffler.