Nyul normalize not working with custom output range about intensity-normalization

[jcreinhold]

Thanks for the feedback. Can you provide your python code?

[jcreinhold]

I suspect you're using the wrong arguments. I updated the package to improve the names of the parameters. See here for a description of the parameters.

I just tested the package changing output_min_value and output_max_value to -2000 and 2000, respectively, and it worked. Please reinstall the package and use the updated parameters and reopen/comment on this issue if you still have a problem.

[jcreinhold]

Related #37

[MLRadfys]

Hi and thanks for your help regarding my issue.

I tested your implementation again and it works great for 2D images. What I tried was to match a two 3D volumes.
When I do this, the output range gets messed up.

Cheers,

Michael

[jcreinhold]

Hi @MLRadfys, I can't reproduce your error. Can you provide the command you ran/snippet of code that's causing the problem. Also provide the version number of intensity-normalization (add --version as an argument to any command line script), and any other relevant info.

If you can share the files you're trying to normalize (e.g., on Google Drive), that would be helpful.

Also, what exactly does it mean that the "range gets messed up". Can you provide the histogram plot (output with -p option in the CLI or with the HistogramPlotter in the python API) and tell me what you expect to happen?

[MLRadfys]

Hi again,

alright, Iam using version 2.1.0 of yor package, so that should be the latest one.
Here is the code Iam using:

cbct = nib.load('...')
cbct = nib.load('...')
ct =nib.load('...')

from intensity_normalization.normalize.nyul import NyulNormalize
from intensity_normalization.plot.histogram import plot_histogram

nyul_normalizer = NyulNormalize(output_max_value=cbct.max(), output_min_value=cbct.min())
nyul_normalizer.fit([cbct])
normalized = nyul_normalizer(ct)
print('ct --> ', np.min(ct), np.max(ct))
print('cbct --> ', np.min(cbct), np.max(cbct))
print('matched --> ', np.min(normalized), np.max(normalized))

Here I expected the output range of the matched volume to be within -1000 and 580.

And here is the histogram I get:

Unfortunately Iam not able to share the scans but hopefully the above information helps...

Best regards,

Michael

[jcreinhold]

Thanks for the extra info.

If you want to rigidly set the min and max value to -1000 and 580, you'll need to set the min_percentile to 0 and max_percentile to 100 or clip the resulting normalized array. For reference, the docs state that output_min_value is mapped to min_percentile and similarly for output_max_value.

Setting the min and max percentiles to be at 0 and 100 isn't a good idea though because the mapping will be sensitive to outliers, so the normalization across datasets won't work well which is the point of the method. A discussion about this implementation choice, with references, can be found here.

The correct check of the algorithm would be to compare the min_percentile of the normalized output to the set output_min_percentile and similarly for the max_percentile. If you do this check and the values aren't very close to the desired values, then re-open the issue.

FWIW, I recommend two options in your scenario: 1) You can lower the min_percentile to 0.1 and raise the max_percentile to 99.9; 2) you can clip the normalized image to -1000 and 580. Or a combination or 1) and 2).

Hope that helps.

[MLRadfys]

Hi and thanks again,

alright, I tested your suggestion and the range stilll ended up to be wrong.

Nevertheless, my images contain negative HU values.
When I shift all HU values to be positive, evertything seems to work as expected!

Cheers,

M

[jcreinhold]

Ah, I see. I forgot to mention that you should provide a foreground mask of the object you're trying to normalize when you have negative values.

For anatomical MRIs, which this package is built for, there aren't negative values (at least in the image that comes off the scanner). If you don't pass in a mask as input, the method will assume that the image is skull-stripped and estimate the foreground with positive values.

When you passed in your CT image, did you not get a warning saying "Data contains negative values; skull-stripped functionality assumes the foreground is all positive. Provide the brain mask if otherwise."?

[jcreinhold]

If intensity-normalization didn't warn you, re-install the latest version of the package (v2.1.1) and try again. Let me know if you still don't see the warning.

[MLRadfys]

Hi and thanks again for all help.

I now tried the latest version which gives me the negative value warning.

I will try to input a mask as well and get back to you asap.

Cheers,

M