Potential TensorFlow issue about raidionics

[Ricatu12]

Good afternoon. I'm a new user of both software and yes I got this error message: Failed to load the native TensorFlow runtime. I'm using a windows 10 operative system on a normal HP laptop.

[andreped]

@Ricatu12 I remember you contacted me on e-mail.

Hopefully, you will be able to test the software on a different computer, such that you are at least able to run your experiments.

Until then, I will be trying to fix this bug relevant for the machine you tested. However, it is challenging as I am unable to reproduce this bug. Will let you know if I have a new version for you to test.

[Ricatu12]

Incredible! You cracked the code! That's time to get a new laptop then 😆 Thank you a lot! PS. Is there a way to get avx support? Il giorno mar 3 mag 2022 alle ore 19:19 André Pedersen < ***@***.***> ha scritto:

…

Found the problem! Your CPU (Celeron(R) N4020 CPU @ 1.10GHz) does not support AVX. If you go here <https://versus.com/en/intel-celeron-n4020> and scroll down, you will see this: [image: Skjermbilde] <https://user-images.githubusercontent.com/29090665/166504643-5170fb05-15a3-40d2-b0e5-0c14f960a6b9.PNG> This issue was talked about in this thread: pyinstaller/pyinstaller#3754 (comment) <pyinstaller/pyinstaller#3754 (comment)> Solution: What we need to do is to use a TensorFlow version that is built without AVX support. Not sure how that will affect runtime and whatnot, but I could test this. I will make an attempt and let you know when a new nightly release is ready for testing. The reason why you got this issue and why most don't is because most modern CPUs come with AVX support. However, *some older CPUs as well as low-end CPUs don't support AVX*. — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AY73DXY2QO2VOPKSBH5AOW3VIFNZPANCNFSM5U6BF2DQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Ricatu12]

Oh nice!!!! That would be great!! Il mer 12 ott 2022, 17:24 André Pedersen ***@***.***> ha scritto:

…

No I need the segmentations for my study. OK, great! Because we already have that working on our end - also on M1 chip. We just need to build and test that the software works on M1, which we cannot do until we have a machine with M1 chip. Might get one tomorrow. If that works, I will let you know, and I will give you the link to download a test version for you to try. Then hopefully it all works out :] — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AY73DXZ3MUPT4OK4IA76MO3WC3J3FANCNFSM5U6BF2DQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Ricatu12]

Keep me updated Il gio 13 ott 2022, 20:02 André Pedersen ***@***.***> ha scritto:

…

@Ricatu12 <https://github.com/Ricatu12> We just bought and setup a Mac Mini with M1 chip. Hence, we should be able to build and test if Raidionics builds and runs well on M1 tomorrow. If all goes well, we will have a version for you to test! Fingers crossed :] — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AY73DX2MKNLLPXZR4GHV4MLWDBFEPANCNFSM5U6BF2DQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[andreped]

Potential fix in recent commit 0773a3b.

See Actions for corresponding binary release(s) stored as Artifact (NOTE: Need to be logged in to both see and download the Artifact "Package", which contains the installers for all three operating systems).

[andreped]

Found the problem!

Your CPU (Celeron(R) N4020 CPU @ 1.10GHz) does not support AVX. If you go here and scroll down, you will see this:

This issue was talked about in this thread:
pyinstaller/pyinstaller#3754 (comment)

Solution: What we need to do is to use a TensorFlow version that is built without AVX support. Not sure how that will affect runtime and whatnot, but I could test this.

I will make an attempt and let you know when a new nightly release is ready for testing.

The reason why you got this issue and why most don't is because most modern CPUs come with AVX support. However, some older CPUs as well as low-end CPUs don't support AVX.

[andreped]

Only way to get AVX support is to use a machine with a CPU that has AVX support.

However, I have attempted to fix this in Raidionics, by swapping the prebuilt TensorFlow wheel, which is built with AVX support by default, with a different wheel built with SSE (which more CPUs should support - at least your's).

I have pushed a potential fix for this, quite the quick fix, just for you to test: ab6aa6d

You can download the new nightly release from here (click on "Package"):
https://github.com/dbouget/Raidionics/actions/runs/2265342438

Please, let me know how it goes. If you get the same issue, I think I know what to do.

[Ricatu12]

Good morning. This time I got a different message error. Still not able to run the diagnosis thou. The message errore is : WARNING:tensorflow:From tensorflow\python\ops\resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer. WARNING:tensorflow:From tensorflow\python\keras\layers\core.py:143: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version. Instructions for updating: Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`. Thanks for helping Erica Il giorno mar 3 mag 2022 alle ore 20:50 André Pedersen < ***@***.***> ha scritto:

…

Only way to get AVX support is to use a machine with a *CPU* that has AVX support. However, I have attempted to fix this in Raidionics, by swapping the prebuilt TensorFlow wheel, which is built with AVX support by default, with a different wheel built with SSE (which more CPUs should support - at least your's). I have pushed a potential fix for this, quite the quick fix, just for you to test: ab6aa6d <ab6aa6d> You can download the new nightly release from here (click on "Package"): https://github.com/dbouget/Raidionics/actions/runs/2265342438 Please, let me know how it goes. If you get the same issue, I think I know what to do. — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AY73DX3NY3XZDXLBKO3FKWTVIFYQLANCNFSM5U6BF2DQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[andreped]

Then the fix I did worked.

What you are seeing here is not really an error, but a warning. Should still run.
I assume the program just crashes, without any prompt?
When you run the software, you should be given some logging info. Stating where it is in the process. Can you tell me where it crashed?

Alternatively we could do a zoom meeting where you show me your screen, to debug this further.

[Ricatu12]

-pred_Brain.nii.gz <https://drive.google.com/file/d/1C2WBZe09kryHxalTntC7eUuXPex9xNJ_/view?usp=drive_web> -pred_Tumor.nii.gz <https://drive.google.com/file/d/1xAWhY5yvXxW1Z3DA3EW6KfDU2Z-gbQzz/view?usp=drive_web> converted_input.nii.gz <https://drive.google.com/file/d/1zQixZhcxRCEXjYABmqPkRviKo6vNk831/view?usp=drive_web> converted_input_masked.nii.gz <https://drive.google.com/file/d/1oksYE0_6bmc_vt7b2XJ8K_CavDk0sXcV/view?usp=drive_web> mni_icbm152_t1_tal_nlin_sym_09a_masked.nii.gz <https://drive.google.com/file/d/1nMRAlhYC9WM3TJ6nH0chu327DIq3AZtk/view?usp=drive_web> The problem is that the results of my processing are generated as file I can't read. I do have no reports. The program doesn't crash. :) indeed I think it works in batch mode! In the file attached the folder the program is generating. Il giorno gio 5 mag 2022 alle ore 12:23 André Pedersen < ***@***.***> ha scritto:

…

Then the fix I did worked. What you are seeing here is not really an error, but a warning. Should still run. I assume the program just crashes, without any prompt? When you run the software, you should be given some logging info. Stating where it is in the process. Can you tell me where it crashed? Alternatively we could do a zoom meeting where you show me your screen, to debug this further. — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AY73DX4M7OJUAXW5RQKB5ATVIOOQXANCNFSM5U6BF2DQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[andreped]

OK, but then the software works as it should. Great!

The *.nii.gz files are NIFTI files (segmentation volumes) that can be opened in 3D Slicer or similar. Are you familiar with 3D Slicer? There should also be a report generated that is stored on disk.

When you run diagnosis, it creates a timestamped directory and saved the result in where you chose the output directory to be, for instance in my case the results are stored in E:\Downloads\05052022_131023 given that I set the output directory to E:\Downloads.

In this directory, when I run diagnosis (tested it for a single patient), it looks like this:

But you did not get a report.txt file? Try running diagnosis in single-mode and report what you get.

[Ricatu12]

So even in a single mode I don't get any report.txt file but just the *.nii.gz, the other folders are just empty. I need to get an other machine, I'm sure. Il giorno gio 5 mag 2022 alle ore 13:25 André Pedersen < ***@***.***> ha scritto:

…

OK, but then the software works as it should. Great! The *.nii.gz files are NIFTI files (segmentation volumes) that can be opened in 3D Slicer or similar. Are you familiar with 3D Slicer? There should also be a report generated that is stored on disk. When you run diagnosis, it creates a timestamped directory and saved the result in where you chose the output directory to be, for instance in my case the results are stored in E:\Downloads\05052022_131023 given that I set the output directory to E:\Downloads. In this directory, when I run diagnosis (tested it for a single patient), it looks like this: [image: Skjermbilde 2022-05-05 132337] <https://user-images.githubusercontent.com/29090665/166913548-30a4dbb6-8248-4b06-bbbe-e344ebfca0a9.png> But you did not get a report.txt file? Try running diagnosis in single-mode and report what you get. — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AY73DX3VVNUPYJZZDKOWAULVIOV2FANCNFSM5U6BF2DQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[andreped]

I think I know the issue, and it is not easy to solve on our end.

Essentially, Raidionics likely depends on some libraries that are built with AVX support. I assumed it was only TensorFlow, but that was rather naive. Nothing else is returned because the algorithm fails at some place. The simplest way to debug this is to run Raidionics from the terminal. That way you will get prompted what went wrong.

In the future we will add a more suitable logging solution which we will prompt what went wrong. Right now I believe nothing i prompted.

But yeah, seems like we are unable to support CPUs that don't have AVX support at the moment.

I would suggest you to run your experiments on a different machine. Remember to use the latest release then (and not the nightly). Release is available from here:
https://github.com/dbouget/Raidionics/releases

Closing for now.

[Ricatu12]

Thank you! Il giorno gio 5 mag 2022 alle ore 15:32 André Pedersen < ***@***.***> ha scritto:

…

Closed #4 <#4>. — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AY73DXYWV2YHH55CHEEHPTTVIPEYTANCNFSM5U6BF2DQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[andreped]

@Ricatu12 were you successful in using Raidionics and doing your study on a separate computer?

[Ricatu12]

Hi! Unfortunately it's not working on all the laptops I have. I put in stand by Il mar 11 ott 2022, 09:39 André Pedersen ***@***.***> ha scritto:

…

@Ricatu12 <https://github.com/Ricatu12> were you successful in using Raidionics and doing your study on a separate computer? — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AY73DX5A2MNZC26N7MJTI6LWCUKTHANCNFSM5U6BF2DQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[andreped]

Hi! Unfortunately it's not working on all the laptops I have. I put in stand by Il mar 11 ott 2022, 09:39 André Pedersen @.***> ha scritto:
…

@Ricatu12 Oh, that's sad to hear. Is it only segmentation you need? You don't need the reporting? Because we might have a version which can perform segmentation of M1 Macs very soon. At least it works on our initial trials.

[Ricatu12]

No I need the segmentations for my study. Thank you a lot Il mer 12 ott 2022, 08:04 André Pedersen ***@***.***> ha scritto:

…

Hi! Unfortunately it's not working on all the laptops I have. I put in stand by Il mar 11 ott 2022, 09:39 André Pedersen *@*.***> ha scritto: … <#m_4886391385448709677_> @Ricatu12 <https://github.com/Ricatu12> Oh, that's sad to hear. Is it only segmentation you need? You don't need the reporting? Because we might have a version for you to try soon. — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AY73DX2BOIRCX7DGMAW3LZLWCZIG7ANCNFSM5U6BF2DQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[andreped]

No I need the segmentations for my study.

@Ricatu12 OK, great! Because we already have that working on our end - also on M1 chip. The TensorFlow issue was solved by using a different inference engine which is compatible on M1 chips.

We just need to build and test that the software works on M1, which we cannot do until we have a machine with M1 chip. Might get one tomorrow.

If that works, I will let you know, and I will give you the link to download a test version for you to try. Then hopefully it all works out :]

[andreped]

@Ricatu12 We just bought and setup a Mac Mini with M1 chip. Hence, we should be able to build and test if Raidionics builds and runs well on M1 tomorrow. If all goes well, we will have a version for you to test! Fingers crossed :]

[dbouget]

Hi @Ricatu12,

We have a version for you to try (can be downloaded here). On our M1 machine we are now able to perform segmentation.
I would recommend you to uninstall the previous version, and manually delete the .raidionics folder located inside your home path (something like /Users/<user_account>/.raidionics)

Haven't had the time to make any additional tests, so some GUI parts might be clunky, and the reporting does not work at all.
The fastest way to see if it works is to do the following when starting Raidionics:

1. Click File > Download test data, and select where to save the test data.
2. Click Single patient
3. Click Import patient (in the bottom-left corner), then Other data type (*.nii), then select the two files inside ApprovedExample/Raw (located where you chose to save the test data during the first step).
4. For the Case27-T1 MRI Series (in the right-hand panel), select T1-CE as Sequence type, then save the patient by clicking the floppy disk icon (left-hand side panel, to the right side of the automatically generated patient name).
5. Click the Actions tab in the right-hand panel, then click Preoperative segmentation and choose the tumor type.
6. When the process has finished, click the Data tab (right-hand side panel), select the T1 MRI, and then click on Annotations, where you should see two elements (one brain segmentation and one tumor segmentation).

Let me know if it works (or not), or if you need any extra help.

[andreped]

@Ricatu12 Just updating you to let you know that we have made a pre-release now, which should work on your macbook with M1 chip:
https://github.com/dbouget/Raidionics/releases/tag/v1.2.0-alpha

See dbouget's reply for how to get started using it. Then let us know how it goes :]