some debugging code is causing BRAINS in Slicer to fail about brainstools

[pieper]

Some more info from the slicer lists:

Hey guys,

I am using BRAINSFit in my extension and noticed about that issue, so I am also using a previous version of Slicer.

The BRAINSFit module seems to have several problems now. The registration seems to last for a long time, and the resulting image is exactly the same as the moving one.

Additionally, I want to add another issue. Setting the "numberOfIterations" parameter does not work. For example, if I set this to 500 (even in the previous version), the algorithm reaches convergence in 795 iterations.

Thank you,

  Ángel Torrado Carvajal

  Research Assistant
  Department of Electronics
  Universidad Rey Juan Carlos

  D. 159 - Departamental II
  Calle Tulipán s/n
  28933 Móstoles - Madrid

[hjmjohnson]

Can we get some example uses please?

We re-wrote so that the the previous ITKv3 based regression tests continue to pass. It seems to be working for all of those tests:

http://testing.psychiatry.uiowa.edu/CDash/index.php?project=BRAINSTools&date=2014-07-06

Hans

From: Steve Pieper <[email protected]mailto:[email protected]>
Reply-To: BRAINSia/BRAINSTools <[email protected]mailto:[email protected]>
Date: Tuesday, July 8, 2014 at 6:41 AM
To: BRAINSia/BRAINSTools <[email protected]mailto:[email protected]>
Subject: Re: [BRAINSTools] some debugging code is causing BRAINS in Slicer to fail (#120)

Some more info from the slicer lists:

Hey guys,

I am using BRAINSFit in my extension and noticed about that issue, so I am also using a previous version of Slicer.

The BRAINSFit module seems to have several problems now. The registration seems to last for a long time, and the resulting image is exactly the same as the moving one.

Additionally, I want to add another issue. Setting the "numberOfIterations" parameter does not work. For example, if I set this to 500 (even in the previous version), the algorithm reaches convergence in 795 iterations.

Thank you,

Ángel Torrado Carvajal

Research Assistant
Department of Electronics
Universidad Rey Juan Carlos

D. 159 - Departamental II
Calle Tulipán s/n
28933 Móstoles - Madrid

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/120#issuecomment-48325601.

---

Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged. If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited. Please reply to the sender that you have received the message in error, then delete it. Thank you.

---

[pieper]

Related issue reported in EMSegment: http://www.na-mic.org/Bug/view.php?id=3561

[cpinter]

To reproduce:
Load MRHead, set it as fixed and moving, choose an output linear transform, select rigid, click apply.

[cpinter]

Also I second Andras' suggestion about reverting to a working version until this is fixed, as many people use the registration and weeks of complete inoperation will result in many reports, and possibly people leaving Slicer.

[hjmjohnson]

I can not fix items that I can not reproduce. Now that you’ve provided information about the failures, I can reproduce and work on them.

Csaba, The problem that you encounter is, I think, a problem with the new scales estimator. I will put out requests for help from the ITK team and work with them to fix this. I believe that BRAINSTools is correct, but ITK has a problem with estimating scales when there is a division by zero due to the fixed and moving images being the same.

The fix will likely be updating ITK.

Hans

From: Csaba Pinter <[email protected]mailto:[email protected]>
Reply-To: BRAINSia/BRAINSTools <[email protected]mailto:[email protected]>
Date: Tuesday, July 8, 2014 at 8:56 AM
To: BRAINSia/BRAINSTools <[email protected]mailto:[email protected]>
Cc: Hans Johnson <[email protected]mailto:[email protected]>
Subject: Re: [BRAINSTools] some debugging code is causing BRAINS in Slicer to fail (#120)

Also I second Andras' suggestion about reverting to a working version until this is fixed, as many people use the registration and weeks of complete inoperation will result in many reports, and possibly people leaving Slicer.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/120#issuecomment-48338912.

---

Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged. If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited. Please reply to the sender that you have received the message in error, then delete it. Thank you.

---

[cpinter]

Hi Hans,

Thank you for working on this!
Actually, setting the same volume as input and output was just an easier way to reproduce the issue I had with a CT and CBCT of the same jar (containing gel for dosimetry, the two volumes are very similar). Registering the two used to take about 2-3 seconds using 6DOF rigid reg. with geometry align. Now the same thing seems to happen with registering these images, as with using MRHead for both volumes.
This is how I set up the registration
https://dl.dropboxusercontent.com/u/3143581/screenshots/20140708_BrainsRegistrationTest.png
And here is the data
https://subversion.assembla.com/svn/slicerrt/trunk/SlicerRt/data/gel-dosimetry-01
There are two CT modalities in this directory, the volumes are used as seen in the screenshot above.

Let me know if you have any questions!

Thanks,
csaba

From: hjmjohnson [mailto:[email protected]]
Sent: July 8, 2014 10:59
To: BRAINSia/BRAINSTools
Cc: Csaba Pinter
Subject: Re: [BRAINSTools] some debugging code is causing BRAINS in Slicer to fail (#120)

I can not fix items that I can not reproduce. Now that you’ve provided information about the failures, I can reproduce and work on them.

Csaba, The problem that you encounter is, I think, a problem with the new scales estimator. I will put out requests for help from the ITK team and work with them to fix this. I believe that BRAINSTools is correct, but ITK has a problem with estimating scales when there is a division by zero due to the fixed and moving images being the same.

The fix will likely be updating ITK.

Hans

From: Csaba Pinter <[email protected]<mailto:[email protected]mailto:[email protected]%3cmailto:[email protected]>>
Reply-To: BRAINSia/BRAINSTools <[email protected]<mailto:[email protected]mailto:[email protected]%3cmailto:[email protected]>>
Date: Tuesday, July 8, 2014 at 8:56 AM
To: BRAINSia/BRAINSTools <[email protected]<mailto:[email protected]mailto:[email protected]%3cmailto:[email protected]>>
Cc: Hans Johnson <[email protected]<mailto:[email protected]mailto:[email protected]%3cmailto:[email protected]>>
Subject: Re: [BRAINSTools] some debugging code is causing BRAINS in Slicer to fail (#120)

Also I second Andras' suggestion about reverting to a working version until this is fixed, as many people use the registration and weeks of complete inoperation will result in many reports, and possibly people leaving Slicer.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/120#issuecomment-48338912.

---

Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged. If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited. Please reply to the sender that you have received the message in error, then delete it. Thank you.

---

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/120#issuecomment-48348699.

[hjmjohnson]

Csaba,

Thank you for the data. This is what we will need to get to the bottom of this problem. I’m pretty sure that the issue is related to the the automated scales space estimators. They work very well in most cases, but it seems that when the problem is nearly registered already there seems to be a problem with stability of that esimtation.

Hans

From: Csaba Pinter <[email protected]mailto:[email protected]>
Reply-To: BRAINSia/BRAINSTools <[email protected]mailto:[email protected]>
Date: Tuesday, July 8, 2014 at 10:37 AM
To: BRAINSia/BRAINSTools <[email protected]mailto:[email protected]>
Cc: Hans Johnson <[email protected]mailto:[email protected]>
Subject: Re: [BRAINSTools] some debugging code is causing BRAINS in Slicer to fail (#120)

Hi Hans,

Thank you for working on this!
Actually, setting the same volume as input and output was just an easier way to reproduce the issue I had with a CT and CBCT of the same jar (containing gel for dosimetry, the two volumes are very similar). Registering the two used to take about 2-3 seconds using 6DOF rigid reg. with geometry align. Now the same thing seems to happen with registering these images, as with using MRHead for both volumes.
This is how I set up the registration
https://dl.dropboxusercontent.com/u/3143581/screenshots/20140708_BrainsRegistrationTest.png
And here is the data
https://subversion.assembla.com/svn/slicerrt/trunk/SlicerRt/data/gel-dosimetry-01
There are two CT modalities in this directory, the volumes are used as seen in the screenshot above.

Let me know if you have any questions!

Thanks,
csaba

From: hjmjohnson [mailto:[email protected]]
Sent: July 8, 2014 10:59
To: BRAINSia/BRAINSTools
Cc: Csaba Pinter
Subject: Re: [BRAINSTools] some debugging code is causing BRAINS in Slicer to fail (#120)

I can not fix items that I can not reproduce. Now that you’ve provided information about the failures, I can reproduce and work on them.

Csaba, The problem that you encounter is, I think, a problem with the new scales estimator. I will put out requests for help from the ITK team and work with them to fix this. I believe that BRAINSTools is correct, but ITK has a problem with estimating scales when there is a division by zero due to the fixed and moving images being the same.

The fix will likely be updating ITK.

Hans

From: Csaba Pinter <[email protected]mailto:[email protected]mailto:[email protected]mailto:[email protected]%3cmailto:[email protected]>
Reply-To: BRAINSia/BRAINSTools <[email protected]mailto:[email protected]mailto:[email protected]mailto:[email protected]%3cmailto:[email protected]>
Date: Tuesday, July 8, 2014 at 8:56 AM
To: BRAINSia/BRAINSTools <[email protected]mailto:[email protected]mailto:[email protected]mailto:[email protected]%3cmailto:[email protected]>
Cc: Hans Johnson <[email protected]mailto:[email protected]mailto:[email protected]mailto:[email protected]%3cmailto:[email protected]>
Subject: Re: [BRAINSTools] some debugging code is causing BRAINS in Slicer to fail (#120)

Also I second Andras' suggestion about reverting to a working version until this is fixed, as many people use the registration and weeks of complete inoperation will result in many reports, and possibly people leaving Slicer.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/120#issuecomment-48338912.

---

Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged. If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited. Please reply to the sender that you have received the message in error, then delete it. Thank you.

---

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/120#issuecomment-48348699.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/120#issuecomment-48355859.

---

Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged. If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited. Please reply to the sender that you have received the message in error, then delete it. Thank you.

---

[lassoan]

Problem 1: Registration takes 100x longer to complete and the result is invalid.

I have the same issue. Registration takes hundred times longer (sometimes forever?). It fails even for the simplest possible scenario:
• Load MRHead sample
• Open General registration (BRAINS)
• Fixed image volume: MRHead
• Moving image volume: MRHead (same as fixed)
• Slicer linear transform: new
• Registration phases: Rigid (6 DOF)

3D Slicer 4.3.1-2014-04-25 => registration is completed in 2 seconds
3D Slicer 4.3.1-2014-06-30 => registration takes several minutes and returns with an error:

[lassoan]

Problem 2: (may be the same as Problem 1)

From: Marta Tibiletti [mailto:[email protected]]
Sent: Monday, July 07, 2014 10:56 AM
To: Andras Lasso
Cc: Dominik Meier
Subject: Re: [slicer-users] Problems with BRAINS transform conver

Dear Andras and Dominik,
an update with the transform converter problem I had. The last nightly built for windows runs the registration (only with BSpline), deforming the output volume, but apparently does not save the values of the BSpline transform (the tfm file exported is only filled with zeros). This also mean that the code to generate the Deformation Field runs smoothly, but generate basically nothing. At least that's my interpretation of the situation.

[lassoan]

Problem 3:

On Jul 3, 2014, at 7:17 AM, Marta Tibiletti [email protected] wrote:

Dear all,
I have a problem with the "BRAINS transform convert".
I can register two 3D datasets with the General Registration Module successfully. I would like to obtain the Deformation Field from the BSpline Transform generated, and to do so I am using the BRAINS transform convert. This seems to be the right method in the latest version, the BSpline to deformation field modules is under "legacy" only now.
The BRAINS transform convert either crashes the program (I guess it's a bug, but given my little experience I wanted to ask first before reporting it), either gives the error you can find in the end of the mail. I've noticed the first case happen when the transform is only "BSpline", the second happen if I include also "Affine".
Has anyone encountered this problem before? Am I missing something here?
Thanks a lot,
Marta Tibiletti
Experimental Cardiovascular Imaging (ExCaVI)
Core Facility Small Animal MRI
University of Ulm
Albert-Einstein-Allee 23
D-89081 Ulm

ERROR: In ........\Slicer-431-1\Base
QTCLI\vtkSlicerCLIModuleLogic.cxx, line 2114
vtkSlicerCLIModuleLogic (000000000F1018F0): BRAINS Transform Convert terminated with an exception:
itk::ExceptionObject (00000000078AB3D8)
Location: "unknown"
File: ..........\ITKv4\Modules\IO\TransformBase\src\itkTransformFileReader.cxx
Line: 50
Description: itk::ERROR: TransformFileReader(00000000148D44E0): Can't Create IO object for file C:/Users/MARTA~1.TIB/AppData/Local/Temp/Slicer/EGEI_AAAAAAAAAEGEAEEA.mrml#vtkMRMLBSplineTransformNode1

[hjmjohnson]

The problem is in ITK, and I am looking at it.

This simple example is the problem. We did not test it. The problem is with the automated scales estimator when the initial derivative is already 0.

Even a small translation will make this example pass. It is a case that we did not have in our current test suite.

I’m investigating as quickly as I can. I’ll get ITK folks to assist as soon as possible.

Hans

From: Andras Lasso <[email protected]mailto:[email protected]>
Reply-To: BRAINSia/BRAINSTools <[email protected]mailto:[email protected]>
Date: Tuesday, July 8, 2014 at 12:35 PM
To: BRAINSia/BRAINSTools <[email protected]mailto:[email protected]>
Cc: Hans Johnson <[email protected]mailto:[email protected]>
Subject: Re: [BRAINSTools] some debugging code is causing BRAINS in Slicer to fail (#120)

Problem 1: Registration takes 100x longer to complete and the result is invalid.

I have the same issue. Registration takes hundred times longer (sometimes forever?). It fails even for the simplest possible scenario:
• Load MRHead sample
• Open General registration (BRAINS)
• Fixed image volume: MRHead
• Moving image volume: MRHead (same as fixed)
• Slicer linear transform: new
• Registration phases: Rigid (6 DOF)

3D Slicer 4.3.1-2014-04-25 => registration is completed in 2 seconds
3D Slicer 4.3.1-2014-06-30 => registration takes several minutes and returns with an error:

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/120#issuecomment-48372987.

---

Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged. If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited. Please reply to the sender that you have received the message in error, then delete it. Thank you.

---

[lassoan]

There are very similar speed and registration result problems with all data sets that we’ve tried (not just with MRHead to MRHead registration, we just wanted to give the simplest possible example). For example, trying to register MRBrainTumor1 to MRBrainTumor2:

• Load MRBrainTumor1 sample
• Load MRBrainTumor2 sample
• Open General registration (BRAINS)
• Fixed image volume: MRBrainTumor1
• Moving image volume: MRBrainTumor2
• Slicer bspline transform: new
• Registration phases: Rigid (6 DOF) + BSpline (>27 DOF)

3D Slicer 4.3.1-2014-04-25 => registration is completed under 2 minutes
3D Slicer 4.3.1-2014-06-30 => registration takes 14 minutes and returns with an error:

ERROR: In ........\Slicer-1\Base\QTCLI\vtkSlicerCLIModuleLogic.cxx, line 2141
vtkSlicerCLIModuleLogic (00000000148EDCF0): General Registration (BRAINS) terminated with an exception:
itk::ExceptionObject (00000000072B1E18)
Location: "unknown"
File: c:\d\n\slicer-1-build\brainstools\brainscommonlib\BRAINSFitHelperTemplate.hxx
Line: 617
Description: itk::ERROR: Exception caught during registration:
itk::ExceptionObject (00000000072B1688)
Location: "unknown"
File: c:\d\n\slicer-1-build\itkv4\modules\io\transformbase\include\itkTransformIOBase.hxx
Line: 96
Description: itk::ERROR: MatlabTransformIOTemplate(000000001438F120): Failed opening fileDEBUGTransform_RegFilterOutput.mat

If it takes more than a few hours to fix everything then please revert the recent BRAINS changes. That would allow you to take as much time as you need for making all the necessary changes in ITK and BRAINS and properly test them, without making life of so many Slicer users and developers more complicated. We have already wasted a lot of time explaining users why this is happening, troubleshooting, directing people to older releases (which have other problems), other modules, etc.

[hjmjohnson]

Andras,

Do what you want with the Slicer build. I’m not going to revert it and deal with all the issues that will arise from that.

I’m focused on making BRAINSFit work for my needs. Your experiments are not consistent with our recent experiences.

Reverting BRAINSTools includes reverting DWIConvert, and other tools.

Hans

From: Andras Lasso <[email protected]mailto:[email protected]>
Reply-To: BRAINSia/BRAINSTools <[email protected]mailto:[email protected]>
Date: Tuesday, July 8, 2014 at 5:05 PM
To: BRAINSia/BRAINSTools <[email protected]mailto:[email protected]>
Cc: Hans Johnson <[email protected]mailto:[email protected]>
Subject: Re: [BRAINSTools] some debugging code is causing BRAINS in Slicer to fail (#120)

There are very similar speed and registration result problems with all data sets that we’ve tried (not just with MRHead to MRHead registration, we just wanted to give the simplest possible example). For example, trying to register MRBrainTumor1 to MRBrainTumor2:

• Load MRBrainTumor1 sample
• Load MRBrainTumor2 sample
• Open General registration (BRAINS)
• Fixed image volume: MRBrainTumor1
• Moving image volume: MRBrainTumor2
• Slicer bspline transform: new
• Registration phases: Rigid (6 DOF) + BSpline (>27 DOF)

3D Slicer 4.3.1-2014-04-25 => registration is completed under 2 minutes
3D Slicer 4.3.1-2014-06-30 => registration takes 14 minutes and returns with an error:

ERROR: In ........\Slicer-1\Base\QTCLI\vtkSlicerCLIModuleLogic.cxx, line 2141
vtkSlicerCLIModuleLogic (00000000148EDCF0): General Registration (BRAINS) terminated with an exception:
itk::ExceptionObject (00000000072B1E18)
Location: "unknown"
File: c:\d\n\slicer-1-build\brainstools\brainscommonlib\BRAINSFitHelperTemplate.hxx
Line: 617
Description: itk::ERROR: Exception caught during registration:
itk::ExceptionObject (00000000072B1688)
Location: "unknown"
File: c:\d\n\slicer-1-build\itkv4\modules\io\transformbase\include\itkTransformIOBase.hxx
Line: 96
Description: itk::ERROR: MatlabTransformIOTemplate(000000001438F120): Failed opening fileDEBUGTransform_RegFilterOutput.mat

If it takes more than a few hours to fix everything then please revert the recent BRAINS changes. That would allow you to take as much time as you need for making all the necessary changes in ITK and BRAINS and properly test them, without making life of so many Slicer users and developers more complicated. We have already wasted a lot of time explaining users why this is happening, troubleshooting, directing people to older releases (which have other problems), other modules, etc.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/120#issuecomment-48405426.

---

Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged. If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited. Please reply to the sender that you have received the message in error, then delete it. Thank you.

---

[lassoan]

Yes, sure, I meant that we should revert changes in Slicer to use the BRAINS version that worked well until problems in ITK, BRAINS, and Slicer are all resolved. Temporarily reverting to an earlier BRAINS version should be quite simple and any limitations are probably less serious as the current regressions.

If standalone tests are passing then probably the issues are in how the tools are used from Slicer (default values are not optimal, some parameters are not passed correctly, etc).

[hjmjohnson]

I believe that the main cause of these failures has been addressed in the BRAINSTools main repository. I am unsure which version is being included in Slicer.

[cpinter]

Great news, thank you for your efforts!