imsy-dkfz / simpa Goto Github PK

• DMAS
• sDMAS

For linear setups, the mapping of the internal list of detection elements matches with the k-Wave internal ordering of elements in the binary mask, however, for non-linear setups, this is not necessarily the case (especially apparent for ring transducers).

I propose we switch to an alternative probe representation using the rectangular detector definition:

http://www.k-wave.org/documentation/example_ivp_opposing_corners_sensor_mask.php

This might resolve the issue and does not necessitate adapter-specific reordering of the detector element list.

Currently, the RAM size required by highly resolved simulations is very high. Since we generated the code focussing mostly on performance, it might be beneficial to revisit especially the core modules to see where we could reduce the RAM usage.

Using the optical_and_acoustic_simulation and setting the ACOUSTIC_SIMULATION_3D Tag to False causes this error: sensor.mask cuboid corners must be within the grid.

This error is probably due to the way the detector definition in the acoustic_forward_module_k_wave_adapter is done.

It seems as if we currently have a flipping error in the simulation pipeline which might occur because MATLAB is indexing in another way than numpy/python. This error doesn't occur when reconstructing real photoacoustic images, only when reconstructing simulated ones. Thus we should solve it when calling the MATLAB code and before the reconstruction. For me a simple left/right flip of the reconstructed image solved the issue.

Currently, the segmentation loader saves the input segmentation twice. Once in the settings and once again in the simulation properties. This unnecessarily increases the savefile size. I think we should remove the array in the settings.

Paths should be defined in a paths_config file.
The access is handled via a "PathManager" (please chose a good name).
Distinction config versus settings is important.

Default is home directory.

Currently, mcxyz is supported only with a special MITK-based implementation of the original work of Steven.

If we want to continue support mcxyz I propose doing so using the original mcxyz programme, or to create a clone (e.g. on Github) of mcxyz such that external researchers are able to use it as well.

I suggest the following:

• remove current mcxyz support
• create a strategy to include mcxyz again
• open new issue to support mcxyz

• move pre- post-processing to "processing" from the reconstruction module
• 3D implementation
• extract field of view from device

Currently, the entire reconstruction code is written without the use of any loops, sacrificing memory consumption for increased performance.
While this is fine for the 2D image reconstruction we are currently conducting, it will not scale to larger images, or 3D use cases.

I propose we implement a heuristic that approximates the memory consumption for the reconstruction and falls back to iterating over the detection elements in case the GRAM of the GPU device is not sufficient for the task.

https://github.com/CAMI-DKFZ/simpa/blob/19114b55857468d0f2b406fa150f1306ff022326/simpa/core/acoustic_simulation/k_wave_adapter.py#L90-L102

We should try and find a cleaner way to do this and properly document why this is done..
Why are some volumes modified with fliplr and rot90 and other only with rot90(-3)? Is this really equivalent?

This is currently implemented really weirdly and pretty dirtily.
Should be completely refactored with the #11 feature integration.

In the past, we have failed to adequately find internally used scripts, for example, to find the path of MATLAB scripts that resides in a python module.
We should finally fix this with the new Python 3.9 changes to the file definition:

From https://note.nkmk.me/en/python-script-file-path/:

In Python, you can get the location (path) of the running script file .py with file. file is useful for reading other files based on the location of the running file.

In Python 3.8 and earlier, file returns the path specified when executing the python (or python3) command. If you specify a relative path, a relative path is returned. If you specify an absolute path, an absolute path is returned.

In Python 3.9 and later, file always returns an absolute path, regardless of whether the path specified with the python command is relative or absolute.

MCX does not support multi-source illuminators the way we imagined them to.
Since the p0 is a multi-source setup is simply the sum of all p0 of the individual illuminators, we could implement a workaround that sequentially simulated the individual sources and compounds them into a single p0 image.

This should be possible in a minimally invasive manner and allows us greater flexibility in the future as well.

There have been different types of errors reported when trying to use mcx-cl.

I suggest adding to the documentation that mcx-cl is not officially supported by us.

Add the possibility to define custom volumes in settings, without the need to define an extra volume creator. This involves the checkup of the parsed volumes, they need to contain a minimum set of tags in order for the optical simulations to work.

This is currently not correctly defined.
Related to #12

• Uniform Noise
• Salt and Pepper (Impulse) Noise
• Gamma Noise
• Poisson Noise

The field of view should be defined as a cuboid the following way:

self.fov _mm = [xmin, xmax, ymin, ymax, zmin, zmax]

This field of view is in mm coordinates relative to the volume sizes that start at 0, 0, 0 and go to xdim, ydim, zdim.

Constraints:
The (xyz)min values must be >= 0 and the (xyz)max values must be <= (xyz)dim.

This fov definition should then be used in the "extract_field_of_view" method as well as the reconstruction algorithms.

• Data axis selection should not update any of the 3D visualization plots
  - [ ] Contour 3D seems to fail for many volumes, and it is slow computations, it could be replaced by dash_slicer https://dash.plotly.com/slicer
• Volume slicer should not modify any of the 3D visualizations
• Changing color scale should not update 3D plots
• Right side slider bottom controller does not seem to update plot, should be fixed
• Color scale should update colors in contour 2D
• Spectral Visualizations should autoscale after each new point is added
• Dimensions of volume should rotate along with volume when axis changes in dropdown menu
• Reset points button should reset the layout of the spectral visualization scatter plot
• Remove duplicated callbacks that update plots plot_11 and plot_12
• Add reconstruction parameters to the dropdown menus so that they can also be visualized
• Visualization tabs should only include plots and not data selection columns
• Tabs should be mutually exclusive. When one is activated, the other is deactivated
• Replace data selection axis from dropdown to radio buttons
• Volume slicer should update after selecting a different axis
• Add interactions to select regions on plot_11 and show histogram on Spectral visualization
• Use segmentation keywords to create color scale with names
• Fix error SVGGeometryElement.getPointAtLength:No path available for measuring
• Show annotations when clicking on data plot_11 and plot_12
• Sync param1 with param3
• Add number of bins controller for histogram plot
• Replace annotation button with switch
• Deactivate n_bins controller when annotations switch is off
• Add documentation to each callback
• Write unit tests for main callbacks
• Data fields should be read from SIMPA output by looking at data fields that start with data_field
• App should support reading SIMPA outputs that contain both 3D and 2D volumes as well as time series data
  - [ ] Disable volume axis radio buttons if the volume is only 2D

A feature is needed that adds convenience functions for data visualisation.
Currently, a script is present in the simpa_examples package that does this and there is some work that is being done on creating a dash visualisation script.

I propose to add a visualise package in SIMPA that contains such matplotlib or dash/plotly visualisations.

It would be beneficial to have a method or class that can check for consistency of volumes before and after simulations are done. Some things that can be checked are listed below.

Before simulations

• All volumes have the same shape
• There are no np.nan, None, np.inf or negative values in volumes
• #100
• #99

After simulations

• There are no np.nan, None, np.inf or negative values in volumes
• That volumes have the same shape

In order to simulate the diffuse reflectance at the surface of the volumes, the top layer has to be all 0's (required by MCX). Therefore, the volume has to be modified in case this requirement is not met. It is not clear how to implement this in the current core of SIMPA.
One option would be to separate diffuse reflectance simulation from fluence simulations. It has to be taken into account that if it is completely separate, then 2 simulations are required for the same volume when both fluence and diffuse reflectance are desired.

Currently the simulation test in PyTorchDASReconstruction.py doesn't work anymore after the settings were restructured. This should be fixed so that we can verify the correctness of the reconstruction algorithm again.

At the moment we use the Tags.MCX_SEED tag in the McxAdapter, but we never defined it:

if Tags.MCX_SEED in settings:
            settings_dict["RNGSeed"] = settings[Tags.MCX_SEED]

Is there any sensible default value? Otherwise creating this tag would simply solve the issue.

It would be nice to also have an additional test that only tests the reconstruction algorithm without running all previous pipeline steps. This could for example be achieved by having previously simulated time series data in an hdf5 file.

When a specific imaging device is defined in the settings file, the illumination geometry should not have to be set separately.
The device instance should take care of correctly defining the illumination geometry.

On simulation.py the results are loaded and saved consecutively and commented as "dirty fix"

# Quick and dirty fix:
all_data = load_hdf5(settings[Tags.SIMPA_OUTPUT_PATH])
save_hdf5(all_data, settings[Tags.SIMPA_OUTPUT_PATH])

This should probably be removed and check that it does not break anything

correctly assign absorption coefficients in blood vessels:

set water volume fraction to 0 in the tissue_library for the following classes:

• blood_generic
• blood_venous
• blood_arterial

With the Photoacoustics modules being removed from core MITK and the internal implementations not being generally available, it might be a good idea to also remove MITK beamforming support in SIMPA?

For all in silico experiments towards qPAI, a comparison to the iterative qPAI algorithm proposed by Cox et al. would be extremely helpful.
Since we have all the essential components for the algorithm already implemented, I propose to add this functionality.

Currently, there is no designated fields for image processing outputs. This is related to #43 and #26.

Currently, the behaviour is:
If no custom path is supplied, look inside the $HOME$ directory for a config file.

Maybe this should be:
If no custom path is supplied, look in your local execution path, if nothing is there, look inside the $HOME$ directory for a config file.

There currently is the option to save the settings dictionary as a JSON file in addition to the HDF5 file.
As far as I am aware, however, there is no functionality to start a simulation from a JSON file.

I have the following issues with this:

• There is no functionality to make use of this JSON file integrated within SIMPA.
• With the new pipelining concept, running a simulation just from the settings file is not possible anymore.
• It clutters the simulation.simulate method with a hidden feature.

I propose to remove the JSON saving feature.
Alternatively, the JSON support should be expanded upon and integrated throughout the pipeline.

The test imports the following, but this module is missing in the repo.

from simpa_tests.test_utils import create_test_structure_of_molecule, create_background_of_molecule, set_settings

The way the generation of the documentation is currently set up is only in a "semi-automatic" manner.
An index file references all code files that should be included into the documentation and allows for the addition of extra text which creates a lot of flexibility.

However, during refactorings, these file references are not changed as well by the PyCharm IDE, meaning that many classes are currently not included into the documentation.

We need to fix these references and rework the documentation to reflect the way the toolkit currently works.

Currently setting the Tags.EXTRACT_FIELD_OF_VIEW flag saves a single slice out of the volume, which is assuming a linear transducer in a certain location and orientation.

This should be implemented after #11.

It is currently not visible to the user that the provided probe designs require a special mcx version that is not available to work.

• Linear Array
• Curved Array
• Re-implement Acuity system
• Re-implement RSOM system
• Re-implement InVision system
• Create a fork of mcx that contains the probe specifications for everyone to use

Currently, our module and component naming conventions are a mess.
I propose the following naming convention:

Any module adapter has to have the following naming convention:

• {PURPOSE}Module{FRAMEWORK}Adapter
• For example: OpticalModuleMcxAdapter or AcousticModuleKWaveAdapter

Any processing component has to have the following naming convention:

• {DESCRIPTION}ProcessingComponent
• For example: LinearUnmixingProcessingComponent or GaussianNoiseProcessingComponent

This way, all framework adapters for the modules and all processing components are easily identifiable as such.

SIMPA currently has no internal logging features.
Everything is logged to the command line, without any regard to the severity of the log entry.

We should create a "log" module that can be used to easily get hold of a logger singleton instance.

Look up the literature values of optical and acoustic properties of the tissue classes and molecule classes.

Optical properties:

• mua (at 700, 800, and 900 nm)
• mus (at 500 nm)
• anisotropy

Acoustic properties:

• density
• speed of sound
• alpha coefficient

Please collect the info in one file (preferably nicely structured :-)) and don't forget to include the references.

If we manually use the gaussian filter in the Matlab scripts for the acoustic simulations, it leads to bad image qualities.

The copyright notice should be extended by including the line

(c) 2021 VISION Lab, Cancer Research UK, Cambridge Institute

Define abstract method in SimulatioModule class

Should we add another hierarchical layer to the structure file separating fields for specific modules?
Currently, everything is nice and flat, but especially full pipeline simulation scripts feel very overloaded.

Since the settings dict is wrapped in a Settings class anyways, we could add convenience functions (settings.get_optical_settings(), get_acoustic_settings(), get_reconstruction_settings()) to access these different hierarchical levels.

I believe this could greatly improve the readability of the settings file.

Also the definition could be structured as follows:

settings = Settings()
settings.add_general_settings(general_dict)
settings.add_optical_settings(optical_dict)
....

where the respective dicts can be defined in nice and short methods.