This repository contains code for the Jterator image analysis pipeline (JtModules), and a collection of support and utility functions (JtLibrary).
For more information, please refer to main TissueMAPS repository.
Please use the https://github.com/TissueMAPS/TissueMAPS instead.
License: Apache License 2.0
This repository contains code for the Jterator image analysis pipeline (JtModules), and a collection of support and utility functions (JtLibrary).
For more information, please refer to main TissueMAPS repository.
This code works fine in the first iteration, but in the next iteration- it gives the error:
Traceback (most recent call last):
File "", line 1, in
runfile('/Users/saadiaiftikhar/miniconda2/bin/jlib/test_20160531.py', wdir='/Users/saadiaiftikhar/miniconda2/bin/jlib')
File "/Users/saadiaiftikhar/miniconda2/lib/python2.7/site-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 807, in runfile
execfile(filename, namespace)
File "/Users/saadiaiftikhar/miniconda2/lib/python2.7/site-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 88, in execfile
builtins.execfile(filename, *where)
File "/Users/saadiaiftikhar/miniconda2/bin/jlib/test_20160531.py", line 99, in
[secondaryLabelMatrixImage,edited_primary_binary_image,th_array] = identify_secondary_objects_iw.identify_secondary_objects_iw(label_image,input_image,correction_factors,min_threshold)
File "identify_secondary_objects_iw.py", line 48, in identify_secondary_objects_iw
min_threshold, max_threshold)
File "segment_secondary.py", line 190, in segment_secondary
labels_used1[actual_objects_label_matrix_image3]
IndexError: index 4 is out of bounds for axis 1 with size 4
Whenever plotting is set to True in a module in the current develop branch (14db3a8), the pipeline fails because of the following import error:
JOB FAILED: cannot import name plotting File "/home/tissuemaps/tmlibrary/tmlib/workflow/cli.py", line 283, in __main__ cli_instance(arguments) File "/home/tissuemaps/tmlibrary/tmlib/workflow/cli.py", line 342, in __call__ method(**method_args) File "/home/tissuemaps/tmlibrary/tmlib/workflow/jterator/cli.py", line 102, in debug api.run_job(batch, assume_clean_state=False) File "/home/tissuemaps/tmlibrary/tmlib/workflow/jterator/api.py", line 679, in run_job store = self._run_pipeline(store, site_id, batch['plot']) File "/home/tissuemaps/tmlibrary/tmlib/workflow/jterator/api.py", line 363, in _run_pipeline module.run(self._engines[module.language]) File "/home/tissuemaps/tmlibrary/tmlib/workflow/jterator/module.py", line 470, in run return self._exec_py_module() File "/home/tissuemaps/tmlibrary/tmlib/workflow/jterator/module.py", line 196, in _exec_py_module py_out = func(**kwargs) File "/home/tissuemaps/jtlibrary/python/jtmodules/src/jtmodules/smooth.py", line 85, in main from jtlib import plotting
The "from jtlib import plotting" doesn't work anymore because of the refactor that reorganized the modules & jtlibrary code. @riccardomurri How can one import the plotting file which is located at "JtLibrary/python/jtlibrary/src/jtlib/"? This import needs to be changed for all modules that use plotting.
The current approach for saving and retrieving objects is based on outline coordinates of objects.
skimage.measure.find_contours()
cv2.fillPoly()
It needs to be tested whether this approach works reliably for different types of objects...
Error in jtlib.segmentSecondary (line 502)
FinalLabelMatrixImagePre = LabelsUsed(ActualObjectsLabelMatrixImage3+1);
Index exceeds matrix dimensions.
Set plot
to True
and don't write pipeline output into the database when running in UI mode.
For some images, the object count is wrong after applying the watershed segmentation. The reason is not clear to me yet. It is related to the shape of the primary objects, however, since smoothing the input mask solves the problem. It could be related to the labeling of objects, which is different in Matlab and Python.
I tried various possibilities, but the error remained the same for the pipeline:
() | ___ _ _ __ | | ___ _ _ tmlib.workflow.jterator (0.0.7)
| | / -) '/ ` | / _ \ '| Image analysis pipeline engine
/ ||| **,|___/_| https://github.com/TissueMAPS/TmLibrary
|**/
2016-06-02 13:07:14 | INFO | tmlib.workflow.cli | delete previous job output
2016-06-02 13:07:14 | INFO | tmlib.workflow.jterator.api | delete existing mapobject types
2016-06-02 13:07:14 | INFO | tmlib.workflow.cli | create batches
The following error occurred while reading from file:
while parsing a block mapping
in "", line 2, column 1:
input:
^
expected , but found '-'
in "", line 8, column 1:
- name: filter_name
^ File "/home/ubuntu/TmLibrary/src/tmlib/workflow/jterator/project.py", line 212, in _create_handles
'description': f.read()
File "/home/ubuntu/TmLibrary/src/tmlib/readers.py", line 182, in read
return load_yaml(self._stream.read())
File "/home/ubuntu/TmLibrary/src/tmlib/readers.py", line 162, in load_yaml
return ruamel.yaml.load(string, ruamel.yaml.RoundTripLoader)
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/main.py", line 75, in load
return loader.get_single_data()
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/constructor.py", line 59, in get_single_data
node = self.get_single_node()
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/composer.py", line 48, in get_single_node
document = self.compose_document()
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/composer.py", line 68, in compose_document
node = self.compose_node(None, None)
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/composer.py", line 99, in compose_node
node = self.compose_mapping_node(anchor)
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/composer.py", line 151, in compose_mapping_node
while not self.check_event(MappingEndEvent):
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/parser.py", line 116, in check_event
self.current_event = self.state()
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/parser.py", line 505, in parse_block_mapping_key
token.start_mark)
The way modules are currently imported makes it difficult to debug them.
They should probably be included into the jtlib
package as subpackage modules
. Thereby one could import them "normally" and set break points in the code.
Maybe we should re-think the whole approach of providing the absolute path to modules. The idea behind that was that one could easily test code located somewhere else, but this may not pay off.
Creation of a jterator module for pixel based classification ร la CellProfiler classifypixels module. The idea is to train a classifier with the Ilastik GUI, save the output (either the classifier as ilastik project HDF5 file or a probability map as an image file) and then apply the output to images in a jterator pipeline. To this end, we need to create a Python module that implements functions to call ilastik independent of CP2.
ubuntu@frontend001:/data/storage/experiments/experiment_1/workflow/jterator_segment_2d_cells_40x$ jterator -v 1 -p segment_2d_cells_40x init
_ _ _
() | ___ _ _ __ | | ___ _ _ tmlib.workflow.jterator (0.0.7)
| | / -) '/ ` | / _ \ '| Image analysis pipeline engine
/ ||| **,|___/_| https://github.com/TissueMAPS/TmLibrary
|**/
2016-06-01 13:24:40 | INFO | tmlib.workflow.cli | delete previous job output
2016-06-01 13:24:40 | INFO | tmlib.workflow.jterator.api | delete existing mapobject types
2016-06-01 13:24:40 | INFO | tmlib.workflow.cli | create batches
The following error occurred while reading from file:
while scanning for the next token
found character '\t' that cannot start any token
in "", line 6, column 16:
correct: true
^ File "/home/ubuntu/TmLibrary/src/tmlib/workflow/jterator/project.py", line 165, in _create_pipe
'description': f.read()
File "/home/ubuntu/TmLibrary/src/tmlib/readers.py", line 182, in read
return load_yaml(self._stream.read())
File "/home/ubuntu/TmLibrary/src/tmlib/readers.py", line 162, in load_yaml
return ruamel.yaml.load(string, ruamel.yaml.RoundTripLoader)
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/main.py", line 75, in load
return loader.get_single_data()
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/constructor.py", line 59, in get_single_data
node = self.get_single_node()
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/composer.py", line 48, in get_single_node
document = self.compose_document()
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/composer.py", line 68, in compose_document
node = self.compose_node(None, None)
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/composer.py", line 99, in compose_node
node = self.compose_mapping_node(anchor)
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/composer.py", line 158, in compose_mapping_node
item_value = self.compose_node(node, item_key)
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/composer.py", line 97, in compose_node
node = self.compose_sequence_node(anchor)
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/composer.py", line 128, in compose_sequence_node
node.value.append(self.compose_node(node, index))
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/composer.py", line 99, in compose_node
node = self.compose_mapping_node(anchor)
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/composer.py", line 158, in compose_mapping_node
item_value = self.compose_node(node, item_key)
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/composer.py", line 77, in compose_node
if self.check_event(AliasEvent):
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/parser.py", line 116, in check_event
self.current_event = self.state()
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/parser.py", line 518, in parse_block_mapping_value
token.move_comment(self.peek_token())
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/scanner.py", line 1543, in peek_token
self._gather_comments()
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/scanner.py", line 1573, in _gather_comments
self.fetch_more_tokens()
File "/home/ubuntu/.local/lib/python2.7/site-packages/ruamel/yaml/scanner.py", line 276, in fetch_more_tokens
% utf8(ch), self.get_mark())
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.