deepcell's Introduction
deepcell's People
Forkers
tsuresh83 chelovekhe nagyistge coocoky jacobkimmel imodpasteur datamol zoonono calebium maincover alxmertz jtsauls bettygirls whiteau guihuay xianqiangsun yubinxie rcasiodu rinawhale xies vyas-pranav trasse saeedseyyedi duerwen lb-hitdeepcell's Issues
License?
Thank you for publishing this interesting and innovative paper.
The code published along looks like a good start for further customization or research, however, currently, I cannot spot licensing terms for the code.
It would be helpful, if you could add licensing terms (e.g. an open source license, if you have not yet decided which one, please choose carefully) to the repository.
Thank you, best regards,
Christian Sachs
Where do cnn_functions come from?
Trying to run your example scripts, but python cannot find cnn_functions. is that a dependency?
ImportError: No module named cnn_functions
running_template.py broken?
Great work.
Though when I do the running template for HeLa. The following code:
cytoplasm_predictions = run_models_on_directory(data_location, cyto_channel_names, cyto_location, model_fn = cyto_fn,
list_of_weights = list_of_cyto_weights, image_size_x = image_size_x, image_size_y = image_size_y,
win_x = win_cyto, win_y = win_cyto, std = False, split = False)
leads to the following runtime error:
theano.tensor.var.AsTensorError: ('Cannot convert Tensor("sparse_convolution2d_1_W/read:0", shape=(3, 3, 280, 64), dtype=float32) to TensorType', <class 'tensorflow.python.ops.variables.Variable'>)
Is there anything about the dimensionality I need to fix? Thanks.
Bug: "from keras import initializations" is outdated
The line
from keras import initializations as initializations
in keras_version/cnn_functions.py
produces the error
ImportError: cannot import name initializations
in recent versions of keras (e.g. 2.0.2), as reported here.
simplified workflow/directory request
It is hard to know which scripts are meant to be demonstrative or are development scripts. Would it be possible to build a folder solely with working scripts for 1) making training sets, 2) training the CNNs and 3) execution. The scripts found here: https://covertlab.github.io/DeepCell/starting/ don't actually work out of the box. There are also duplicated scripts across keras_version
and prototypes
. It is hard to know if a bug is because of my system or because of the code. I know it is "buyer beware" but it would be a great help to have a baseline working set of scripts, one for each process for just one cell type as an introductory example. Thanks and thanks for making the code available.
TypeError: __call__() got an unexpected keyword argument 'name'
I follow the instruction from https://covertlab.github.io/DeepCell/running_convnets.html
and code below
(In[6])
cytoplasm_predictions = run_models_on_directory(data_location, cyto_channel_names, cyto_location, model_fn = cyto_fn, list_of_weights = list_of_cyto_weights, image_size_x = image_size_x, image_size_y = image_size_y, win_x = win_cyto, win_y = win_cyto, std = False, split = False)
always responsed with error below
Traceback (most recent call last):
File "test.py", line 52, in
win_x = win_cyto, win_y = win_cyto, std = False, split = False)
File "/home/user/DeepCell/keras_version/cnn_functions.py", line 1491, in run_models_on_directory
model = model_fn(batch_input_shape = batch_input_shape, n_features = n_features, weights_path = list_of_weights[0])
File "/home/user/DeepCell/keras_version/model_zoo.py", line 528, in sparse_bn_feature_net_61x61
model.add(sparse_Convolution2D(64, 3, 3, d = d, init = init, batch_input_shape = batch_input_shape, border_mode='valid', W_regularizer = l2(reg)))
File "/home/user/.pyenv/versions/DeepCell/lib/python2.7/site-packages/keras/models.py", line 436, in add
layer(x)
File "/home/user/.pyenv/versions/DeepCell/lib/python2.7/site-packages/keras/engine/topology.py", line 569, in __ call __
self.build(input_shapes[0])
File "/home/user/DeepCell/keras_version/cnn_functions.py", line 1012, in build
self.W = self.init(self.W_shape, name='{}_W'.format(self.name))
TypeError: __ call __() got an unexpected keyword argument 'name'
keras=2.0.6
python=2.7.4
running convnets example error
Hi Dave,
sorry I run into trouble again. i switched to using old version of keras (1.2)
I can run the running_template for theriot and give good results.
However when i'm trying to modify for it to run for segmentation on HeLa, 3T3, or MCF10A data set, I'm getting error.
I wonder if it's possible for ask for your kind help?
import os
import datetime
import numpy as np
os.environ["KERAS_BACKEND"]="theano"
os.environ["THEANO_FLAGS"]="mode=FAST_RUN,device=gpu,floatX=float32"
import keras
keras.backend.set_image_dim_ordering('th')
print 'keras version is ' + keras.__version__
import h5py
import tifffile as tiff
from keras.backend.common import _UID_PREFIXES
from cnn_functions import nikon_getfiles, get_image, run_models_on_directory, get_image_sizes, segment_nuclei, segment_cytoplasm, dice_jaccard_indices
from model_zoo import sparse_bn_feature_net_61x61 as cyto_fn
from model_zoo import sparse_bn_feature_net_61x61 as nuclear_fn
direc_name = '/home/davince/DeepCell-master/validation_data/HeLa/'
data_location = os.path.join(direc_name, 'RawImages')
cyto_location = os.path.join(direc_name, 'Cytoplasm')
nuclear_location = os.path.join(direc_name, 'Nuclear')
mask_location = os.path.join(direc_name, 'Masks')
cyto_channel_names = ['phase']
nuclear_channel_names = ['farred']
trained_network_cyto_directory = "/home/davince/DeepCell-master/trained_networks/HeLa/"
trained_network_nuclear_directory = "/home/davince/DeepCell-master/trained_networks/Nuclear/"
cyto_prefix = "2016-07-12_HeLa_all_61x61_bn_feature_net_61x61_"
nuclear_prefix = "2016-07-12_nuclei_all_61x61_bn_feature_net_61x61_"
win_cyto = 30
win_nuclear = 30
image_size_x, image_size_y = get_image_sizes(data_location, nuclear_channel_names)
list_of_cyto_weights = []
for j in xrange(5):
cyto_weights = os.path.join(trained_network_cyto_directory, cyto_prefix + str(j) + ".h5")
list_of_cyto_weights += [cyto_weights]
list_of_nuclear_weights = []
for j in xrange(5):
nuclear_weights = os.path.join(trained_network_nuclear_directory, nuclear_prefix + str(j) + ".h5")
list_of_nuclear_weights += [nuclear_weights]
cytoplasm_predictions = run_models_on_directory(data_location, cyto_channel_names, cyto_location, model_fn = cyto_fn,
list_of_weights = list_of_cyto_weights, image_size_x = image_size_x, image_size_y = image_size_y,
win_x = win_cyto, win_y = win_cyto, split = False)
nuclear_predictions = run_models_on_directory(data_location, nuclear_channel_names, nuclear_location, model_fn = nuclear_fn,
list_of_weights = list_of_nuclear_weights, image_size_x = image_size_x, image_size_y = image_size_y,
win_x = win_nuclear, win_y = win_nuclear, split = False)
and the error is like this
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
<ipython-input-20-1bae99843176> in <module>()
13 nuclear_predictions = run_models_on_directory(data_location, nuclear_channel_names, nuclear_location, model_fn = nuclear_fn,
14 list_of_weights = list_of_nuclear_weights, image_size_x = image_size_x, image_size_y = image_size_y,
---> 15 win_x = win_nuclear, win_y = win_nuclear, split = False)
/home/davince/DeepCell-master/keras_version/cnn_functions.pyc in run_models_on_directory(data_location, channel_names, output_location, model_fn, list_of_weights, n_features, image_size_x, image_size_y, win_x, win_y, std, split, process, save)
1482
1483 batch_input_shape = (1,len(channel_names),image_size_x+win_x, image_size_y+win_y)
-> 1484 model = model_fn(batch_input_shape = batch_input_shape, n_features = n_features, weights_path = list_of_weights[0])
1485 n_features = model.layers[-1].output_shape[1]
1486
/home/davince/DeepCell-master/keras_version/model_zoo.pyc in sparse_bn_feature_net_61x61(batch_input_shape, n_features, reg, init, weights_path)
567 model.add(Activation(tensorprod_softmax))
568
--> 569 model = set_weights(model, weights_path)
570
571 return model
/home/davince/DeepCell-master/keras_version/cnn_functions.pyc in set_weights(model, weights_path)
90
91 for layer in model.layers:
---> 92 if layer.name in f['model_weights'].keys():
93 if 'bn' in layer.name:
94 g = f['model_weights'][layer.name]
h5py/_objects.pyx in h5py._objects.with_phil.wrapper (/tmp/pip-4rPeHA-build/h5py/_objects.c:2684)()
h5py/_objects.pyx in h5py._objects.with_phil.wrapper (/tmp/pip-4rPeHA-build/h5py/_objects.c:2642)()
/usr/local/lib/python2.7/dist-packages/h5py/_hl/group.pyc in __getitem__(self, name)
164 raise ValueError("Invalid HDF5 object reference")
165 else:
--> 166 oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
167
168 otype = h5i.get_type(oid)
h5py/_objects.pyx in h5py._objects.with_phil.wrapper (/tmp/pip-4rPeHA-build/h5py/_objects.c:2684)()
h5py/_objects.pyx in h5py._objects.with_phil.wrapper (/tmp/pip-4rPeHA-build/h5py/_objects.c:2642)()
h5py/h5o.pyx in h5py.h5o.open (/tmp/pip-4rPeHA-build/h5py/h5o.c:3570)()
KeyError: "Unable to open object (Object 'model_weights' doesn't exist)"
But the weights seems to be loaded in the list_of_weights
is it the same problem you mentioned keras changed the way the weights are saved?
(now i reverted back to 1.2)
the h5py version is 2.6.0
Also, i know from your paper, you wrote the nuclear channel help segmentation.
Is this limited to semantic segmentation in coculture examples?
Or is it general for all experiment? Could the segmentation be also good if there is no nuclear channel?
Thank you.
error in file IO of a training data npz exactly where it is but can't find it.
Hi,
deepcell looks really really awesome and seems to be doing a great job segmenting touching cells. So I am really interested in doing cell tracking taken on phase contrast microscope.
I've been trying to do it in Unet and modified Segnet but the results haven't been very promising.
But when i'm just following your tutorial on training convnets notebook.
(I run the docker version. because trying to build has it's own problem from newer version of keras)
It shows a error trying to find the training data npz.
I have the file exactly where it is but it keep showing it cannot find the file.
IOErrorTraceback (most recent call last)
<ipython-input-12-bbf0fd3fe821> in <module>()
8 direc_data = direc_data,
9 lr_sched = lr_sched,
---> 10 rotate = True, flip = True, shear = False)
11
12 del model
/root/DeepCell/keras_version/cnn_functions.py in train_model_sample(model, dataset, optimizer, expt, it, batch_size, n_epoch, direc_save, direc_data, lr_sched, rotate, flip, shear, class_weight)
1326 file_name_save_loss = os.path.join(direc_save, todays_date + "_" + dataset + "_" + expt + "_" + str(it) + ".npz")
1327
-> 1328 train_dict, (X_test, Y_test) = get_data_sample(training_data_file_name)
1329
1330 # the data, shuffled and split between train and test sets
/root/DeepCell/keras_version/cnn_functions.py in get_data_sample(file_name)
360
361 def get_data_sample(file_name):
--> 362 training_data = np.load(file_name)
363 channels = training_data["channels"]
364 batch = training_data["batch"]
/root/.pyenv/versions/2.7.4/envs/DeepCell/lib/python2.7/site-packages/numpy/lib/npyio.pyc in load(file, mmap_mode, allow_pickle, fix_imports, encoding)
360 own_fid = False
361 if isinstance(file, basestring):
--> 362 fid = open(file, "rb")
363 own_fid = True
364 else:
IOError: [Errno 2] No such file or directory: '/home/davince/DeepCell2/training_data_npz/3T3/3T3_all_61x61.npz'
But is there something i misunderstood?
I also have interest of doing cell tracking.
But there is another software lineage mapper from NIST seems could work as long there is a good segmentation mask (which is exactly what deepcell results looks so awesome to me)
Thanks
same setting for nuclei data, but bad results?
Hi, Valen~
I can get the pretty segmentation results using your trained model ( '2016-07-12_nuclei_all_61x61_bn_feature_net_61x61_'). But the model trained by using your code of 'training_template', the result is too bad. I also used your data (nulei_all_6161) and the model bn_feature_net_61*61. I noticed your trained model is 1.9M, while mine is 3.8M. What is the problems?
how to run your code?
hello, your code is big, but can you give some detailed introductions about how to run your code, including train and test.
thank you vry much!
disregard
Error in Training conv-nets
Hi, I was using your example code to run the DeepCell. However, I got error in the Training conv-nets .
It seems the error is from:
from model_zoo import bn_feature_net_61x61 as the_model
model = the_model(n_channels = 2, n_features = 3, reg = 1e-5)
ValueError: negative dimensions are not allowed
cannot get the output segmentation in the paper with the provided weight file
Hi,
with the provided weight file, I cannot get the output images shown in your PLOS paper. The prediction accuracy is pretty low.
Thanks
numpy can't read the sample training data (.npz)
I got this error when running DeepCell:
IOError: Failed to interpret file './train_data/HeLa_all_61x61.npz' as a pickle
I tried a few other training data sets provide in this repository and got the same error.
Is this a problem with the data set, or numpy version incompatibility? What numpy version was used to create the sample train data sets (.npz)?
Error when checking input: expected conv2d_1_input to have shape (None, 61, 61, 2)
Good evening,
In last days I have found your article about DeepCell and it is very interesting and useful, thus I wish to apply it in future in my research. Currently I am trying to get into it with data provided by you, however, I have a problem with training the cov-net. I constantly get an error:
Error when checking input: expected conv2d_22_input to have shape (None, 61, 61, 2) but got array with shape (58859, 2, 61, 61)
As far as I understood (probably), script expects to have a TensorFlow format (NWHC, N-samples, width, height, class), but gets Theano format (NCWH). Is this a correct conclusion? Because I am using your npz files, I didn't modify them at all, python files as well, I am clueless what to do.
I used "bn_feature_net_61x61" model on 3T3_all_61x61.npz file (Python 2.7, Ubuntu 16.04.3). Previously I was trying E. coli data as I am working with these organisms, but with 3T3 I progressed most.
I am hoping for your answer and advise.
Kindly regards,
Bartosz
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.