covertlab / deepcell

DeepCell2

deepcell's Issues

License?

Thank you for publishing this interesting and innovative paper.

The code published along looks like a good start for further customization or research, however, currently, I cannot spot licensing terms for the code.

It would be helpful, if you could add licensing terms (e.g. an open source license, if you have not yet decided which one, please choose carefully) to the repository.

Thank you, best regards,
Christian Sachs

Where do cnn_functions come from?

Trying to run your example scripts, but python cannot find cnn_functions. is that a dependency?

ImportError: No module named cnn_functions

running_template.py broken?

Great work.

Though when I do the running template for HeLa. The following code:

cytoplasm_predictions = run_models_on_directory(data_location, cyto_channel_names, cyto_location, model_fn = cyto_fn, 
    list_of_weights = list_of_cyto_weights, image_size_x = image_size_x, image_size_y = image_size_y, 
    win_x = win_cyto, win_y = win_cyto, std = False, split = False)

leads to the following runtime error:

theano.tensor.var.AsTensorError: ('Cannot convert Tensor("sparse_convolution2d_1_W/read:0", shape=(3, 3, 280, 64), dtype=float32) to TensorType', <class 'tensorflow.python.ops.variables.Variable'>)

Is there anything about the dimensionality I need to fix? Thanks.

Bug: "from keras import initializations" is outdated

The line

from keras import initializations as initializations

in keras_version/cnn_functions.py produces the error

ImportError: cannot import name initializations

in recent versions of keras (e.g. 2.0.2), as reported here.

simplified workflow/directory request

It is hard to know which scripts are meant to be demonstrative or are development scripts. Would it be possible to build a folder solely with working scripts for 1) making training sets, 2) training the CNNs and 3) execution. The scripts found here: https://covertlab.github.io/DeepCell/starting/ don't actually work out of the box. There are also duplicated scripts across keras_version and prototypes . It is hard to know if a bug is because of my system or because of the code. I know it is "buyer beware" but it would be a great help to have a baseline working set of scripts, one for each process for just one cell type as an introductory example. Thanks and thanks for making the code available.

TypeError: __call__() got an unexpected keyword argument 'name'

I follow the instruction from https://covertlab.github.io/DeepCell/running_convnets.html

and code below
(In[6])
cytoplasm_predictions = run_models_on_directory(data_location, cyto_channel_names, cyto_location, model_fn = cyto_fn, list_of_weights = list_of_cyto_weights, image_size_x = image_size_x, image_size_y = image_size_y, win_x = win_cyto, win_y = win_cyto, std = False, split = False)

always responsed with error below

Traceback (most recent call last):
File "test.py", line 52, in
win_x = win_cyto, win_y = win_cyto, std = False, split = False)
File "/home/user/DeepCell/keras_version/cnn_functions.py", line 1491, in run_models_on_directory
model = model_fn(batch_input_shape = batch_input_shape, n_features = n_features, weights_path = list_of_weights[0])
File "/home/user/DeepCell/keras_version/model_zoo.py", line 528, in sparse_bn_feature_net_61x61
model.add(sparse_Convolution2D(64, 3, 3, d = d, init = init, batch_input_shape = batch_input_shape, border_mode='valid', W_regularizer = l2(reg)))
File "/home/user/.pyenv/versions/DeepCell/lib/python2.7/site-packages/keras/models.py", line 436, in add
layer(x)
File "/home/user/.pyenv/versions/DeepCell/lib/python2.7/site-packages/keras/engine/topology.py", line 569, in __ call __
self.build(input_shapes[0])
File "/home/user/DeepCell/keras_version/cnn_functions.py", line 1012, in build
self.W = self.init(self.W_shape, name='{}_W'.format(self.name))
TypeError: __ call __() got an unexpected keyword argument 'name'

keras=2.0.6
python=2.7.4

running convnets example error

Hi Dave,
sorry I run into trouble again. i switched to using old version of keras (1.2)
I can run the running_template for theriot and give good results.
However when i'm trying to modify for it to run for segmentation on HeLa, 3T3, or MCF10A data set, I'm getting error.

I wonder if it's possible for ask for your kind help?

import os
import datetime
import numpy as np
os.environ["KERAS_BACKEND"]="theano"
os.environ["THEANO_FLAGS"]="mode=FAST_RUN,device=gpu,floatX=float32"
import keras
keras.backend.set_image_dim_ordering('th')

print 'keras version is ' + keras.__version__

import h5py
import tifffile as tiff
from keras.backend.common import _UID_PREFIXES

from cnn_functions import nikon_getfiles, get_image, run_models_on_directory, get_image_sizes, segment_nuclei, segment_cytoplasm, dice_jaccard_indices
from model_zoo import sparse_bn_feature_net_61x61 as cyto_fn
from model_zoo import sparse_bn_feature_net_61x61 as nuclear_fn

direc_name = '/home/davince/DeepCell-master/validation_data/HeLa/'
data_location = os.path.join(direc_name, 'RawImages')
cyto_location = os.path.join(direc_name, 'Cytoplasm')
nuclear_location = os.path.join(direc_name, 'Nuclear')
mask_location = os.path.join(direc_name, 'Masks')
cyto_channel_names = ['phase']
nuclear_channel_names = ['farred']

trained_network_cyto_directory = "/home/davince/DeepCell-master/trained_networks/HeLa/"
trained_network_nuclear_directory = "/home/davince/DeepCell-master/trained_networks/Nuclear/"

cyto_prefix = "2016-07-12_HeLa_all_61x61_bn_feature_net_61x61_"
nuclear_prefix = "2016-07-12_nuclei_all_61x61_bn_feature_net_61x61_"

win_cyto = 30
win_nuclear = 30

image_size_x, image_size_y = get_image_sizes(data_location, nuclear_channel_names)


list_of_cyto_weights = []
for j in xrange(5):
	cyto_weights = os.path.join(trained_network_cyto_directory,  cyto_prefix + str(j) + ".h5")
	list_of_cyto_weights += [cyto_weights]

list_of_nuclear_weights = []
for j in xrange(5):
	nuclear_weights = os.path.join(trained_network_nuclear_directory,  nuclear_prefix + str(j) + ".h5")
	list_of_nuclear_weights += [nuclear_weights]

cytoplasm_predictions = run_models_on_directory(data_location, cyto_channel_names, cyto_location, model_fn = cyto_fn, 
	list_of_weights = list_of_cyto_weights, image_size_x = image_size_x, image_size_y = image_size_y, 
	win_x = win_cyto, win_y = win_cyto, split = False)

nuclear_predictions = run_models_on_directory(data_location, nuclear_channel_names, nuclear_location, model_fn = nuclear_fn, 
	list_of_weights = list_of_nuclear_weights, image_size_x = image_size_x, image_size_y = image_size_y, 
	win_x = win_nuclear, win_y = win_nuclear, split = False)

and the error is like this

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-20-1bae99843176> in <module>()
     13 nuclear_predictions = run_models_on_directory(data_location, nuclear_channel_names, nuclear_location, model_fn = nuclear_fn, 
     14         list_of_weights = list_of_nuclear_weights, image_size_x = image_size_x, image_size_y = image_size_y,
---> 15 	win_x = win_nuclear, win_y = win_nuclear, split = False)

/home/davince/DeepCell-master/keras_version/cnn_functions.pyc in run_models_on_directory(data_location, channel_names, output_location, model_fn, list_of_weights, n_features, image_size_x, image_size_y, win_x, win_y, std, split, process, save)
   1482 
   1483         batch_input_shape = (1,len(channel_names),image_size_x+win_x, image_size_y+win_y)
-> 1484         model = model_fn(batch_input_shape = batch_input_shape, n_features = n_features, weights_path = list_of_weights[0])
   1485         n_features = model.layers[-1].output_shape[1]
   1486 

/home/davince/DeepCell-master/keras_version/model_zoo.pyc in sparse_bn_feature_net_61x61(batch_input_shape, n_features, reg, init, weights_path)
    567         model.add(Activation(tensorprod_softmax))
    568 
--> 569         model = set_weights(model, weights_path)
    570 
    571         return model

/home/davince/DeepCell-master/keras_version/cnn_functions.pyc in set_weights(model, weights_path)
     90 
     91         for layer in model.layers:
---> 92                 if layer.name in f['model_weights'].keys():
     93                         if 'bn' in layer.name:
     94                                 g = f['model_weights'][layer.name]

h5py/_objects.pyx in h5py._objects.with_phil.wrapper (/tmp/pip-4rPeHA-build/h5py/_objects.c:2684)()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper (/tmp/pip-4rPeHA-build/h5py/_objects.c:2642)()

/usr/local/lib/python2.7/dist-packages/h5py/_hl/group.pyc in __getitem__(self, name)
    164                 raise ValueError("Invalid HDF5 object reference")
    165         else:
--> 166             oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
    167 
    168         otype = h5i.get_type(oid)

h5py/_objects.pyx in h5py._objects.with_phil.wrapper (/tmp/pip-4rPeHA-build/h5py/_objects.c:2684)()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper (/tmp/pip-4rPeHA-build/h5py/_objects.c:2642)()

h5py/h5o.pyx in h5py.h5o.open (/tmp/pip-4rPeHA-build/h5py/h5o.c:3570)()

KeyError: "Unable to open object (Object 'model_weights' doesn't exist)"

But the weights seems to be loaded in the list_of_weights
is it the same problem you mentioned keras changed the way the weights are saved?
(now i reverted back to 1.2)
the h5py version is 2.6.0

Also, i know from your paper, you wrote the nuclear channel help segmentation.
Is this limited to semantic segmentation in coculture examples?
Or is it general for all experiment? Could the segmentation be also good if there is no nuclear channel?

Thank you.

error in file IO of a training data npz exactly where it is but can't find it.

Hi,
deepcell looks really really awesome and seems to be doing a great job segmenting touching cells. So I am really interested in doing cell tracking taken on phase contrast microscope.
I've been trying to do it in Unet and modified Segnet but the results haven't been very promising.

But when i'm just following your tutorial on training convnets notebook.
(I run the docker version. because trying to build has it's own problem from newer version of keras)
It shows a error trying to find the training data npz.
I have the file exactly where it is but it keep showing it cannot find the file.

IOErrorTraceback (most recent call last)
<ipython-input-12-bbf0fd3fe821> in <module>()
      8                 direc_data = direc_data,
      9                 lr_sched = lr_sched,
---> 10 		rotate = True, flip = True, shear = False)
     11 
     12         del model

/root/DeepCell/keras_version/cnn_functions.py in train_model_sample(model, dataset, optimizer, expt, it, batch_size, n_epoch, direc_save, direc_data, lr_sched, rotate, flip, shear, class_weight)
   1326         file_name_save_loss = os.path.join(direc_save, todays_date + "_" + dataset + "_" + expt + "_" + str(it) + ".npz")
   1327 
-> 1328         train_dict, (X_test, Y_test) = get_data_sample(training_data_file_name)
   1329 
   1330         # the data, shuffled and split between train and test sets

/root/DeepCell/keras_version/cnn_functions.py in get_data_sample(file_name)
    360 
    361 def get_data_sample(file_name):
--> 362         training_data = np.load(file_name)
    363         channels = training_data["channels"]
    364         batch = training_data["batch"]

/root/.pyenv/versions/2.7.4/envs/DeepCell/lib/python2.7/site-packages/numpy/lib/npyio.pyc in load(file, mmap_mode, allow_pickle, fix_imports, encoding)
    360     own_fid = False
    361     if isinstance(file, basestring):
--> 362         fid = open(file, "rb")
    363         own_fid = True
    364     else:

IOError: [Errno 2] No such file or directory: '/home/davince/DeepCell2/training_data_npz/3T3/3T3_all_61x61.npz'

But is there something i misunderstood?

I also have interest of doing cell tracking.
But there is another software lineage mapper from NIST seems could work as long there is a good segmentation mask (which is exactly what deepcell results looks so awesome to me)
Thanks

same setting for nuclei data, but bad results?

Hi, Valen~
I can get the pretty segmentation results using your trained model ( '2016-07-12_nuclei_all_61x61_bn_feature_net_61x61_'). But the model trained by using your code of 'training_template', the result is too bad. I also used your data (nulei_all_6161) and the model bn_feature_net_61*61. I noticed your trained model is 1.9M, while mine is 3.8M. What is the problems?

how to run your code?

hello, your code is big, but can you give some detailed introductions about how to run your code, including train and test.

thank you vry much!

disregard

Error in Training conv-nets

Hi, I was using your example code to run the DeepCell. However, I got error in the Training conv-nets .

It seems the error is from:

 from model_zoo import bn_feature_net_61x61 as the_model
 model = the_model(n_channels = 2, n_features = 3, reg = 1e-5)

ValueError: negative dimensions are not allowed

cannot get the output segmentation in the paper with the provided weight file

Hi,

with the provided weight file, I cannot get the output images shown in your PLOS paper. The prediction accuracy is pretty low.

Thanks

numpy can't read the sample training data (.npz)

I got this error when running DeepCell:
IOError: Failed to interpret file './train_data/HeLa_all_61x61.npz' as a pickle
I tried a few other training data sets provide in this repository and got the same error.
Is this a problem with the data set, or numpy version incompatibility? What numpy version was used to create the sample train data sets (.npz)?

Error when checking input: expected conv2d_1_input to have shape (None, 61, 61, 2)

Good evening,

In last days I have found your article about DeepCell and it is very interesting and useful, thus I wish to apply it in future in my research. Currently I am trying to get into it with data provided by you, however, I have a problem with training the cov-net. I constantly get an error:

Error when checking input: expected conv2d_22_input to have shape (None, 61, 61, 2) but got array with shape (58859, 2, 61, 61)

As far as I understood (probably), script expects to have a TensorFlow format (NWHC, N-samples, width, height, class), but gets Theano format (NCWH). Is this a correct conclusion? Because I am using your npz files, I didn't modify them at all, python files as well, I am clueless what to do.

I used "bn_feature_net_61x61" model on 3T3_all_61x61.npz file (Python 2.7, Ubuntu 16.04.3). Previously I was trying E. coli data as I am working with these organisms, but with 3T3 I progressed most.

I am hoping for your answer and advise.

Kindly regards,
Bartosz