iyah4888 / siggraph18sss Goto Github PK

Hi,
when I ran the code, it came up with this result:

Traceback (most recent call last):
  File "main_hyper.py", line 25, in <module>
    from deeplab_resnet import HyperColumn_Deeplabv2, read_data_list
  File "/Users/xxx/Desktop/machine learning/SIGGRAPH18SSS-master/deeplab_resnet/__init__.py", line 2, in <module>
    from .hc_deeplab import HyperColumn_Deeplabv2
  File "/Users/xxx/Desktop/machine learning/SIGGRAPH18SSS-master/deeplab_resnet/hc_deeplab.py", line 15, in <module>
    from tensorflow.python.keras._impl.keras.initializers import he_normal
ModuleNotFoundError: No module named 'tensorflow.python.keras._impl'

any idea?

Cloned only this repository and downloaded the pre-trained weights from the link mentioned in the repository. Added a sample image and ran the bash script from terminal and got this:

RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
return f(args, **kwds)
[] Loading checkpoints...
[!] Load failed...
0 Processing ./samples/input_110.png
Traceback (most recent call last):
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1323, in _do_call
return fn(*args)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1302, in _run_fn
status, run_metadata)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value bn4a_branch2a/moving_variance
[[Node: bn4a_branch2a/moving_variance/read = IdentityT=DT_FLOAT, _class=["loc:@bn4a_branch2a/moving_variance"], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "main_hyper.py", line 117, in
cur_embed = model.test(pad_img.eval())
File "/home/saurabh/Documents/code/soft_segmentation/SIGGRAPH18SSS/deeplab_resnet/hc_deeplab.py", line 346, in test
embedmap = self.sess.run(self.netcontainer['out_hypcol'], feed_dict=feed)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1120, in _run
feed_dict_tensor, options, run_metadata)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run
options, run_metadata)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value bn4a_branch2a/moving_variance
[[Node: bn4a_branch2a/moving_variance/read = IdentityT=DT_FLOAT, _class=["loc:@bn4a_branch2a/moving_variance"], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

Caused by op 'bn4a_branch2a/moving_variance/read', defined at:
File "main_hyper.py", line 98, in
model = HyperColumn_Deeplabv2(sess, args)
File "/home/saurabh/Documents/code/soft_segmentation/SIGGRAPH18SSS/deeplab_resnet/hc_deeplab.py", line 90, in init
self.build_model()
File "/home/saurabh/Documents/code/soft_segmentation/SIGGRAPH18SSS/deeplab_resnet/hc_deeplab.py", line 143, in build_model
net = DeepLabResNetModel({'data': input_img}, is_training=self.args.is_training, num_classes=self.args.num_classes)
File "/home/saurabh/Documents/code/soft_segmentation/SIGGRAPH18SSS/kaffe/tensorflow/network.py", line 48, in init
self.setup(is_training, num_classes)
File "/home/saurabh/Documents/code/soft_segmentation/SIGGRAPH18SSS/deeplab_resnet/model.py", line 113, in setup
.batch_normalization(is_training=is_training, activation_fn=tf.nn.relu, name='bn4a_branch2a')
File "/home/saurabh/Documents/code/soft_segmentation/SIGGRAPH18SSS/kaffe/tensorflow/network.py", line 22, in layer_decorated
layer_output = op(self, layer_input, *args, **kwargs)
File "/home/saurabh/Documents/code/soft_segmentation/SIGGRAPH18SSS/kaffe/tensorflow/network.py", line 269, in batch_normalization
scope=scope)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 181, in func_with_args
return func(*args, **current_args)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/contrib/layers/python/layers/layers.py", line 592, in batch_norm
scope=scope)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/contrib/layers/python/layers/layers.py", line 384, in _fused_batch_norm
collections=moving_variance_collections)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 181, in func_with_args
return func(*args, **current_args)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/contrib/framework/python/ops/variables.py", line 262, in model_variable
use_resource=use_resource)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 181, in func_with_args
return func(*args, **current_args)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/contrib/framework/python/ops/variables.py", line 217, in variable
use_resource=use_resource)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1203, in get_variable
constraint=constraint)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1092, in get_variable
constraint=constraint)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 425, in get_variable
constraint=constraint)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 394, in _true_getter
use_resource=use_resource, constraint=constraint)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 805, in _get_single_variable
constraint=constraint)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 213, in init
constraint=constraint)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 356, in _init_from_args
self._snapshot = array_ops.identity(self._variable, name="read")
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 125, in identity
return gen_array_ops.identity(input, name=name)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 2071, in identity
"Identity", input=input, name=name)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)
File "/home/saurabh/anaconda3/envs/soft_seg/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1470, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

FailedPreconditionError (see above for traceback): Attempting to use uninitialized value bn4a_branch2a/moving_variance
[[Node: bn4a_branch2a/moving_variance/read = IdentityT=DT_FLOAT, _class=["loc:@bn4a_branch2a/moving_variance"], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

When trying to run main_hyper.py python main_hyper.py --data-dir ./samples got error TypeError: __call__() got an unexpected keyword argument 'partition_info'

System:
Windows 10 x64
TensorFlow 1.10 with GPU

Stack trace:

Traceback (most recent call last):
  File "main_hyper.py", line 96, in <module>
    model = HyperColumn_Deeplabv2(sess, args)
  File "D:\LLDevelopment\Programming\Python\SIGGRAPH18SSS\deeplab_resnet\hc_deeplab.py", line 90, in __init__
    self.build_model()
  File "D:\LLDevelopment\Programming\Python\SIGGRAPH18SSS\deeplab_resnet\hc_deeplab.py", line 163, in build_model
    full_act1 = tf.nn.relu(lowrank_linear(full_activation, 256, 512, name="linear"))
  File "D:\LLDevelopment\Programming\Python\SIGGRAPH18SSS\deeplab_resnet\hc_deeplab.py", line 40, in lowrank_linear
    weights1 = tf.get_variable("fc_weights1", [input_.get_shape()[-1], dim_bottle], initializer=he_normal())
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 1328, in get_variable
    constraint=constraint)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 1090, in get_variable
    constraint=constraint)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 435, in get_variable
    constraint=constraint)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 404, in _true_getter
    use_resource=use_resource, constraint=constraint)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 796, in _get_single_variable
    use_resource=use_resource)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 2234, in variable
    use_resource=use_resource)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 2224, in <lambda>
    previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 2207, in default_variable_creator
    constraint=constraint)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\variables.py", line 259, in __init__
    constraint=constraint)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\variables.py", line 368, in _init_from_args
    initial_value(), name="initial_value", dtype=dtype)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 780, in <lambda>
    shape.as_list(), dtype=dtype, partition_info=partition_info)
TypeError: __call__() got an unexpected keyword argument 'partition_info'

How can we fix it?

Hi,
I follow your guide and run the model to predict './samples/docia.png', but i get a out of memory error ...
The error message says there is no free memory , and the memory does be exhausted when I monitor GPU status.
I use Nvidia GTX 1060 6G with ubuntu 18.04 .
If possible , can you tell me how many memorys do you use for predict?
thanks.

How to run “run_extract_feat.sh” file in Windows 10？

The new link of pre-trained model seems not to be working anymore, Can you provide a new link?

In the paper,
Fig. 12. Our soft segments and the corresponding mattes for the foreground objects.

How to generate mattes for the foreground objects?

How to transform mat into PNG? How to reduce dimensions?

Hi,

I followed your installation tutorial in the readme and I got the 128-dimensional feature vector per pixel for a given image. But I can not find the code for filtering each of the 128 dimensions with guided filter and applying the dimensionality reduction with PCA to 3 in order to get the segmentation mask.

Is it possible that you upload this code? and could you please give more details about how to get each segmented object (mask) from this feature vector?

Thanks!

Hi @iyah4888, from my understanding, the shapes of semantic features are irregular. How to obtain the feature vectors of an irregular region? Thanks.

Hi,
I successfully run the model and get a 128-D features, but such many features really make me confused, what should I do to choose a better feature,and how do I do to extract the detected object with white background.
Thanks

Hi,
I have this error as below:

$ CUDA_VISIBLE_DEVICES=0 python main_hyper.py --data-dir ./samples
2018-10-10 10:11:47.248664: I C:\tf_jenkins\home\workspace\rel-win\M\windows\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
 [*] Loading checkpoints...
 [!] Load failed...
0 Processing ./samples\docia.png
Traceback (most recent call last):
  File "main_hyper.py", line 113, in <module>
    _, ori_img = read_img(local_imgflist[i], input_size = None, img_mean = IMG_MEAN)
  File "main_hyper.py", line 79, in read_img
    img2 = tf.image.resize_images(img, tf.shape(img)[0:2,]*2)
  File "D:\Python36\lib\site-packages\tensorflow\python\ops\image_ops_impl.py", line 741, in resize_images
    raise ValueError('\'images\' contains no shape.')
ValueError: 'images' contains no shape.

I'm not sure why, but glob seems to return all jpg files as well (despite mentioning *.png at the end) and I was able to generate feature vectors using it.

Just thought I'd mention it here. Maybe one can remove that comment which says jpg not supported.

Hi,
I used your semantic feature generation code to generate the features of the images 'docia.png'. But I am facing difference in the results. Please see the images below and guide me if I am doing anything wrong. You can see the difference in the features generated. After getting the feature vector I am feeding that vector to 'preprocessFeatures.m' file. Is there something else that I need to do?

In this picture I generated the feature through the code and then fed the image as input in demo.m file.

This result is generated by the image you provided as docia.png

tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value fc3_matting/linear/fc_weights1. when I try to restore the net by the pretrained model I got this error. How can i fix it? Thank you!

After the setup and all requirements met, running ./run_extract_feat.sh may produce such error in Linux environment.

SyntaxError: Non-ASCII character '\xc4' in file /SIGGRAPH18SSS-master/deeplab_resnet/hc_deeplab.py on line 7, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

In your paper , the train datasets is COCO-Stuff datasets, but I want to use other datasets to train the model , and test the results of the model ,what can I do ?

the mat file how to visualizate ？

Thanks for making the code publicly available.
I would like to test this model, but the link to the pre-trained model seems to be broken.

I'm looking forward to hearing from you.

After generating the 128 dim feature vector. How to proceed to remove background alpha from the image, I just need some direction. After that I have been successful in generating the segmentation using demo.png but still confused if its using the docia.mat file or docia.png file to generate segmentation.

Please give a direction to proceed in order to remove the background alphas from this image.