foamliu / deep-image-matting

Deep Image Matting

Just in case you are interested, Deep Image Matting v2 is an upgraded version of this.

Deep Image Matting

This repository is to reproduce Deep Image Matting.

Dependencies

• NumPy
• Tensorflow 1.9.0
• Keras 2.1.6
• OpenCV

Dataset

Adobe Deep Image Matting Dataset

Follow the instruction to contact author for the dataset.

MSCOCO

Go to MSCOCO to download:

• 2014 Train images

PASCAL VOC

Go to PASCAL VOC to download:

• VOC challenge 2008 training/validation data
• The test data for the VOC2008 challenge

ImageNet Pretrained Models

Download VGG16 into "models" folder.

Usage

Data Pre-processing

Extract training images:

$ python pre_process.py

Train

$ python train.py

If you want to visualize during training, run in your terminal:

$ tensorboard --logdir path_to_current_dir/logs

Demo

Download pre-trained Deep Image Matting Model to "models" folder then run:

$ python demo.py

Image/Trimap	Output/GT	New BG/Compose

deep-image-matting's Issues

Question about loading weights in demo.py

Hi Mr. Liu,

Thanks for sharing the implementation of DIM. I met an issue when loading the weights (trained in an 8 GPUs machine) in demo.py. Seems the checkpoint has several layers more than the model built, saying 'ValueError: You are trying to load a weight file containing 47 layers into a model with 33 layers.' Reaaly appreciate it if you please hint how to deal with this error.

Thanks

Training is very slow with GPU utility

Hello Yang,

Thanks for your efforts to implement this. I found training process is quite slow and GPU util percent is quite low (0% during a large portion of each iteration). Is this the same case in your training?

BTW, can this code reproduce the claimed performance in the paper?

Thanks

Doubt about alpha and trimap

首先，非常感谢您提供了代码实现了这篇论文。关于最后的结果我有一些疑问。
我们这篇文章的输入虽然是trimap，但是trimap的输入依赖于alpha.也就是说首先我们必须提供alpha，是这个意思吗？
最后的输出结果out的损失是和提供的alpha对比吗？也就是代码中的真实值其实就是我们提供的alpha吗？
First of all, thank you very much for providing the code to implement this paper. I have some questions about the final result.

The input of this article is trimap, but the input of trimap depends on alpha. Is the loss of the final output out compared to the provided alpha? Is the GT in the code actually the alpha that we provide?

Failed to get convolution algorithm

When execute "python demo.py"
Tensorflow-gpu - 1.13
But working with tensorflow cpu

Crash on -> y_pred = final.predict(x_test)

tensorflow.python.framework.errors_impl.UnknownError: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[{{node conv1_1/convolution}}]]

question about "custom_loss_wrapper"

Hi，Thanks for sharing your source code! Here is my question:

• There are several lines of code in the train_refinement.py script：

from utils import custom_loss_wrapper, get_available_cpus
# ...
refinement.compile(optimizer='nadam', loss=custom_loss_wrapper(refinement.input))

• My question: where is the custom_loss_wrapper function in utils.py ? It doesn't seem to exist in utils.py.

Waiting for your reply, thank you！

integer argument expected, got float

hello,thanks for your coder,but there is an error when I train the modle. python train.py
integer argument expected, got float

How long time have you trained for converege

Question just like the title.
Thanks very much

train higher resolution image

Thanks for you work!
I want to know if it is possible to train 800*600 image ? If it is possible , how can I change the code?

Unexpected inference result on custom dataset

Hi,

I used the pretrained weights to test on my own building dataset. Here are some examples of input patches, trimaps and corresponding predictions.

Even if the pretrained model never fine-tuned on my dataset, I didn't expect its prediction to have such jagged edges (which shapes like the gray part in the trimaps). What could have gone wrong?

How to preduce the trimap?

I do some test but didn't get result as good as the images shown in the README.md.
I think trimap have a huge impact to the result. I use Mask RCNN to generator segmentation and than the trimap.
Could you tell how do you produce the trimap in the README.md demo images ?
Thanks!

AttributeError: 'NoneType' object has no attribute 'shape'

AttributeError: 'NoneType' object has no attribute 'shape'

FileNotFoundError

@foamliu-ms
when I run pre-process.py ,report the error:FileNotFoundError: [Errno 2] No such file or directory: 'Adobe_Deep_Matting_Dataset.zip'. Where can I download this data set of "Adobe_Deep_Matting_Dataset.zip"

when I run demo.py ,report a error: FileNotFoundError: [Errno 2] No such file or directory: 'Combined_Dataset/Training_set/training_fg_names.txt' , I don't know where to find the "training_fg_name.txt" file. In my download Training_set dir don't have "training_fg_names.txt" file.

Can you help me ? Thanks

loss computational error

Hi，Thanks for sharing your source code! Here is my question:
K.sum(K.sqrt(K.square(diff) + epsilon_sqr)) / (num_pixels + epsilon) is equal to （diff_1+diff_2+..+diff_n）/（mask_1+mask_2+...+diff_n) is it right?
but I think the real way to do this is [(diff_1/mask_1)+(diff_2/mask_2)+..+(diff_n/mask_n)]/n

The decoder network is hard to understand

Hi, can you explain the decoder part of the encoder-decoder network? It's very different from the common decoder network.
For example, block like this is hard to understand:

x = Conv2D(512, (1, 1), activation='relu',padding='same',name='deconv6',
kernel_initializer='he_normal',bias_initializer='zeros')(x)                       
x = BatchNormalization()(x)                                   
x = UpSampling2D(size=(2, 2))(x)                               
the_shape = K.int_shape(orig_5)                               
shape = (1, the_shape[1], the_shape[2], the_shape[3])         
origReshaped = Reshape(shape)(orig_5)                         
# print('origReshaped.shape: ' + str(K.int_shape(origReshaped)))
xReshaped = Reshape(shape)(x)                                  
# print('xReshaped.shape: ' + str(K.int_shape(xReshaped)))
together = Concatenate(axis=1)([origReshaped, xReshaped])          
x = Unpooling()(together)          

I thik it's very different from the structure in paper, can you expalain it?
Thank you very much!

Error is training_names.txt

Do you miss some files to upload?

can't get the pretrained model

sorry for bothering, I can't download the pre-trained model in the provided link……
could you please share the pre-trained model to me? such as baiduyun,etc.

Training errors?

Hi, and thank you for making this code available. Is it fast enough for real time use? How long does the function take to return an alpha from the trimap/colour input?

I am trying to train some data, and am getting the following error:

python train.py Using TensorFlow backend. 2019-04-25 16:09:02.261034: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 WARNING:tensorflow:From C:\Users\bsharp\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\framework\op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer. WARNING:tensorflow:From C:\Python37\lib\site-packages\keras\backend\tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version. Instructions for updating: Please use rateinstead ofkeep_prob. Rate should be set to rate = 1 - keep_prob`.

---

Layer (type) Output Shape Param # Connected to

input_1 (InputLayer) (None, 320, 320, 4) 0

---

zero_padding2d_1 (ZeroPadding2D (None, 322, 322, 4) 0 input_1[0][0]

---

conv1_1 (Conv2D) (None, 320, 320, 64) 2368 zero_padding2d_1[0][0]

---

zero_padding2d_2 (ZeroPadding2D (None, 322, 322, 64) 0 conv1_1[0][0]

---

conv1_2 (Conv2D) (None, 320, 320, 64) 36928 zero_padding2d_2[0][0]

---

max_pooling2d_1 (MaxPooling2D) (None, 160, 160, 64) 0 conv1_2[0][0]

---

zero_padding2d_3 (ZeroPadding2D (None, 162, 162, 64) 0 max_pooling2d_1[0][0]

---

conv2_1 (Conv2D) (None, 160, 160, 128 73856 zero_padding2d_3[0][0]

---

zero_padding2d_4 (ZeroPadding2D (None, 162, 162, 128 0 conv2_1[0][0]

---

conv2_2 (Conv2D) (None, 160, 160, 128 147584 zero_padding2d_4[0][0]

---

max_pooling2d_2 (MaxPooling2D) (None, 80, 80, 128) 0 conv2_2[0][0]

---

zero_padding2d_5 (ZeroPadding2D (None, 82, 82, 128) 0 max_pooling2d_2[0][0]

---

conv3_1 (Conv2D) (None, 80, 80, 256) 295168 zero_padding2d_5[0][0]

---

zero_padding2d_6 (ZeroPadding2D (None, 82, 82, 256) 0 conv3_1[0][0]

---

conv3_2 (Conv2D) (None, 80, 80, 256) 590080 zero_padding2d_6[0][0]

---

zero_padding2d_7 (ZeroPadding2D (None, 82, 82, 256) 0 conv3_2[0][0]

---

conv3_3 (Conv2D) (None, 80, 80, 256) 590080 zero_padding2d_7[0][0]

---

max_pooling2d_3 (MaxPooling2D) (None, 40, 40, 256) 0 conv3_3[0][0]

---

zero_padding2d_8 (ZeroPadding2D (None, 42, 42, 256) 0 max_pooling2d_3[0][0]

---

conv4_1 (Conv2D) (None, 40, 40, 512) 1180160 zero_padding2d_8[0][0]

---

zero_padding2d_9 (ZeroPadding2D (None, 42, 42, 512) 0 conv4_1[0][0]

---

conv4_2 (Conv2D) (None, 40, 40, 512) 2359808 zero_padding2d_9[0][0]

---

zero_padding2d_10 (ZeroPadding2 (None, 42, 42, 512) 0 conv4_2[0][0]

---

conv4_3 (Conv2D) (None, 40, 40, 512) 2359808 zero_padding2d_10[0][0]

---

max_pooling2d_4 (MaxPooling2D) (None, 20, 20, 512) 0 conv4_3[0][0]

---

zero_padding2d_11 (ZeroPadding2 (None, 22, 22, 512) 0 max_pooling2d_4[0][0]

---

conv5_1 (Conv2D) (None, 20, 20, 512) 2359808 zero_padding2d_11[0][0]

---

zero_padding2d_12 (ZeroPadding2 (None, 22, 22, 512) 0 conv5_1[0][0]

---

conv5_2 (Conv2D) (None, 20, 20, 512) 2359808 zero_padding2d_12[0][0]

---

zero_padding2d_13 (ZeroPadding2 (None, 22, 22, 512) 0 conv5_2[0][0]

---

conv5_3 (Conv2D) (None, 20, 20, 512) 2359808 zero_padding2d_13[0][0]

---

max_pooling2d_5 (MaxPooling2D) (None, 10, 10, 512) 0 conv5_3[0][0]

---

up_sampling2d_1 (UpSampling2D) (None, 20, 20, 512) 0 max_pooling2d_5[0][0]

---

reshape_1 (Reshape) (None, 1, 20, 20, 51 0 conv5_3[0][0]

---

reshape_2 (Reshape) (None, 1, 20, 20, 51 0 up_sampling2d_1[0][0]

---

concatenate_1 (Concatenate) (None, 2, 20, 20, 51 0 reshape_1[0][0]
reshape_2[0][0]

---

unpooling_1 (Unpooling) (None, 20, 20, 512) 0 concatenate_1[0][0]

---

deconv5_1 (Conv2D) (None, 20, 20, 512) 2359808 unpooling_1[0][0]

---

batch_normalization_1 (BatchNor (None, 20, 20, 512) 2048 deconv5_1[0][0]

---

deconv5_2 (Conv2D) (None, 20, 20, 512) 2359808 batch_normalization_1[0][0]

---

batch_normalization_2 (BatchNor (None, 20, 20, 512) 2048 deconv5_2[0][0]

---

deconv5_3 (Conv2D) (None, 20, 20, 512) 2359808 batch_normalization_2[0][0]

---

batch_normalization_3 (BatchNor (None, 20, 20, 512) 2048 deconv5_3[0][0]

---

up_sampling2d_2 (UpSampling2D) (None, 40, 40, 512) 0 batch_normalization_3[0][0]

---

reshape_3 (Reshape) (None, 1, 40, 40, 51 0 conv4_3[0][0]

---

reshape_4 (Reshape) (None, 1, 40, 40, 51 0 up_sampling2d_2[0][0]

---

concatenate_2 (Concatenate) (None, 2, 40, 40, 51 0 reshape_3[0][0]
reshape_4[0][0]

---

unpooling_2 (Unpooling) (None, 40, 40, 512) 0 concatenate_2[0][0]

---

deconv4_1 (Conv2D) (None, 40, 40, 256) 1179904 unpooling_2[0][0]

---

batch_normalization_4 (BatchNor (None, 40, 40, 256) 1024 deconv4_1[0][0]

---

deconv4_2 (Conv2D) (None, 40, 40, 256) 590080 batch_normalization_4[0][0]

---

batch_normalization_5 (BatchNor (None, 40, 40, 256) 1024 deconv4_2[0][0]

---

deconv4_3 (Conv2D) (None, 40, 40, 256) 590080 batch_normalization_5[0][0]

---

batch_normalization_6 (BatchNor (None, 40, 40, 256) 1024 deconv4_3[0][0]

---

up_sampling2d_3 (UpSampling2D) (None, 80, 80, 256) 0 batch_normalization_6[0][0]

---

reshape_5 (Reshape) (None, 1, 80, 80, 25 0 conv3_3[0][0]

---

reshape_6 (Reshape) (None, 1, 80, 80, 25 0 up_sampling2d_3[0][0]

---

concatenate_3 (Concatenate) (None, 2, 80, 80, 25 0 reshape_5[0][0]
reshape_6[0][0]

---

unpooling_3 (Unpooling) (None, 80, 80, 256) 0 concatenate_3[0][0]

---

deconv3_1 (Conv2D) (None, 80, 80, 128) 295040 unpooling_3[0][0]

---

batch_normalization_7 (BatchNor (None, 80, 80, 128) 512 deconv3_1[0][0]

---

deconv3_2 (Conv2D) (None, 80, 80, 128) 147584 batch_normalization_7[0][0]

---

batch_normalization_8 (BatchNor (None, 80, 80, 128) 512 deconv3_2[0][0]

---

deconv3_3 (Conv2D) (None, 80, 80, 128) 147584 batch_normalization_8[0][0]

---

batch_normalization_9 (BatchNor (None, 80, 80, 128) 512 deconv3_3[0][0]

---

up_sampling2d_4 (UpSampling2D) (None, 160, 160, 128 0 batch_normalization_9[0][0]

---

reshape_7 (Reshape) (None, 1, 160, 160, 0 conv2_2[0][0]

---

reshape_8 (Reshape) (None, 1, 160, 160, 0 up_sampling2d_4[0][0]

---

concatenate_4 (Concatenate) (None, 2, 160, 160, 0 reshape_7[0][0]
reshape_8[0][0]

---

unpooling_4 (Unpooling) (None, 160, 160, 128 0 concatenate_4[0][0]

---

deconv2_1 (Conv2D) (None, 160, 160, 64) 73792 unpooling_4[0][0]

---

batch_normalization_10 (BatchNo (None, 160, 160, 64) 256 deconv2_1[0][0]

---

deconv2_2 (Conv2D) (None, 160, 160, 64) 36928 batch_normalization_10[0][0]

---

batch_normalization_11 (BatchNo (None, 160, 160, 64) 256 deconv2_2[0][0]

---

up_sampling2d_5 (UpSampling2D) (None, 320, 320, 64) 0 batch_normalization_11[0][0]

---

reshape_9 (Reshape) (None, 1, 320, 320, 0 conv1_2[0][0]

---

reshape_10 (Reshape) (None, 1, 320, 320, 0 up_sampling2d_5[0][0]

---

concatenate_5 (Concatenate) (None, 2, 320, 320, 0 reshape_9[0][0]
reshape_10[0][0]

---

unpooling_5 (Unpooling) (None, 320, 320, 64) 0 concatenate_5[0][0]

---

deconv1_1 (Conv2D) (None, 320, 320, 64) 36928 unpooling_5[0][0]

---

batch_normalization_12 (BatchNo (None, 320, 320, 64) 256 deconv1_1[0][0]

---

deconv1_2 (Conv2D) (None, 320, 320, 64) 36928 batch_normalization_12[0][0]

---

batch_normalization_13 (BatchNo (None, 320, 320, 64) 256 deconv1_2[0][0]

---

lambda_11 (Lambda) (None, 320, 320, 3) 0 input_1[0][0]

---

pred (Conv2D) (None, 320, 320, 1) 577 batch_normalization_13[0][0]

---

concatenate_6 (Concatenate) (None, 320, 320, 4) 0 lambda_11[0][0]
pred[0][0]

---

conv2d_1 (Conv2D) (None, 320, 320, 64) 2368 concatenate_6[0][0]

---

batch_normalization_14 (BatchNo (None, 320, 320, 64) 256 conv2d_1[0][0]

---

conv2d_2 (Conv2D) (None, 320, 320, 64) 36928 batch_normalization_14[0][0]

---

batch_normalization_15 (BatchNo (None, 320, 320, 64) 256 conv2d_2[0][0]

---

conv2d_3 (Conv2D) (None, 320, 320, 64) 36928 batch_normalization_15[0][0]

---

batch_normalization_16 (BatchNo (None, 320, 320, 64) 256 conv2d_3[0][0]

---

refinement_pred (Conv2D) (None, 320, 320, 1) 577 batch_normalization_16[0][0]

Total params: 25,019,458
Trainable params: 25,013,186
Non-trainable params: 6,272

---

None
Epoch 1/1000
Using TensorFlow backend.
Using TensorFlow backend.
Using TensorFlow backend.
Using TensorFlow backend.
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "C:\Python37\lib\multiprocessing\pool.py", line 121, in worker
result = (True, func(*args, **kwds))
File "C:\Python37\lib\site-packages\keras\utils\data_utils.py", line 401, in get_index
return _SHARED_SEQUENCES[uid][i]
File "X:\dev\sandbox_bsharp\ML\Deep-Image-Matting\data_generator.py", line 123, in getitem
im_name = fg_files[fcount]
IndexError: list index out of range
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "train.py", line 79, in
workers=2
File "C:\Python37\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "C:\Python37\lib\site-packages\keras\engine\training.py", line 1418, in fit_generator
initial_epoch=initial_epoch)
File "C:\Python37\lib\site-packages\keras\engine\training_generator.py", line 181, in fit_generator
generator_output = next(output_generator)
File "C:\Python37\lib\site-packages\keras\utils\data_utils.py", line 601, in get
six.reraise(*sys.exc_info())
File "C:\Python37\lib\site-packages\six.py", line 693, in reraise
raise value
File "C:\Python37\lib\site-packages\keras\utils\data_utils.py", line 595, in get
inputs = self.queue.get(block=True).get()
File "C:\Python37\lib\multiprocessing\pool.py", line 657, in get
raise self._value
IndexError: list index out of range
`
What am i doing wrong?
Thanks!

Why is the result pixel not 0 or 1?

how to run this project?

i can not run the project.what should i do?
when i run pre_process.py ,it will show " No module named 'tqdm' "

unable to run demo

Hello, I'm trying to run the demo with the pre-trained model. I downloaded it and moved it inside a models/ directory, but I get this error.

python demo.py

Using TensorFlow backend.
Traceback (most recent call last):
  File "demo.py", line 9, in <module>
    from data_generator import generate_trimap, random_choice, get_alpha_test
  File "/Users/andy/Documents/projects/Deep-Image-Matting/data_generator.py", line 17, in <module>
    with open('Combined_Dataset/Training_set/training_fg_names.txt') as f:
IOError: [Errno 2] No such file or directory: 'Combined_Dataset/Training_set/training_fg_names.txt'

BrokenPipeError: [Errno 32] Broken pipe

Hi, I Rebuild the Dataset But when I want to Train the model this happened:

Epoch 1/1000
Exception in thread Thread-9:
Traceback (most recent call last):
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\site-packages\keras\utils\data_utils.py", line 565, in _run
with closing(self.executor_fn(_SHARED_SEQUENCES)) as executor:
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\site-packages\keras\utils\data_utils.py", line 548, in
initargs=(seqs,))
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\multiprocessing\context.py", line 119, in Pool
context=self.get_context())
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\multiprocessing\pool.py", line 174, in init
self._repopulate_pool()
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\multiprocessing\pool.py", line 239, in _repopulate_pool
w.start()
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\multiprocessing\popen_spawn_win32.py", line 65, in init
reduction.dump(process_obj, to_child)
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
BrokenPipeError: [Errno 32] Broken pipe
Exception in thread Thread-8:
Traceback (most recent call last):
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\site-packages\keras\utils\data_utils.py", line 565, in _run
with closing(self.executor_fn(_SHARED_SEQUENCES)) as executor:
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\site-packages\keras\utils\data_utils.py", line 548, in
initargs=(seqs,))
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\multiprocessing\context.py", line 119, in Pool
context=self.get_context())
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\multiprocessing\pool.py", line 174, in init
self._repopulate_pool()
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\multiprocessing\pool.py", line 239, in _repopulate_pool
w.start()
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\multiprocessing\popen_spawn_win32.py", line 65, in init
reduction.dump(process_obj, to_child)
File "C:\Users\Bobykhani\Anaconda2\envs\tensorflow\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
BrokenPipeError: [Errno 32] Broken pipe

Any Solution?
thx

Ask

annotate image

Hi, I am a newbie, I want to know what annotation tool do we need to annotate images for Deep Image Matting?

FileNotFoundError: [Errno 2] No such file or directory: 'data/VOC2008test.tar'

Hi, when I run pre_process.py meet a FileNotFoundError as fallow:

(tensorflow) ruichengxie@amax:~/jupyter2/segmentation/Deep-Image-Matting$ python pre_process.py
Extracting data/Adobe_Deep_Matting_Dataset.zip...
Extracting data/train2014.zip...
Extracting data/VOCtrainval_14-Jul-2008.tar...
Extracting data/VOC2008test.tar...
Traceback (most recent call last):
  File "pre_process.py", line 97, in <module>
    tar = tarfile.open(tar_file)
  File "/home/ruichengxie/.conda/envs/tensorflow/lib/python3.6/tarfile.py", line 1568, in open
    return func(name, "r", fileobj, **kwargs)
  File "/home/ruichengxie/.conda/envs/tensorflow/lib/python3.6/tarfile.py", line 1633, in gzopen
    fileobj = gzip.GzipFile(name, mode + "b", compresslevel, fileobj)
  File "/home/ruichengxie/.conda/envs/tensorflow/lib/python3.6/gzip.py", line 163, in __init__
    fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'data/VOC2008test.tar'

The question may cause by trying to open an unexist file,

Deep-Image-Matting/pre_process.py

Lines 94 to 99 in a5ddd97

	tar_file = 'data/VOC2008test.tar'
	print('Extracting {}...'.format(tar_file))
	tar = tarfile.open(tar_file)
	tar.extractall('data')
	tar.close()

Unfortunately, I couldn't find the download link of this file in README or PASCAL VOC website. Thus, I comment those lines, and the code seems can ran.

Is VOC2008test.tar necessary? Where can I get this file?

Thx.

Can we test on arbitrary size images?

I noticed that test.py assumes that the input image size is 320 x 320 pixels. How can I test on an arbitrary image size without the need for cropping or resizing before inputting it into the model?

Ask for metrics code

Hello,
sorry for bothering
recently, I am confused about the matting metrics like 'SAD' and 'MSE', 'Gradient' , 'Connectivity' , do you have codes for that? It would be great help to me. Thanks~

sdf

How can we run demo without Adobe training set?

Hi,

Apparently the original image training sets is not available to anyone not associated with a university. Is there a way to run the demo at least without it. You have graciously provided the trained model so why is the training data needed for processing new images?

Test.py model not found

Hi foamliu,

I've been having some errors implementing the test.py. The code requires a model named model.35-0.03.hdf5, which is not available in your repository. I tried using the model for the demo.py, but it says that I'm "trying to load a weight file containing 33 layers into a model with 26 layers". Could you please upload the correct model file?

Best,

Oscar

Noisy output

Hello, first of all I want to thank @foamliu for the implementation, it was a huge help for training and testing this network.

However I have problem with noisy matte as output. My output mask looks like this (white and gray pixels outside object area, sometimes black and grey pixels inside object as well):

No matter which weights I'm using - provided by you or retrained by me. It seems that noise came from the border around uncertain (gray) area on trimap:

Does anyone have any idea what the problem might be?

How to test on my own dataset?

Hi,
As I see, both thedemo.pyand test.py use the trimap to output the results:

trimap = generate_trimap(alpha)---demo.py(it seems trimap generated from ground truth)
ap.add_argument("-t", "--trimap", help="path to the trimap file")---test.py(load from existed file)
so, how to get the trimap of my own dataset and test on the pretrained model?
thanks~

问题解决了

问题解决了.

How to implement the unpool

请问您是如何定义unpool的操作的呢，注意到您的unpool操作不是简单的upsample？
还有在decoder阶段，您使用的卷积核的数量没有按照原文当中的来，有没有其他的影响呢

ValueError: You are trying to load a weight file containing 47 layers into a model with 33 layers.

I have trained DIM on a custom dataset, while loading weights from the saved .hdf5 file in test.py I am getting this above value error.

Keras 2.2.4
Python 3.6
TF 1.14

StopIteration: 'NoneType' object has no attribute 'shape'

could you help me to sovel the problem?thanks

What do you think of the method in paper?

The method introduced in paper is very simple. And the result of my implementation is much lower than paper. But I think I have done all the detail. So, do you think there are any tricks which the paper doesn't describe?

version of Tensorflow and keras

Hello, thank you very much for sharing the code. Could you tell me which version of Tensorflow and keras are used for replication?

Is there a difference in the architecture?

Thanks for sharing your code!
I have two questions regarding it, it would be nice if someone could help me out a bit:

1. You added skip connections (which I couldn't find in the description in the original "Deep Image Matting" adobe paper) to the encoder-decoder network and removed/commented the (1x1x4096) convolution in the model.py, why is that?

2. Why did you define for training a different model (in segnet.py) than for testing (demo.py uses modle.py and train.py uses segnet.py)? Also the segnet.py model version has no convolution operation between the last maxpooling operation of the encoder and the first upsampling operation of the decoder, and thus the first convolution operation receives a concatenation of the same input, except that one version is "blurred"...?

Could it be that these changes lead to better results? If yes it would be nice if you could share some metrics :)

Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2

which tensorflow could i select ?

my cpu was Intel(R) Core(TM) i5-4210U CPU @1.70GHz 2.39

How can we train on higher resolution images ?

My training images are of 1200 * 800 resolution. How can i train using your pre trained model ?

What is accuracy in your model ?

In keras, we can add the validation & training accuracy parameter.
During training, keras will print those accuracy values. But are they relevant ? if yes then how are they calculated ?

some error

When i run the test.py, it show:
OSError: Unable to open file (Unable to open file: name = 'models/final.42-0.0398.hdf5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)

but i had put the final.42-0.0398.hdf5 in this folder.

Cannot find the pre-trained model in the provided link.

Cannot find the pre-trained model in the provided link.:
"Download pre-trained Deep Image Matting Model to "models" folder then run."

No file Adobe_Deep_Matting_Dataset.zip

I follow the instruction to contact author for the dataset. But had not received the Adobe_Deep_Matting_Dataset.zip dataset file yet. How to get the dataset file now ?
email: [email protected]

recreate datasets after each epoch

@foamliu Hi, thanks for your implementation of that paper. In that paper, it was said :

" Finally, the training inputs are recreated randomly after each training epoch."(in 4.1 - implementation part)

in your code (https://github.com/foamliu/Deep-Image-Matting/blob/master/data_generator.py#L156), you just did a shuffle of your name list, without recreating (recomposting) the input images. I think this would be an essential reason that your train model cannot achieve that paper's results, am I right?

cheers

I want to check the mapping skills for transparent items. How can I get the corresponding data set?

I want to check the mapping skills for transparent items. How can I get the corresponding data set?

vgg16

Thank you very much for sharing the code. I have two questions to ask and hope to get your reply:

During the training, VGG16 was selected for the pre-training model, but where did you import the VGG16 model?

In addition, if I want to follow up the final.42-0.0398.hdf5 you gave me and continue the training, where should I import this weight?

No such file or directory: 'Combined_Dataset/Training_set/training_fg_names.txt'

I just want to try the demo part, still met with
"No such file or directory: 'Combined_Dataset/Training_set/training_fg_names.txt'"??

question about ‘safe_crop’ function in utils.py

question about "compositional_loss"

Hi，Thanks for sharing your source code! Here is my question:
First, data_generator.py will generate batch_y. The shape of batch_y is (batch, img_rows, img_cols, 2).
Second, compositional_loss(y_true, y_pred) function defined in the utils.py:

def compositional_loss(y_true, y_pred):
    #...
    image = y_true[:, :, :, 2:5]
    fg = y_true[:, :, :, 5:8]
    bg = y_true[:, :, :, 8:11]
    #...

• My understanding is that the generated batch_y corresponds to y_true
• batch_y only has 2 channels in the fourth dimension.
• My quesion: for the code y_true[:, :, :, 2:5] or y_true[:, :, :, 5:8] or y_true[:, :, :, 8:11]. Why can I access the channels greater than 2 in the fourth dimension of y_true?
• My understanding is that I can only do such operations, such as y_true[:, :, :, 0] or y_true[:, :, :, 1].

Waiting for your reply, thank you！