nijkah / masktrack_box

This is simple implementation of MaskTrack_Box only requiring a bounding box for video object segmentation.

MaskTrack_Box

This is a simple implementation of MaskTrack_Box in pytorch.

Compared to conventional semi-supervised video object segmentation methods, MaskTrack_Box requires only a bounding box of the target for video object segmentation.

Althohugh original MaskTrack_Box consists of two models(1. Model extracting a mask from the box 2. Mask propagation model), I simplified these models into one.

Main differences between this project and original paper

• Model is just one, whereas Masktrack_box in the paper consists of two models.
• The deformation method is simplified by imgaug library, running on-the-fly.

• I added fine-tuning code, but this is not necessary to evaluate.

Environment setup

All the code has been tested on Ubuntu 16.04, python3.6, Pytorch0.4.1, CUDA 9.0, GTX TITAN x GPU

• Clone the repository

git clone https://github.com/nijkah/masktrack_box.git && cd masktrack_box

• Setup python environment

conda create -n masktrack_box python=3.6
source activate masktrack_box 
conda install pytorch=0.4.1 cuda90 -c pytorch
pip install -r requirments.txt

• Download data

cd data
sh download_datasets.sh
cd ..

and you can download the pre-trained deeplab model from here. Put this in the 'data' folder.

• train the model

cd train
python train.py

• evaluate the model

cd evaluatation
python evaluate.py

You can download the trained MaskTrack_Box model from here. Put this in the 'data/snapshots' folder.

Results

Model	DAVIS2016 mean IoU
Paper (MaskTrack_box)	69.6
This (No fine-tuning)	65.4

Citations

The original paper is

@inproceedings{Perazzi2017,
  author={F. Perazzi and A. Khoreva and R. Benenson and B. Schiele and A.Sorkine-Hornung},
  title={Learning Video Object Segmentation from Static Images},
  booktitle = {Computer Vision and Pattern Recognition},
  year = {2017}
}

Acknowledgement

The original base code is borrowed from https://github.com/isht7/pytorch-deeplab-resnet.

This project is inspired by https://github.com/omkar13/MaskTrack.

masktrack_box's Issues

problem about ESCCD_strem datasets

Hello, I am very interested in your work. I saw that the ESCCD_strem data set is called in the ESCCD function in the dataset_pair.py function in your training data set. How did you get this data set?

Training Data and Validation Data Different From Paper

Hi,
In your code, I notice that you use DAVIS-2016 for training, and testing data in DAVIS-2016 validation sets.
But In paper, it seems that author didn't use DAVIS for offline training.
And for evaluation, the author use DAVIS-2016 train+val set, total 50 sequences and 3455 frames to evaluate and get mIOU 69.6%.

1. author mentioned datasets
   
2. evaluation sequence lists
   

I'm wondering whether the pretrained model that you kindly provided is trained by DAVIS-2016 train set, so that it may probably not propriate to directly compare with the mIOU in the paper.

How many epochs during training?

Hi. If I want to use train.py with pre-trained deeplab model according to your default configs, how many epochs needed to reach mIOU=65.4 ?

Simplied Model

Hi,
In the README, it mentions that the model you provided is simplified.
Is the model trained by only segmentation mask?

thanks for helping!