rnri / exploiting-temporality-for-semi-supervised-video-segmentation

Code for the paper "Exploiting Temporality for Semi Supervised Video Segmentation" (ICCV '19)

Exploiting Temporality For Semi Supervised Video Segmentation

This repository contains the corresponding training for the paper.

Training on Cityscapes

1. Set the root of your cityscapes folder in the cityscapes dataloader. For training the sequence models, you need to ask permission for the sequence version of the cityscapes dataset from https://www.cityscapes-dataset.com/.
2. Check the available parser options.
3. Train the networks using the provided training scripts (1, 2):

• Frame-by-Frame: bash train_cityscapes.sh unet fbf 512,256 4 amsgrad 0.0001 0 1 8 128
• Modelling temporality at the bottleneck: bash train_cityscapes.sh unet sequence-1234 512,256 4 amsgrad 0.0001 0 2 8 128 0 convlstm
• Modelling temporality at every skip connection level: bash train_cityscapes.sh unet sequence-1234 512,256 4 amsgrad 0.0001 0 2 8 128 0 convlstm+temporal_skip
• Modelling temporality in the encoder: bash big_train_cityscapes.sh unet sequence-1234 512,256 4 amsgrad 0.0001 0 2 8 128 0 convlstm+temporal_encoder

5. If you want to use a different temporal module, replace convlstm with tcn (Pointwise TCN), tcn2d or tcn2dhw. More details here

Results

Effect of temporal module locations

Effect of temporal module types

Citation:

If you use this code, please consider citing our work-

@inproceedings{Sibechi_2019_ICCV,
author = {Sibechi, Radu and Booij, Olaf and Baka, Nora and Bloem, Peter},
title = {Exploiting Temporality for Semi-Supervised Video Segmentation},
booktitle = {The IEEE International Conference on Computer Vision (ICCV) Workshops},
month = {Oct},
year = {2019}
}