uzomacollins / awesome-referring-image-segmentation

:books: A collection of papers about Referring Image Segmentation.

Awesome-Referring-Image-Segmentation

A collection of referring image segmentation papers and datasets.

Feel free to create a PR or an issue.

Outline

• 1. Datasets
• 2. Traditional Referring Image Segmentation
• 3. Interactive Referring Image Segmentation
• 4. Referring Video Segmentation

1. Datasets

Short name	Paper	Source	Code/Project Link
ReferIt	Referit game: Referring to objects in photographs of natural scenes	EMNLP 2014	[project]
Google-Ref	Generation and comprehension of unambiguous object descriptions	CVPR 2016	[dataset]
UNC	Modeling context in referring expressions	ECCV 2016	[dataset]
UNC+	Modeling context in referring expressions	ECCV 2016	[dataset]
CLEVR-Ref+	CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressions	CVPR 2019	[project]
VGPhraseCut	PhraseCut: Language-based Image Segmentation in the Wild	CVPR 2020	[project]

---

2. Traditional Referring Image Segmentation

Short name	Paper	Source	Code/Project Link
CGAN	Cascade Grouped Attention Network for Referring Expression Segmentation	ACM MM 2020
LSCM	Linguistic Structure Guided Context Modeling for Referring Image Segmentation	ECCV 2020
CMPC-Refseg	Referring Image Segmentation via Cross-Modal Progressive Comprehension	CVPR 2020	[code]
BRINet	Bi-directional Relationship Inferring Network for Referring Image Segmentation	CVPR 2020	[code]
PhraseCut	PhraseCut: Language-based Image Segmentation in the Wild	CVPR 2020	[code] [project]
MCN	Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentation	CVPR 2020	[code]
-	Dual Convolutional LSTM Network for Referring Image Segmentation	TMM 2020
STEP	See-Through-Text Grouping for Referring Image Segmentation	ICCV 2019
lang2seg	Referring Expression Object Segmentation with Caption-Aware Consistency	BMVC 2019	[code]
CMSA	Cross-Modal Self-Attention Network for Referring Image Segmentation	CVPR 2019	[code]
KWA	Key-Word-Aware Network for Referring Expression Image Segmentation	ECCV 2018	[code]
DMN	Dynamic Multimodal Instance Segmentation Guided by Natural Language Queries	ECCV 2018	[code]
RRN	Referring Image Segmentation via Recurrent Refinement Networks	CVPR 2018	[code]
MAttNet	MAttNet: Modular Attention Network for Referring Expression Comprehension	CVPR 2018	[code] [Demo]
RMI	Recurrent Multimodal Interaction for Referring Image Segmentation	ICCV 2017	[code]
LSTM-CNN	Segmentation from natural language expressions	ECCV 2016	[code] [project]

3. Interactive Referring Image Segmentation

Short name	Paper	Source	Code/Project Link
PhraseClick	PhraseClick: Toward Achieving Flexible Interactive Segmentation by Phrase and Click	ECCV 2020

4. Referring Video Segmentation

Short name	Paper	Source	Code/Project Link
RefVOS	RefVOS: A Closer Look at Referring Expressions for Video Object Segmentation	arxiv
URVOS	URVOS: Unified Referring Video Object Segmentation Network with a Large-Scale Benchmark	ECCV 2020	[code]
	Video Object Segmentation with Language Referring Expressions	ACCV 2018