A collection of referring image segmentation papers and datasets.
Feel free to create a PR or an issue.
Outline
- 1. Datasets
- 2. Traditional Referring Image Segmentation
- 3. Interactive Referring Image Segmentation
- 4. Referring Video Segmentation
- 5. Referring 3D Instance Segmentation
Short name | Paper | Source | Code/Project Link |
---|---|---|---|
ReferIt | Referit game: Referring to objects in photographs of natural scenes | EMNLP 2014 | [project] |
Google-Ref | Generation and comprehension of unambiguous object descriptions | CVPR 2016 | [dataset] |
UNC | Modeling context in referring expressions | ECCV 2016 | [dataset] |
UNC+ | Modeling context in referring expressions | ECCV 2016 | [dataset] |
CLEVR-Ref+ | CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressions | CVPR 2019 | [project] |
VGPhraseCut | PhraseCut: Language-based Image Segmentation in the Wild | CVPR 2020 | [project] |
ScanRefer | ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language | ECCV 2020 | [project] |
ClevrTex | ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentation | NeurIPS 2021 | [project] |
Short name | Paper | Source | Code/Project Link |
---|---|---|---|
PhraseClick | PhraseClick: Toward Achieving Flexible Interactive Segmentation by Phrase and Click | ECCV 2020 |
Short name | Paper | Source | Code/Project Link |
---|---|---|---|
Video Object Segmentation with Language Referring Expressions | ACCV 2018 | ||
RefVOS | RefVOS: A Closer Look at Referring Expressions for Video Object Segmentation | arxiv 20.10 | |
URVOS | URVOS: Unified Referring Video Object Segmentation Network with a Large-Scale Benchmark | ECCV 2020 | [code] |
YOFO | You Only Infer Once: Cross-Modal Meta-Transfer for Referring Video Object Segmentation | AAAI 2022 | |
MTTR | End-to-End Referring Video Object Segmentation with Multimodal Transformers | CVPR 2022 | [code] |
Short name | Paper | Source | Code/Project Link |
---|---|---|---|
TGNN | Text-Guided Graph Neural Networks for Referring 3D Instance Segmentation | AAAI 2021 | |
InstanceRefer | InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referring | ICCV 2021 | [code] |