dylancer1998 / awesome-visual-localization-papers

The relocalization task aims to estimate the 6-DoF pose of a novel (unseen) frame in the coordinate system given by the prior model of the world.

Visual-Based-Localization-Papers

The camera re-localization task aims to estimate the 6-DoF pose of a novel (unseen) frame in the coordinate system given by the prior model of the world. The most related academic topics are SLAM and SfM and it's widely applied in AR, Robotic, etc.

Feel free to make a PR or contribute. 😄

---

Table of Contents

• survey
• system
• direct methods
• feature extraction
• feature match
• retrieval
• robust pose estimation
• multi-sensors fusion
• slam
• sfm
• waiting to sort

---

Survey

• [Image-based camera localization: an overview] Yihong Wu. Visual Computing for Industry, Biomedicine, and Art, 2018. [paper]

System

• [Wide area localization on mobile phones] Clemens Arth. ISMAR, 2009. [paper]

• [Parallel Tracking and Mapping on a Camera Phone] ISMAR, 2009. [paper]

• [Real-time self-localization from panoramic images on mobile devices] Clemens Arth. ISMAR, 2011. [paper]

• [Scalable 6-DOF Localization on Mobile Devices] Iven Middelberg, Torsten Sattler. ECCV, 2014. [paper]

• [6D dynamic camera relocalization from single reference image] Feng W. CVPR 2016. [paper]

• [Image Matching Across Wide Baselines: From Paper to Practice] Yuehe, Jin. CVPR, 2020. [paper] [code]

• [GN-Net: The Gauss-Newton Loss for Multi-Weather Relocaliza-tion] L. von Stumberg, P. Wenzel, Q. Khan, and D. Cremers. ICRA, 2020. [paper] [code]

• [Using Image Sequences for Long-Term Visual Localization] Erik Stenborg, Torsten Sattler and Lars Hammarstrand. 3DV, 2020. [paper] [code]

• [LM-Reloc: Levenberg-Marquardt Based Direct Visual Relocalization] Lukas von Stumberg, Patrick Wenzel, Nan Yang, Daniel Cremers. 3DV, 2020. [paper]

• [Efficient 2D-3D Matching for Multi-Camera Visual Localization] Marcel Geppert, Peidong Liu, Zhaopeng Cui, Marc Pollefeys, Torsten Sattler. ICRL, 2020. [paper]

• [KFNet: Learning Temporal Camera Relocalization using Kalman Filtering] Lei Zhou, Zixin Luo, Tianwei Shen... CVPR, 2020 [paper]

• [Robust Neural Routing Through Space Partitions for Camera Relocalizationin Dynamic Indoor Environments] Siyan Dong, Qingnan Fan... CVPR, 2021, oral [paper]

• [CrossLoc: Scalable Aerial Localization Assisted by Multimodal Synthetic Data] Qi Yan, Jianhao Zheng, ... CVPR, 2022. [paper] [code] [dataset] [video] [website]

Direct Method

• [PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization] A. Kendall, M. ICCV, 2015. [code][paper]

Feature Extracting

• [Semantic Visual Localization] J. L. Sch�onberger. CVPR, 2018. [paper]

• [R2D2: Repeatable and Reliable Detector and Descriptor] Jerome Revaud. NeurIPS, 2019. [paper][code]

• [Learning Feature Descriptors using Camera Pose Supervision] Qianqian Wang. ECCV, 2020, oral. [paper] [code]

• [ASLFeat: Learning Local Features of Accurate Shape and Localization] Zixin, Lup. CVPR, 2020. [paper] [code]

• [DISK: learning local features with policy gradient] Michał J. Tyszkiewicz. NeurIPS, 2020. [paper] [code]

• [Reinforced Feature Points: Optimizing Feature Detection and Description for a High-Level Task] Aritra Bhowmik... CVPR oral, 2020. [paper][code]

• [FisheyeSuperPoint: Keypoint Detection andDescription Network for Fisheye Images] Anna Konrad. 2021. [paper]

• [ALIKE: Accurate and Lightweight Keypoint Detection and Descriptor Extraction] Xiaoming Zhao, Xingming Wu, Jinyu Miao, Weihai Chen, Peter C. Y. Chen, Zhengguo Li. TM, 2021. [paper][code]

• [PoSFeat: Decoupling Makes Weakly Supervised Local Feature Better] Kunhong Li, Longguang Wang, Li Liu, Qing Ran, Kai Xu, Yulan Guo. CVPR, 2022. [paper][code]

Feature Matching

• [Learning to Find Good Correspondences] Kwang Moo Yi. CVPR, 2018, oral. [paper] [code]

• [OANet: Learning Two-View Correspondences and Geometry Using Order-Aware Network] Zhang, Jiahui and Sun. ICCV, 2019. [paper] [code]

• [ACNe: Attentive Context Normalization for Robust Permutation Equivariant Learning] Sun, W. CVPR, 2020. [paper] [code]

• [Is there anything new to say about SIFT matching?] Fabio Bellavia. IJCV, 2020. [paper]

• [Deep Keypoint-Based Camera Pose Estimation with Geometric Constraints] You-Yi Jau, Rui Zhu. IROS, 2020 [paper] [code]

• [SuperGlue: Learning Feature Matching with Graph Neural Networks] Paul-Edouard Sarlin. CVPR, 2020. [paper] [code]

• [LoFTR: Detector-Free Local Feature Matching with Transformers] Jiaming Sun, Zehong Shen, Yu'ang Wang. CVPR, 2021. [paper] [code]

• [COTR: Correspondence Transformer for Matching Across Images] Wei Jiang. ICCV, 2021. [paper] [code]

• [Patch2Pix for Accurate Image Correspondence Estimation] Qunjie Zhou, Torsten Sattle, Laura Leal-Taix ́e. CVPR, 2021. [paper][code]

• [DFM: A Performance Baseline for Deep Feature Matching] Ufuk Efe, Kutalmis Gokalp Ince, A. Aydin Alatan. CVPR, 2021 [paper]

• [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose] Paul-Edouard Sarlin. CVPR, 2021 [paper] [code]

• [Cross-Descriptor Visual Localization and Mapping] Mihai Dusmanu. ICCV, 2021 [paper] [[code]](https://github.com/mihaidusmanu/cross-descriptor-vis-loc-map

• [MatchFormer: Interleaving Attention in Transformersfor Feature Matching] Qing Wang, Jiaming Zhang. arxiv, 2022 [paper] [code]

• [ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer] Hongkai Chen, Zixin Luo, Lei Zhou, Yurun Tian, Mingmin Zhen, Tian Fang, David Mckinnon, Yanghai Tsin, Long Quan. ECCV, 2022 [paper] code(coming soon)

• [Deep Kernelized Dense Geometric Matching] Johan Edstedt, Mårten Wadenbäck, Michael Felsberg. arXiv, 2022 [paper] [code]

Retrieval Methods

• [Visual Categorization with Bags of Keypoints] G. Csurka. ECCV, 2004. [paper]

• [Total Recall: Automatic Query Expansion with a Generative Feature Model for Object Retrieval] Chum, O. ICCV, 2007. [paper]

• [Fisher Kernels on Visual Vocabularies for Image Categorization] F. Perronnin and C. Dance. CVPR, 2007. [paper]

• [Aggregating Local Descriptors Into a Compact Image Representation] H. Jegou. CVPR, 2010. [paper]

• [Fast image-based localization using direct 2D to-3D matching] Sattler T. ICCV, 2011. [paper] [code]

• [Improving image-based localization by active correspondence search] ECCV, 2012. [paper]

• [Aggregating Deep Convolutional Features for Image Retrieval] A. Babenko and V. Lempitsky. ICCV, 2015. [paper]

• [A Vote-and-Verify Strategy for Fast Spatial Verification in Image Retrieval] Johannes L. Sch¨onberger. ACCV, 2016. [paper] [code]

• [NetVLAD: CNN Architecture for Weakly Supervised Place Recognition] R. Arandjelovic. CVPR, 2016. [paper]

• [Crossdimensional Weighting for Aggregated Deep Convolutional Features] Y. Kalantidis. ECCV, 2016. [paper] [code]

• [Fine-Tuning CNN Image Retrieval with no Human Annotation] F. Radenovic, G. PAMI, 2017. [paper] [code]

• [Efficient diffusion on region manifolds: Recovering small objects with compact cnn representations] A. Iscen. CVPR, 2017. [paper]

• [Revisiting Oxford and Paris: Large-scale Image Retrieval Benchmarking] F. Radenovic, G. CVPR, 2018. [paper]

• [Learning with Average Precision: Training Image Retrieval with a Listwise Loss] J. Revaud. ICCV, 2019. [paper]

• [Benchmarking Image Retrieval for Visual Localization] Noé Pion,..., Torsten Sattler. 3DV, 2020. [paper] [code]

Robust Pose Estimation

• [Fixing the Locally Optimized RANSAC] Karel Lebeda. BMVC, 2012. [paper]

• [Camera Pose Voting for Large-Scale Image-Based Localization] B. Zeisl, T. Sattler. ICCV, 2015. [paper]

• [City-Scale Localization for Cameras with Known Vertical Direction] Linus Svarm. TPAMI, 2016. [paper]

• [DSAC - Differentiable RANSAC for Camera Localization] E.Brachmann. CVPR, 2017. [code][paper]

• [MAGSAC: marginalizing sample consensus] Barath, D. CVPR, 2019. [paper]

• [GC-RANSAC: Graph-Cut RANSAC] Daniel Barath, Jiri Matas. CVPR, 2020. [paper][code]

• [AdaLAM: Revisiting Handcrafted Outlier Detection] Luca Cavalli... ECCV, 2020. [paper][code]

• [DegenSac] 2021 [code]

• [Learning Bipartite Graph Matching for Robust Visual Localization] Hailin Yu, Weicai Ye. ISMAR, 2020. [paper]

• [Learnable Motion Coherence for Correspondence Pruning] Yuan Liu, Lingjie Liu, Cheng Lin, Zhen Dong, Wenping Wang. CVPR, 2021. [paper][code]

• [CLNet: Progressive Correspondence Pruning by Consensus Learning] Zhao, Chen and Ge, Yixiao and Zhu, Feng and Zhao, Rui and Li, Hongsheng and Salzmann, Mathieu. ICCV, 2021. [paper][code]

Multi-sensors Fusion

Fusion with IMU

• [DARNavi: An Indoor-Outdoor Immersive Navigation System with Augmented Reality] Didi Chuxing. CVPR, 2020. [paper]

Fusion with GPS

• [Multi-sensor navigation algorithm using monocular camera, imu and gps for large scale augmented reality] T. Oskiper. ISMAR, 2012. [paper]

• [Gomsf: Graph-optimization based multi-sensor fusion for robust uav pose estimation] R. Mascaro, L. ICRA 2018. [paper]

• [Intermittent GPS-aided VIO: Online Initialization and Calibration] Woosik Lee. ICRA, 2020. [paper]

• [Vins Fusion — A General Optimization-based Framework for Global Pose Estimation with Multiple Sensors] Tong Qin. [paper]

SLAM

• [Towards SLAM-based Outdoor Localization using Poor GPS and 2.5D Building Models] R.Liu et al. ISMAR, 2019. [code] [paper]

• [Neural Topological SLAM for Visual Navigation] Devendra Singh Chaplot. CVPR, 2020. [paper]

• [TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo] Lukas Koestler et al. CoRL, 2021. [paper] [code]

SfM

• [DPSNet: End-to-end Deep Plane Sweep Stereo] Sunghoon Im. ICLR, 2019. [paper]

• [Consistent Video Depth Estimation] XUAN LUO. SIGGRAPH 2020. [paper]

• [DeepSFM: Structure From Motion Via Deep Bundle Adjustment] ECCV 2020. [paper]

• [Multi-View Optimization of Local Feature Geometry] Mihai Dusmanu et al. ECCV 2020. [paper] [paper]

• [Deepv2d: Video to depth with differentiable structure from motion] Zachary Teed, Jia Deng. ICLR, 2020. [paper]

• [Hybrid Rotation Averaging: A Fast and Robust Rotation Averaging Approach] Yu Chen. CVPR, 2021. [paper] [code]

• [Pixel-Perfect Structure-from-Motion with Featuremetric Refinement] Philipp Lindenberger, Paul-Edouard Sarlin, Viktor Larsson, Marc Pollefeys. ICCV, oral, 2021. [paper][code]

• [Monocular RGB Scene Reconstruction using Transformers] Aljaž Božiˇc, Pablo Palafox, Justus Thies, Angela Dai, Matthias Nießner. NeurIPS, 2021. [paper][code]

Waiting to sort

• [FREAK:Fast Retina Keypoint.] A. Amit. CVPR, 2012.

• [Three things evereyone should know to improve object retrieval] R. Arandjelovic. CVPR, 2012.

• [Learning local feature descriptors with triplets and shallow convolutional neural networks] V. Balntas. BMVC, 2016.

• [Learning 6D Object Pose Estimation Using 3D Objet Coordinates] E.Brachmann. ECCV, 2014.

• [Discriminative Learning of Local Image Descriptors] TPAMI, 2011.

• [MatchNet: Unifying feature and metric learning for patch-based matching] X. Han. CVPR, 2015.

• [Comparative evaluation of binary features] J. Heinly. ECCV.

• [LIFT: Learned Invariant Feature Transform] M.Kwang. ECCV, 2016.