haha911 / gim

GIM: Learning Generalizable Image Matcher From Internet Videos (ICLR 2024 Spotlight)

GIM: Learning Generalizable Image Matcher From Internet Videos

✅ TODO List

• Inference code
• Training code

Thank you everyone for your interest in GIM. I am currently catching up with the DDL for my paper. After the completion of my paper, I will continue with the code releasing of GIM. Thank you for your patience. Code releasing is expected to continue in late March.

🤗 Online demo

Go to Huggingface to quickly try our model online.

⚙️ Environment

My code running environment is:

• GeForce RTX 3090
• Ubuntu 20.04.3
• Python (3.8.10)
• Pytorch 1.10.2 (py3.8_cuda11.3_cudnn8.2.0_0)

For the specific environment, please run the following command to install anaconda

conda env create -f environment.yml

If the above command fails to install the environment directly, please refer to the clean environment in environment.txt to install each package.

🔨 Usage

Clone the repository

git clone https://github.com/xuelunshen/gim.git
cd gim
mkdir weights

Download model weight from Google Drive

Put it on the folder weights

Run the following command

python demo.py

The code will match a.png and b.png in the folder assets/demo
, and output a_b_match.png and a_b_warp.png.

Click to show a.png and b.png.

Click to show a_b_match.png.

a_b_match.png is a visualization of the match between the two images

Click to show a_b_warp.png.

a_b_warp.png shows the effect of projecting `image b` onto `image a` using homography

📌 Citation

If our code helps your research, please give a citation to our paper ❤️ Thank you very much.

@inproceedings{
xuelun2024gim,
title={GIM: Learning Generalizable Image Matcher From Internet Videos},
author={Xuelun Shen and Zhipeng Cai and Wei Yin and Matthias Müller and Zijun Li and Kaixuan Wang and Xiaozhi Chen and Cheng Wang},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024}
}

🌟 Star History

License

This repository is under the MIT License. This content/model is provided here for research purposes only. Any use beyond this is your sole responsibility and subject to your securing the necessary rights for your purpose.