The official repo of the CVPR 2024 paper Multi-Object Tracking in the Dark
- [2024-06] 🔥 LMOT Dataset are available now !
Low-light scenes are prevalent in real-world applications (e.g. autonomous driving and surveillance at night). Recently, multi-object tracking in various practical use cases have received much attention, but multi-object tracking in dark scenes is rarely considered. In this paper, we focus on multi-object tracking in dark scenes. To address the lack of datasets, we first build a Low-light Multi-Object Tracking (LMOT) dataset. LMOT provides well-aligned low-light video pairs captured by our dual-camera system, and high-quality multi-object tracking annotations for all videos. Then, we propose a low-light multi-object tracking method, termed as LTrack. We introduce the adaptive lowpass downsample module to enhance low-frequency components of images outside the sensor noises. The degradation suppression learning strategy enables the model to learn invariant information under noise disturbance and image quality degradation. These components improve the robustness of multi-object tracking in dark scenes. We conducted a comprehensive analysis of our LMOT dataset and proposed LTrack. Experimental results demonstrate the superiority of the proposed method and its competitiveness in real night low-light scenes
LMOT dataset is collected using our dual-camera sysyem, which provide well-aligned low-light and well-lit video pairs (LMOT-dual). We also collect a real low-light MOT dataset to evaluate performance in real night dark scene, which are captured using a single camera with the same camera settings (LMOT-real).
We provide the dataset in both RAW (RGGB) and sRGB format, with 20FPS, 10ms exposure time, and
Detailed statics and data splits for LMOT dataset
| Dataset | Split | Videos | Bbox | Tracks | Paired Well-lit |
| LMOT-dual | train | 11 | 309,466 | 1,533 | ✔ |
| val | 4 | 131,781 | 626 | ✔ | |
| test | 11 | 312,742 | 1,644 | ✔ | |
| LMOT-real | real | 6 | 61,561 | 287 | ✖ |
LMOT dataset can be downloaded from Baidu Drive (code:xedx).
Note: Currently, we only release the training set and validation set. Test set and the remaining part will be released later along with the challenges.
Organize the files into the following structure.
{LMOT ROOT}
└── LMOT_release/
├── train
├── LMOT-02
├── gt
└── gt.txt
├── img_dark
├── 000001.tiff
└── ...
├── img_dark_rgb
├── 000001.tiff
└── ...
├── img_light_rgb
├── 000001.jpg
└── ...
├── img_light
├── 000001.tiff
└── ...
└── seqinfo.ini
├── LMOT-04
└── ...
├── val
└── ...
├── test
└── ...
└── real
├── RLMOT-01
├── gt
└── gt.txt
├── img_real
├── 000001.tiff
...
├── img_real_rgb
├── 000001.jpg
...
└── seqinfo.ini
└── ...
LMOT dataset is organized in the form of MOT Challenge 17. Each line in gt.txt contains
fn, id, x1, y1, w, h, ignore, classid, vis_ratio
The 6 categories of annotated objects include
'person', 'bicycle', 'car', 'motorcycle', 'bus', 'truck'
In addition to the authors of the paper, some friends also helped with data collection and annotation, they are Li Yichen, Wang Binfeng, Wang Haoyu, Wang Yuran, Zhang Taoying, and Wang Jianan. We sincerely thank them for their contributions to this work.
- LMOT dataset is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
- LMOT dataset is available for non-commercial research purposes only.
- The code of this repository is released under the MIT License.
If you our dataset or code for research, please cite our paper:
@InProceedings{wang2024lmot,
author = {Wang, Xinzhe and Ma, Kang and Liu, Qiankun and Zou, Yunhao and Fu, Ying},
title = {Multi-Object Tracking in the Dark},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2024},
pages = {382-392}
}
If you have any question about our dataset, please email to [email protected].
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent