tostenzel / mcmot-transformer

Multi-camera Multi-object Tracking with Transformer

Hi there, I am Tobias. I like the fundamental concepts of Deep Learning and aim at consolidating my knowledge in this field through my blog and programming projects.

mcmot-transformer's Issues

expand training on xywh to training on cylinders

eval on multicam 3D and monitor visually

Create multi-camera 3D training dataset

switch MCMOT to DETR

Train on internal (xmin/W, ymin/H, w/W, h/H) bbox format and make evaluation work

(xmin/W, ymin/H, w/W, h/H) format with normal DETR

Train Trackformer on WILDTRACK

• Generate train/test/val split (75%/12.5%/12.5%) for WILDTRACK in COCO and MOT format and check that these are correct
• Register WILDTRACK as dataset in the ML pipeline and check that it works properly (No MOTA below 0)
• Register fine-tuning datasets with additional training data in ML pipeline
• Provide all config.yamls for different training setups for the baseline ablation in the thesis

train on 3D only and monitor visually

Another try with 3d training

train and eval on 3D multicam WILDTRACK data

Working version without preprocessing data and targets

Feed multiple image sequences at once into trackformer

This requires the following features (different sequences refer to different cameras/views):

• A torch.DataLoader for the whole dataset that can call images from all sequences and not only from one sequence
• The possibility to load images into the CNN backbone by alternating over images of the the same time period from different sequences (possibly in parallel) instead of sequence-by-sequence. For one time period, this will return k token sequences
• The possibility to append the token sequences for one time period and feed it into the transformer

Points 2 and 3 may require to (temporarily) reduce the dimension of the CNN backbone output / transformer input tokens to make this computationally feasible

• Adding the sinusoidal location embedding to each token sequence before appending
• Adding one learned embedding vector to every token. But the k learned embeddings differ per sequence