Questions about the matching loss about up-detr

[dddzg]

Thanks for your attention!

1. You are right. I also have ever considered this problem beforing experiments. However, after experiments, we find that the model has learn to predict from the same group. So, we did not change the matching implementation.
2. Yes, there could be a mis-matching theoretically. But, I did not observe a noticable problem with current setting. It indicates that transformers learn to predict bbox from the given query patches.
3. On the one way, with label c=1, we can get the model prediction result in the inference stage(like the visualization). One the other way, with c=1 and c=0, we can construct a many-to-one matching mechanism and the model can learn to a NMS-like mechanism during the pre-training.