Inference on test images? about letr

[yix081]

Thanks for your interest. I will upload one tomorrow if possible, and will keep you updated.

[tianchen-ns]

Thanks for your interest. I will upload one tomorrow if possible, and will keep you updated.

Thank you so much. Looking forward to the update.

[yix081]

Hi, Here is a quick usage demo. Hope it helps.
https://github.com/mlpc-ucsd/LETR/blob/master/src/demo_letr.ipynb

[tianchen-ns]

Thank you! The example works perfectly on my new data. Some suggestions:

1. The default pretrained weights (DETR) is not exactly match the model near the output (shape mismatch). To make it work I assign the weights only if the shape match.
2. Is it possible to apply NMS postprocessing as in LCNN to remove duplicate/close lines? Currently in my custom dataset the overall result is great, better than LCNN, but with some results have duplicate lines.
3. Have you considered using imgaug to do augmentation/transforms? imgaug supports the augmentation on lines. The current implementation is good but hard to further customize.

[yix081]

Thanks for your suggestions.

1. DETR weights are different from LETR slightly. Please refer to main.py
   

   LETR/src/main.py

   Lines 91 to 96 in 005b4c1

   	for k in checkpoint['model']:
   	if ("class_embed" in k) or ("bbox_embed" in k) or ("query_embed" in k):
   	continue
   	if ("input_proj" in k) and args.layer1_num != 3:
   	continue
   	new_state_dict[k] = checkpoint['model'][k]

   Let me know if you run into any problem with the provided training scripts, e.g. https://github.com/mlpc-ucsd/LETR/blob/master/script/train/a1_train_stage1_res101.sh.
2. It is possible to apply NMS. However, the main purpose of LETR is to introduce a method that is post-processing and heuristics-guided intermediate processing (edge/junction/region detection) free. To further boost the performance, you are welcome to try this direction.
3. Imgaug is another good idea to boost the performance further as well. We have not tried it yet. Transformers are data-hungry. We experience that more data augmentation would almost always help the training.