neurai-lab / bihome Goto Github PK

Thank you very much for your work, but I have some questions:

1. In https://github.com/NeurAI-Lab/biHomE/blob/main/src/heads/PerceptualHead.py#L177 used DSAC to calculate homography and four points offset
2. In https://github.com/NeurAI-Lab/biHomE/blob/main/src/heads/PerceptualHead.py#L371 used four points offsets to calculate homography and warp feature map.
   Actually homography matrix was calculated twice, but the homography matrix is a little different.
   Thanks!

Which Torch version did you use during developing a project?

Hi, I have question about the triplet loss in your paper.
Here you said that the margin you use is infinite value, and you write the formulation as follow:

I wonder whether the value of this loss can be negative? That means ap_{i,j} < an_{i, j} (or the distance between the anchor to positive sample is smaller the that between negative sample, and this seems easy to reach?) What does it mean when loss is less than zero? how can it be optimized?

Hello,in PFNet ,we should use ransac to sample point from PF, how u implement it , i use the kornia the cuda out of memory

Where is the pretraining weight biHome loss model? Can you offer a pretraining weight of the biHome loss model? I don't know how to train the loss network alone.

Hi,
Thanks for providing the source code. I want to test your model with my own data. I would be very grateful if you can provide a well-trained model.

Hi,
Is trained Model weights available to download?
Thanks in advance

Hi
Would you please explain outputs of model?
https://github.com/NeurAI-Lab/biHomE/blob/main/train.py#L357
What are delta_gt & delta_hat in train.py and eval.py? What is the difference?
How to train and inference this model on my own dataset?
Thanks in advanced.

Hello guys! Awesome work - really liked reading the paper. It's very interesting that you've managed non-VGG backbones to work as a perceptual loss.

Unfortunately, I currently don't have compute resources to train the model from scratch, so would you mind sharing some pretrained checkpoints?

Thanks!

I have trained pseudo homography dataset(augmented with a photometrical distortion), by when I evaluated with a real dataset, the performance is weakly(the smaller the better).

Hello.

1. What are the expected MACE values from the models trained with following config files:

• config/pds-coco/zeng-bihome-lr-1e-3.yaml
• config/pds-coco/zhang-bihome-lr-1e-2.yaml
• config/s-coco/nguyen-orig-lr-5e-3.yam

I am doing everything as is written in the README.md and after writting following commands after training models:

python3 eval.py --config_file config/pds-coco/zeng-bihome-lr-1e-3.yaml --ckpt log/zeng-bihome-pdscoco-lr-1e-3/model_090000.pth
python3 eval.py --config_file config/pds-coco/zhang-bihome-lr-1e-2.yaml --ckpt log/zhang-bihome-pdscoco-lr-1e-2/model_090000.pth
python3 eval.py --config_file config/s-coco/nguyen-orig-lr-1e-3.yaml --ckpt log/nguyen-orig-scoco-lr-1e-3/model_090000.pth

I get the following MACE values:

• MACE: 2.9905893802642822 for zeng+biHomE
• MACE: 2.0459039211273193 for zhang+biHomE
• MACE: 1.5663892030715942 for nguyen

Based on what you have written in the paper, the MACE should be the smallest for the zeng as hen.

2. Also are the MACE values for Nguyen and Zhang from figure 1 from your paper received from models trained with following config files?:

• config/s-coco/zhang-bihome-lr-1e-2.yaml
• config/s-coco/zhang-bihome-lr-1e-2.yaml

Thank you for your brilliant work, but I have some questions:
(1) As far as I know, the image pairs in the FLIR Dataset dataset are not aligned, how could you calculated the MACE without groundtruth?
(2) In your paper, you said "To test this vulnerability we compare Zhang et al. [44] method with the original and our biHomE loss on FLIR Dataset [7] preprocessed similar to S-COCO." Could you please provide a more detailed description about this process?
(3) I try to replace the original FLIR dataset with the manually registered dataset (https://github.com/StaRainJ/road-scene-infrared-visible-images), but only the Zeng + Orig strategy can successfully train (maybe because there are too few training data, MACE is only 13 in test data), and the networks of other unsupervised methods cannot converge (loss does not decline, and the value is abnormally large).
I'm confused about how the FLIR experiment in your paper works....

Thank you for your work and your contribution.

I have a question about the training process.

When I trained the model, the total loss is a negative value.

I didn't change anything and only use to following commands.

CUDA_VISIBLE_DEVICES=7 python train.py --config_file config/pds-coco/zhang-bihome-lr-1e-2.yaml

Do you have any idea about how to respond to this situation?

Also, Could you provide a config file about flir-adas..? @dkoguciuk