grumpyzhou / patch2pix Goto Github PK

Dear Author,
You've mentioned that to evaluate this method on Aachen Day-Night Benchmark, you did quantization for the matches. Will you release the code for the quantization part?

Thank you for your impressive achievement. My question is as the title suggests: How to use custom datasets (paired images) for registration and evaluation?

Hi,

Thank you for your excellent work! I'd like to ask for some advice for training this model.

When I modified this model on my own datasets, I encounter a problem that it converges to a point where the loss of cls_fine = 0, but the metric, Specificity of cls_fine, is about 0.01, which means it has a weak ability to distinguish the false match. May I get some advice for improving the performance further?

Thank you for your outstanding work. My question is how many of the four kinds of loss (cls_mid/cls_fine/epi_mid/epi_fine) finally converge to? And how long the training will take？

Hi,

Looking here:
https://github.com/GrumpyZhou/patch2pix/blob/main/train_patch2pix.py#L138-L200

cdist is used for the medium level prediction, and mdist is used for the fine level.
I'm not quite sure I follow the logic here. Why is mdist and fdist not used for the medium/fine level?

I want to create my own dataset to train with patch2pix. If I have fundemental matrix between source and destination images, would it be sufficent as supervision to train patch2pix ?

Hello Authors,

I am getting Segfault error when I am trying to run visualize_matches.ipynb code. I debugged this error and specifically it happens at

from utils.common.plotting import plot_matches

I think there is some issue with module importing. I am running code in Linux.

Has anyone faced this issue?

Hi, Could you give me any advice? I want to output some results like the results you illustrated in your paper(Figure7). How can I modify the code in 'visualize_matches.py' to get the outcome like that?

Hi @GrumpyZhou , Greetings!!
Thanks for the amazing work on patch2pix.

I was trying out patch2pix, however, the checkpoint paths seem invalid. Could you please confirm if the checkpoints have been migrated?

-Thanks

Hello Authors,
I am getting Not Found error when I am using the url in data_pair to download traing data.
The url is https://vision.in.tum.de/webshare/u/zhouq/patch2pix/data_pairs/megadepth_pairs.ov0.35_imrat1.5.pair500.excl_test.npy.
Has anyone faced this issue?

Thank you for sharing your brilliant work.

I have a little question.

Can the pre-trained patch2pix_pretrained.pth model be used both indoor and outdoor ?

Hi Qunjie,

Thanks for releasing the code!

I observed a test-time crash that happens when there are no matches found. The trace is

  File "../localize.py", line 385, in main
    matches, _, _, _ = matcher(img1_name, img2_name)
  File "../localize.py", line 314, in <lambda>
    matcher = lambda im1, im2: model.match_pairs(im1, im2)
  File "/local/datasets/aachen_day_night/temp/image-matching-toolbox/immatch/modules/patch2pix.py", line 86, in match_pairs
    io_thres=self.match_threshold)        
  File "/local/datasets/aachen_day_night/temp/image-matching-toolbox/immatch/modules/../../third_party/patch2pix/networks/patch2pix.py", line 296, in refine_matches
    regressor=self.regress_mid)
  File "/local/datasets/aachen_day_night/temp/image-matching-toolbox/immatch/modules/../../third_party/patch2pix/networks/patch2pix.py", line 215, in forward_fine_match
    psize, ptype, regressor)
  File "/local/datasets/aachen_day_night/temp/image-matching-toolbox/immatch/modules/../../third_party/patch2pix/networks/patch2pix.py", line 177, in forward_fine_match_mini_batch
    f1s = f1s.view(-1, N, psize, psize).permute(1, 0, 2, 3)
RuntimeError: cannot reshape tensor of 0 elements into shape [-1, 0, 16, 16] because the unspecified dimension size -1 can be any value and is ambiguous

Here, I am using your image matching toolbox to use patch2pix with superglue features to match two images.
Do you know what the best fix would be?

AttributeError Traceback (most recent call last)
Cell In[1], line 29
25 else:
26 # Initialize patch2pix matcher
27 args = Namespace(io_thres=0.25, imsize=1024, ksize=2,
28 ckpt='../pretrained/patch2pix_pretrained.pth')
---> 29 matcher = init_patch2pix_matcher(args)
33 # Visualize matches on D2Net example pairs
34 for i in range(1, 4):

File ~/CodesWorld/Codes/patch2pix/examples/../utils/eval/model_helper.py:11, in init_patch2pix_matcher(args)
10 def init_patch2pix_matcher(args):
---> 11 net = load_model(args.ckpt, method='patch2pix')
12 matcher = lambda imq, imr: estimate_matches(net, imq, imr,
13 ksize=args.ksize,
14 io_thres=args.io_thres,
15 eval_type='fine',
16 imsize=args.imsize)
17 return matcher

File ~/CodesWorld/Codes/patch2pix/examples/../utils/eval/model_helper.py:60, in load_model(ckpt_path, method, lprint)
58 else:
59 lprint('Wrong method name.')
---> 60 net = Patch2Pix(config)
...
1613 return modules[name]
-> 1614 raise AttributeError("'{}' object has no attribute '{}'".format(
1615 type(self).name, name))

AttributeError: 'Conv4d' object has no attribute 'weight'

when l try to download hagia_sophia_interior.tar.gz, it is shown that Error 404. Where can l get this file? Thanks!

Hi,

This is really a nice work.

I am trying to reproduce the results in your paper.

However, I couldn't find data_pairs/prep_megadepth_training_pairs.ipynb and also don't have access to the link.

Could you upload the ipynb to the repo please?

Thanks.

Notice that there are R and t as inputs when computing F matrix which is used to generate geometric loss, but from my knowledge each image will have a R matrix and t matrix which are extracted from absolute pose. R and t here are elements from relative pose?