yzhq97 / cnn-registration Goto Github PK

Would you like to upgrade the project to be compatible with newer library versions and Python version?

Hello, I came across to this nice project and it seems quite powerful. Have you experienced to use it also for large histology images? Would you be interested in participating in AHNIR challenge?
You may also see other interesting methods from this domain Borda/BIRL#2

Hi!

First of all congratulations for your work!

I would like to know if there is the possibility to incorporate your algorithm inside
a wider network that some point uses it.

Dear Z. Yang and T. Dan and Y. Yang,

This is not much of an issue with the work that is produced by you, but more of a question regarding the capabilities of your work.

I am currently working on a project that tries to employ machine learning networks to detect, describe, and match keypoints for the purpose of matching 2D robot LiDAR maps. So far, I have been tinkering around with panorama techniques utilising SIFT and the OpenCV Stitcher class, and they both yield successful results only to a certain extent due to the nature of the images being feature-poor.

As such, I would like to ask: Can your cnn-registration work stitch overlapping images? (From what I understand, the registration produced by the code is equivalent to homography that is produced by OpenCV's cv2.findHomography)

If proven successful, I would like to referrence you in my Final Year Project.

Hi, @yzhq97

I met this error when run demo.py

oem@sgi:/cnn-registration/src$
oem@sgi:/cnn-registration/src$ python3 demo.py
Traceback (most recent call last):
File "demo.py", line 2, in
import Registration
File "/home/oem/cnn-registration/src/Registration.py", line 10, in
from lap import lapjv
ModuleNotFoundError: No module named 'lap'
oem@sgi:/cnn-registration/src$
oem@sgi:/cnn-registration/src$

What's wrong with me?
I already installed pip3 install lap.
but this error happen.

Thanks in advance~

from @bemoregt.

Hello, I have so problem with running the demo even after porting to py3...
Could you add package version to requirements.txt to be able to run the demo, Thx

I was wrong about before, so I deleted the comment.

I am curious however if this supports images of different dimensions or if the input image must always be a static size?

Hello,I have some questions.
Do you need to ensure that the resolution of the two pictures is the same?
Thank you！

Hi,

How can we utilise this to image change detection ie 👍

any idea ?

Is it possible to extract the deformation matrix in this registration? And if so, where in the code can this be found?

Hi,When i run python3 demo.py in the root src , i faced with that error ,how should i solve it?
thank you.

I notice that if I translate IX by:

rows,cols = IX.shape[:-1]
M = np.float32([[1,0,100],[0,1,50]])
IY = cv2.warpAffine(IX,M,(cols,rows))

the registered image output is highly warped. One would think that it would detect a simple rigid body translation in this case...? Is there a way to force only rigid transformations?

While executing the script found that the numpy.linalg.norm() process is taking much time for execution. I have tried by reducing the number of points passed to the function and got better throughput but not such satisfiable for me. In the current situation, it is taking about 3-4 seconds for one comparison. How can I reduce it to below 1 second without much compromising the accuracy.

Hello zhuoqian. Could you please share the codes in the paper "Non-Rigid Image Registration With Dynamic Gaussian Component Density and Space Curvature Preservation"? I'd appreciate if you can show your email address. Look forward to your reply. Thanks a lot.

In addition, your homepage ( http://yzhq97.github.io/) seems inaccessible.

what happen?

Did somebody figure out how to train this model on their own data set?

Can anyone help me by telling me how to display feature detected from the images and the matched features between the two images from the code?
As there are images for them in the paper but I can't display them from the code.

What does the third picture in the imshow window mean? , it seems that the first two pictures are just cropped and misplaced. Shouldn't image registration be to get an affine transformed image?

Hi, thank you for sharing your exciting work. I would ask how did you compute the RMSD, MAD, MED and STD? have you computed them between the registered and sensed image OR registered image and ground truth?

Please assist with this as it is not clear to me.
Kind regards,

There are some minor modifications in order to make this great project work with python3

1. In shape_context.py, change from utils import pairwise_distance to from utils.utils import pairwise_distance

2. In utils.py, change all instances of xrange to range

3. In utils.py, line 117, change hi, wi = height/n, width/n to hi, wi = height//n, width//n

Hope it helps.

img_homo_gray1= cv2.cvtColor(IX, cv2.COLOR_BGR2GRAY)
img_homo_gray2= cv2.cvtColor(registered, cv2.COLOR_BGR2GRAY)

MIN_MATCH_COUNT =5
if matches>MIN_MATCH_COUNT:
M, mask = cv2.findHomography(X,Z, cv2.RANSAC,5.0)
matchesMask = mask.ravel().tolist()
print(matchesMask)
h,w =img_homo_gray1.shape
pts = np.float32([ [0,0],[0,h-1],[w-1,h-1],[w-1,0] ]).reshape(-1,1,2)
dst = cv2.perspectiveTransform(pts, M)
img_homo_gray2= cv2.polylines(img_homo_gray2, [np.int32(dst)], True, 255, 3, cv2.LINE_AA)
else:
matchesMask = None

#calaulate RANSAC matching point
sum = 0
for i in range(1,matches, 1):
if mask[i][0]==1:
sum += 1
print("RANSAC matching point: ",sum)
mismatching=matches-sum
print("mismatching:",mismatching)

#---------------------------------------------------------------------------------

Question: I don't know how to draw the matching line after doing RANSAC?

Hi,
When i run python3 demo.py in the root src , i faced with that error , why?
How do i change to vgg15 with other cnn?

Is there a code to tr-train this model for my dataset?

Hello,
Thanks for the model! I ran it to align two "satellite" images from different sensors (NAIP, DigitalGlobe). Unfortunately, the results look pretty unsatisfactory (see image). Do you have any ideas/thoughts how the results could be increased?

Thank you!
Björn

Dear author,

I have tried to use your CNN based image registration method but got unexpected results. Here I have attached the fixed, sensed, registered and cp images. please take a look at it and let me know if the results are supposed to be like that. Initially, I was expecting an unbent registered image which doesn't have those extra black pixels.

Thanks.

Hi ,I want to run your code ,but i do not have the dataset,so where can i download the dataset.thank you.

I saw have two image ( 1a.jpg and 1b.jpg) in the img folder. They was used for comment line (python demo.py).
Can you explain for me, how do the registration work? And If we have many sameples, how do we use this code? For example, we have many samples, as following:

thank.

how to draw the matches just like done in the paper?

Thank @yzhq97.
Can you share the train.py file in order to use for the training step?
Because, with your code we only use the demo.py file. Now, We want to do train step from the begining.

Thank you @yzhq97.

Great code, thanks for sharing!
However, the filtering of matching feature points and the display of incorrect and correct matching points in blue and yellow respectively mentioned in the article seem to have no function definition. Could you please update this part?
Because I mainly want to use the extracted correctly matched feature points for other tasks!
I can't understand the code well enough to write the function to realize the above functions. If I can receive your share, it will be very helpful to my work. Looking forward to your reply!!
Thank you very much!!

Thank you very much for sharing the code. Now I want to take a look at the matching of feature points, and I have some difficulties in adding drawmatches function. Could you please provide the code to implement this function according to your code idea?
This will be of great help to me, thank you very much!