Error : Instance no Aligned! about fast-ppf

[ktgiahieu]

If it is okay, you can provide some images of your point cloud and your 3D object so I can understand the situation better.

It is very hard for me to guess what is wrong with your data, since the method does not work on ALL type of objects and the parameters (such as leaf size) is very dependent on which type of camera you use, lighting circumstances, is the object mostly planar or edges, etc.
The method in this Fast-PPF repo is suitable for objects constructed by planar and cylinder shape, where as my other repo PPF-MEAM is more suitable for objects with a lot of edges. If your object does not falls into these 2 category, I would suggest choosing another method.

[Aotemanliu]

Dear author
Thank you very much for your reply.
Attachment 1 is the original data, and Attachment 2 is the adjusted data. Here's what I tested with this data set.
Thanks again for your help and looking forward to your reply!
附件2_adjusted data.zip

附件1_origin data.zip

[ktgiahieu]

With first glance, it seems that the background has not been discarded successfully, can you recheck the result of the background filter?

[ktgiahieu]

I have just had a look of your posted dataset and I must say the part has some small interior details. Because of it, when the camera captured the scene with it, the details are not going to be on the scene pointcloud and the normal vector are gonna be computed wrongly, hence the wrong detection. You should check the resolution of the captured scene and verify if my assumption is correct.

[Aotemanliu]

Dear author,
After receiving your reply, I checked the data set again and tested it again, but there was a new problem. What happened? Which parameter should I adjust?

[ktgiahieu]

It is related to ICP problems. Try tuning these:
https://github.com/ktgiahieu/Fast-PPF/blob/master/include/PPF/PPF.h#L26-L27

[Aotemanliu]

When I tested using your data set, the model and scenario were attached to each other, as shown below.

But when I tested with my own data set, the model and the scene didn't stick together, they were far apart.as shown below.

What is the reason for this?
For your convenience, I will upload some files to you. Attachment 1 is the original data set, Attachment 2 is the adjusted data set (to be consistent with the size of your data set), and Attachment 3 is how I ran the program.
附件1 original_data.zip
附件2 adjusted data.zip
附件3 Program processing.zip

[ktgiahieu]

There is something wrong with the Normal Calculation step, let me check.

[Aotemanliu]

Yes, when I tested with the data you provided, when I ran to step 7, the predicted model was attached to the scenario. But when I tested it with my own data set, it didn't stick together.

[ktgiahieu]

Did you change any lines of code?

[Aotemanliu]

As I had the error "Not enough correspondences found. Relax your threshold parameters."
I modified the "icp_max_iter" and "icp_corr_distance" parameters in the ppf. h header file in Fast PPF. The diagram below. That's what you gave me. The above error will not occur after modification.
Other than that, I didn't change any lines of code.

[ktgiahieu]

But after changing those lines, I guess that the Normals is still misplaced, am I correct?

[Aotemanliu]

But I don't have the idea to solve it now. Could you give me some hints?

[ktgiahieu]

I am guessing that there is some transformation done to the scene by mistake, and it effect the normal calculations. https://github.com/ktgiahieu/Fast-PPF/blob/master/include/PPF/PPF.cpp#L246-L247

[ktgiahieu]

I also noticed the cloud captured from your camera is orthogonal with Oy(green), while mine is Oz(blue). Maybe you should transformed the inputCloud so that it is similar to mine?
(P/s: somehow your normals in step 6 is also orthogonal to Oz, I guess that this approach should work)

[ktgiahieu]

Try adding
pcl::transformPointCloud(*latestCloud , *latestCloud , Eigen::Vector3f(0, 0, 0), Eigen::Quaternionf(0.7071, 0.7071, 0, 0));
after
https://github.com/ktgiahieu/Fast-PPF/blob/master/include/PPF/PPF.cpp#L159
(this Eigen::Quaternionf(0.7071, 0.7071, 0, 0) means rotates around Ox axis 90 degree, you can tune these values to suit your needs)

[wujilingfeng]

Try adding pcl::transformPointCloud(*latestCloud , *latestCloud , Eigen::Vector3f(0, 0, 0), Eigen::Quaternionf(0.7071, 0.7071, 0, 0)); after https://github.com/ktgiahieu/Fast-PPF/blob/master/include/PPF/PPF.cpp#L159 (this Eigen::Quaternionf(0.7071, 0.7071, 0, 0) means rotates around Ox axis 90 degree, you can tune these values to suit your needs)

Why is the normal calculation error？

[ktgiahieu]

I am just guessing at this point, we have to wait for him to clarify.

[Aotemanliu]

After adding code after line 159 in "ppf.cpp", the scene is perpendicular to OZ(blue). Unfortunately, the model and scenario did not match.

After the normals are calculated, the position of the scene changes.

[ktgiahieu]

Let's do a little debug. We will check if scene and scene_keypoints is transformed wrongly or not.
Try adding these lines after https://github.com/ktgiahieu/Fast-PPF/blob/master/include/PPF/PPF.cpp#L258

cout << "DEBUG: scene " << std::endl;
customViewer.viewer->removeAllShapes();
customViewer.viewer->removeAllPointClouds();
customViewer.viewer->addPointCloud(scene);
std::getchar();
cout << "DEBUG: scene_keypoints" << std::endl;
customViewer.viewer->removeAllShapes();
customViewer.viewer->removeAllPointClouds();
customViewer.viewer->addPointCloud(scene_keypoints);
std::getchar();

Then can you captured those 2 debug positions and post them here?

[Aotemanliu]

I've added the code the way you did, the 2 debug positions are shown below.

[ktgiahieu]

Is that with or without the pcl::transformPointCloud(*latestCloud , *latestCloud , Eigen::Vector3f(0, 0, 0), Eigen::Quaternionf(0.7071, 0.7071, 0, 0)); ?

[Aotemanliu]

yes, i added this code to it.

[ktgiahieu]

Everything is fine haha, I really have no idea. Do you mind sharing your https://github.com/ktgiahieu/Fast-PPF/blob/master/include/PPF/PPF.cpp file?

[Aotemanliu]

this is "PPF.cpp"file.
PPF.txt

[ktgiahieu]

I tried printing out the values of the pointclouds and there is nothing wrong, only the visualizer displayed them wrongly. I guess that this is caused by converting .ply to .pcd file using some other software than PCL (the way you converted it). But the result is actually correct.
My advice is that you should try avoid converting .ply to .pcd

[Aotemanliu]

Dear author,
After receiving your reply yesterday, I checked the code and data set again.

1. In order to avoid the trouble caused by the conversion of ".ply "to".pcd "by the third party software, I wrote a piece of code to obtain the field cloud in".pcd "format and reduce its size to be similar to your data set (of course, the model will also be reduced in the same scale). Unfortunately, after the test, did not get good results.
2. Considering the error caused by normal vector calculation, I added normal vector visualization code in the project. When testing with my own data set, the normal orientation of the model was all inward, as shown in Fig 1. I thought it was this problem that caused the mismatch, I added the following code with all its normal directions facing outward, as shown in Fig 2. However, this change does not change the result of the mismatch.
   for (int i = 0; i < normals->points.size(); ++i) { (*normals)[i].normal_x = (*normals)[i].normal_x * (-1); (*normals)[i].normal_y = (*normals)[i].normal_y * (-1); (*normals)[i].normal_z = (*normals)[i].normal_z * (-1); }
   
   Fig 1
   
   Fig 2
   Because I see that in your data set, the normal orientation that you get when you compute the normal vector is not always inward or outward. So I think it is not necessarily the problem of normal calculation, of course you can give me some advice on normal calculation.
3. Yesterday, you reminded me that there might be a problem with the location of scene cloud in the coordinate system, and added code.
   pcl::transformPointCloud(*latestCloud , *latestCloud , Eigen::Vector3f(0, 0, 0), Eigen::Quaternionf(0.7071, 0.7071, 0, 0));
   When I used your data set for testing today, I found that although the axis of green (OY) was perpendicular to the field cloud, as shown in Fig. 3, good matching results could also be obtained, as shown in Fig. 4.
   
   Fig 3
   
   Fig 4
   Now I don't know what went wrong. Could you give me some hints? In addition, do you have other data sets? If convenient, could you please give me a copy so that I can test it again?
   Looking forward to your reply!

[ktgiahieu]

All of the scenes I captured is already uploaded in this repo, and I am sorry to tell you that I cannot obtain more scene data because my setup is already gone after I finished this project, unfortunately.
I also doesn't know the source of the issue, a hotfix could be adding pcl::transformPointCloud(*latestCloud , *latestCloud , Eigen::Vector3f(0, 0, 0), Eigen::Quaternionf(0.7071, 0.7071, 0, 0)); only to the scene and scene_keypoints before displaying them, not when doing calculations.

[Aotemanliu]

Where exactly should I add this code?

[ktgiahieu]

PPF.txt
I tried added it and it should be working for you (hotfix). However the matching part is not correct yet, feel free to tune the parameters.

[ktgiahieu]

It still needs some tuning for better result. Also feel free to remove some uncessary std::cout for debug purpose.
Maybe your "normals inwards, outwards" idea is crucial here.

[Aotemanliu]

Dear author
After testing with the "ppf.cpp" file you gave me, my data set also matched the result you uploaded, but the match was incomplete (inaccurate), as shown in Fig 1.

Fig 1
In addition, I also used other PPF projects to test my data sets, and this project did not include ICP algorithms. The PPF algorithm in the project can achieve better matching. As shown in Fig 2.

Fig 2
In your project, there is no initial matching between the model and the scene cloud before the implementation of ICP algorithm, as shown in Fig 3. Only after the implementation of ICP algorithm, that is after Step 9, can the model and the scene cloud match.

Fig 3
Can I understand that there is a problem in normal calculation, which leads to the failure of PPF algorithm to match?

Or before the implementation of ICP algorithm, PPF algorithm did not play its due role?

[ktgiahieu]

I think you should try detecting 1, then 2, then more parts in the scene. By testing with less objects, we can understand better those questions you asked earlier. It is hard for me to guess the problem as well.