Comments (7)
Are you referring to the calibration result from section 2.3.4 of the readme? With the csv titled "calibration_YYYY-MM-DD_HH-MM-SS.csv"? If so, just follow section 2.4 to get the final calibration parameters.
To go from roll, pitch, yaw, x, y, z to a transformation matrix, we first need to convert roll, pitch, yaw into a rotation matrix with rotation order XYZ.
Once you have the resulting 3x3 Rotation matrix from above, you can combine it with the, translation (x, y, z) in a single 4x4 homogenous transformation matrix T as shown below. This can be then used to convert points from camera to lidar frame.
There are different libraries for the above to help with rotation matrix/transformation like Eigen in C++ or scipy.spatial.transform.Rotation in python.
from cam_lidar_calibration.
Thanks for your reply.
I'm studying the paper you linked and I have some questions.
I understood that you extract normal vectors by using RANSAC and calculate some errors on lidar coordinates.
However, I want to know how you extract vectors from camera coordinates and how can calculate roll, pitch, yaw, x, y, and z from them.
from cam_lidar_calibration.
This can be then used to convert points from camera to lidar frame
I think I have a fundamental misunderstanding here @darrenjkt: Aren't projections from 2D -> 3D tricky, because (from the pinhole perspective) a point in 2d can be projected to any point along it's projection in 3d? The explanation above only projects camera to lidar, but how do I know where this projection and the lidar points intersect?
from cam_lidar_calibration.
@jiyou384 To extract vectors from camera coordinates, check out this function:
The main idea is that we define the chessboard in it's own 3D object frame (i.e. the chessboard is centred on the origin) using the hand-measured dimensions. In the object frame you can just define the normal vector as a unit vector on the +z axis, centred on the origin. We compute PnP using the 3D-2D correspondence of the hand-measured chessboard and cv::findChessboardCorners
to get the chessboard's pose in the camera's frame. After getting the pose (tvec, rvec), we transform the camera-captured board from it's 3D object frame to the 3D camera frame.
Once the chessboard is in the camera frame, that's when we run the optimisation to minimise the distance between normals, and board centres of the captured chessboard in camera and lidar frame.
@juliangaal Yes in certain cases with insufficient collected chessboard samples, there are many solutions for projecting 2D to 3D and that is why it is important to capture sufficient and good quality samples for calibration. Quality samples are those that have enough variance in their normal vectors as explained in Fig 5 of our paper. I'm not sure if this was exactly what you were asking but hopefully it provides some insight.
from cam_lidar_calibration.
Very helpful, thank you!
Yes in certain cases with insufficient collected chessboard samples, there are many solutions for projecting 2D to 3D
That's what I don't get: given the calibration T_L_C = [R t], applying this to a 2d image (T_L_C * img) isn't possible without extending the dimensions to 3d (technically 4D, for homogeneous coordinates). Even if there were a sensible way to do this, T_L_C doesn't include information about where to "cut" the projection vector from 2D to 3D.
Therefore, the only direction the transformation makes sense, is T_L_C_^-1 (as you do in image projection).
from cam_lidar_calibration.
Sorry for the confusion. When I mentioned "there are many solutions for projecting 2D to 3D", I was referring to the fact that the optimization can return different extrinsic transformation matrix as "solutions", but they might not all be the optimal one.
Image to world projection is a separate topic. You can't just apply T_C->L to the 2D image because T_C->L is a 3D transform, and the image pixels are in their own image plane. You need to first transform the pixels from the image plane to the camera frame itself. If you wanted to transform pixels to lidar, you'd need to go from Image Pixels (aka. plane) -> Camera frame (projection centre of your physical camera) -> Lidar frame
To get pixels in 3D coordinates in camera frame, you first define a 3D object (like our chessboard). Then you need to use PnP to get the T_O->C (Transform from object frame to camera frame). Once you have this, you can specify the object in 3D coordinates in the camera frame. From here, you can use the extrinsic transformation matrix you obtain with this repo T_C->L to transform the object's 3D coordinates to the lidar frame.
from cam_lidar_calibration.
Closing this because there is no activity. Feel free to open it again.
from cam_lidar_calibration.
Related Issues (20)
- Reprojection Image Seems Strange HOT 2
- tf static_transform_broadcaster does not fit calibration result HOT 3
- Help me please. HOT 1
- Does it creaate a problem with ros distriubutions. HOT 2
- Error when running the quick start HOT 3
- The chessboard has been filtered success. But the RANSAC unsuccessful occurs if i click capture sample HOT 1
- I am getting pcl::PCLBase<pcl::PointXYZIR> and undefined reference to `pcl::KdTreeFLANN<pcl::PointXYZIR, flann::L2_Simple<float> >::KdTreeFLANN(bool) while library linking HOT 2
- transforming results to lidar to camera frame
- Question about the paper HOT 1
- frame_id error Causing RViz Freeze During Background Capture HOT 2
- How the 'pose.csv' file in the article was obtained?
- Problem about `distance_from_origin` HOT 2
- Error when running the quickstart using Noetic HOT 1
- Strange calibration results and reprojection HOT 7
- why the data/vlp files in calib-v2 branches don't have pcd files? HOT 1
- Strange issue when using my own data. HOT 2
- checkerboard is not found HOT 1
- Excessive deviation in the estimated size of the checkerboard grid
- Can't filter point cloud with rqt HOT 1
- RANSAC Fails to Detect Model - Need More Lidar Points
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cam_lidar_calibration.