koide3 / st_handeye_graph

General hand-eye calibration based on reprojection error minimization and pose graph optimization

st_handeye_graph

This package provides a general hand-eye calibration method which can be applied to pinhole and source-detector camera models.

Description

Unlike usual hand-eye calibration techniques, this method directly takes the images of the calibration pattern (like chessboard) and estimates the hand-eye transformation and the pattern pose such that the projection error of the pattern is minimized. Since it doesn't rely on algorithms dedicated for pinhole cameras, such as PnP algorithms, it can be easily adapted to different camera model by changing only the projection model.

Dependencies

• g2o
• VISP
• handeye_calib_camodocal (optional, for only evaluation)

Usage

Include "st_handeye/st_handeye.hpp" and call "st_handeye::spatial_calibration_graph()". See "st_handeye.hpp" for the details.

Reproduce figures

git clone https://github.com/koide3/st_handeye_graph.git
git clone https://bitbucket.org/koide3/st_handeye_eval.git

mkdir st_handeye_graph/build
cd st_handeye_graph/build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j4

cd ../scripts
# run simulations and plot figures
./run_simulation.sh

# run calibration on real data and perform 3d reconstruction
./run_calibration.sh

# see the reconstructed point cloud
pcl_viewer data/points_3d_graph.pcd

Paper

Kenji Koide and Emanuele Menegatti, General Hand-Eye Calibration based on Reprojection Error Minimization, IEEE Robotics and Automation Letters/ICRA2019 [link].

Contact

Kenji Koide, Intelligent Autonomous Systems Laboratory, University of Padova, Italy. [email protected]

st_handeye_graph's Issues

Confusing notation

Thank you for making this codes public, first of all.

Why are there many codes such as following :

std::cout << "hand2eye ---\n" << hand2eye.matrix().inverse() << std::endl;

cv::Mat cv_object2eye = eigen2cvmat(object2eyes[i].inverse().matrix());

Shouldn't be the inverse of A2B B2A ?

Calibration board

Adding the calibration target as PDF to the repository would help me.

Best, Ralf

Calibration target

Adding the calibration target as PDF to the repository would help me.

Best, Ralf

Building Code on a New OS

Hello - is there any chance you could create a Docker image for this repository (and maybe update repository dependencies)? I am having many issues building it on a newer Ubuntu 20.04 OS with python3, OpenCV4, and C++17. If I recall correctly, this worked well on an Ubuntu 18.04 OS with python2, C++11, and OpenCV2. (I used C++11 for the build, but I have several OpenCV version conflicts).

Error:

[ 91%] Built target calibrate
/usr/bin/ld: CMakeFiles/handeye_simulation_xray.dir/src/handeye_simulation_xray.cpp.o: in function `HandeyeSimulationXray::visualize(Eigen::Matrix<double, -1, -1, 0, -1, -1> const&, int) const [clone .constprop.0]':
handeye_simulation_xray.cpp:(.text+0x456): undefined reference to `cv::imshow(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cv::_InputArray const&)'
/usr/bin/ld: CMakeFiles/handeye_simulation_xray.dir/src/handeye_simulation_xray.cpp.o: in function `main':
handeye_simulation_xray.cpp:(.text.startup+0x2aa9): undefined reference to `cv::imshow(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cv::_InputArray const&)'
/usr/bin/ld: handeye_simulation_xray.cpp:(.text.startup+0x3889): undefined reference to `cv::imshow(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cv::_InputArray const&)'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/handeye_simulation_xray.dir/build.make:157: handeye_simulation_xray] Error 1
make[1]: *** [CMakeFiles/Makefile2:198: CMakeFiles/handeye_simulation_xray.dir/all] Error 2
make: *** [Makefile:91: all] Error 2