ishanchadha01 / photometric-reconstruction-endoscopy

Photometric Single-view Dense 3D Reconstruction in Endoscopy

Implementation for MICCAI 2022 paper Photometric single-view dense 3D reconstruction in endoscopy by Víctor M. Batlle, J.M.M. Montiel, and Juan D. Tardós.

**[Paper]

Setup

Make sure Eigen, OpenCV, and Ceres are installed. Add Ceres installation location to environment variable called CERES_PATH.

After cloning the repo, run the following:

mkdir build
cd build
cmake ../photometric_reconstruction
make

Edit the get_camera_params() function with your camera's parameters based on the paper.

Reconstruction/Visualization

Example images included in images directory:

Original:

Reconstructed:

Evaluation

TODO

🤝 Contributing

Contributions from the community are welcome! If you have any ideas or suggestions for improving the library, please feel free to: submit a pull request, raise an issue, e.t.c.

Bibtex

If you find this work helpful, you can cite our paper as follows:

@INPROCEEDINGS{
  9981742,
  author={Batlle, Víctor M. and Montiel, J.M.M. and Tardós, Juan D.},
  booktitle={2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, 
  title={Photometric single-view dense 3D reconstruction in endoscopy}, 
  year={2022},
  volume={},
  number={},
  pages={4904-4910},
  doi={10.1109/IROS47612.2022.9981742}
  }

photometric-reconstruction-endoscopy's Issues

Add dataset and calibration instructions to README

Make camera model unprojection faster

Bottleneck in computing unprojection coefficients:

// Kannala-Brandt camera model
std::vector<double> unproject_camera_model(cv::Vec3b pixel) {
  std::vector<double> camera_params = get_camera_params();
  double cx = camera_params[0];
  double cy = camera_params[1];
  double fx = camera_params[2];
  double fy = camera_params[3];
  double k1 = camera_params[4];
  double k2 = camera_params[5];
  double k3 = camera_params[6];
  double k4 = camera_params[7];

  // Unproject from 2D pixel to 3D point in the world
  double mx = (pixel[0] - cx) / fx;
  double my = (pixel[1] - cy) / fy;
  double r = sqrt(pow(mx, 2) + pow(my, 2));

  // Find roots for model k4(x^9) + k3(x^7) + k2(x^5) + k1(x^3) + x = r where x=theta
  Eigen::VectorXd coeffs(10);
  coeffs << -1*r, 1.0, 0.0, k1, 0.0, k2, 0.0, k3, 0.0, k4; 
  Eigen::PolynomialSolver<double, Eigen::Dynamic> solver;
  solver.compute(coeffs); // TODO: bottleneck, is there a faster way to do this??
  const Eigen::PolynomialSolver<double, Eigen::Dynamic>::RootsType &roots = solver.roots();
  for (auto root : roots) {
    // Check if complex part of root is 0, if so return solution
    if (root.imag() == 0) {
      return {mx, my, (double) root.real()};
    }
  }
  return {mx, my, 1};
}