hugbed / opens3d Goto Github PK

Integrate VideoCaptureDeviceDecklink in S3DAnalyzer.

This seems promising. Can I use it to just produce the stereoscopic image without any sort of putting in the file or rendering, so to get rid of ffmpeg or qt?

Thanks a lot.

Instead of left right images, only the right image is shown twice in the StereoSideBySideEntity

Currently only the inliers in the right image are returned.
Having both could be useful since vertical disparities are lost if we only have points in one image and disparities as the results.

Check for missing noexcept or const (for member functions not modifying member variables)

To rectify images from rectification matrix.

• Add function to compute rectification matrix from STAN model.
• Include debugging tools to see images after rectification.

If you press compute again, instead of rectifying the original image, it rectifies the previous rectified image since it retrieves the image from the textureManager.

Give better variable names for StanAlignment. Currently the variable names are based on the article (a, c, f). They should have readable names such as verticalOffset, rollAngle, Zoom, tiltOffset.

Note: where to put the units for this so it's not confusing and variable names are not too long?

It is hardcoded to 720p 60fps. It was 1080p 30fps before since the camera used previously only supported these specs. It would be nice to be able to set the resolution fps to different resolutions supported by the Decklink interface.

There will be classes children to computation classes (MatchFinder, DisparityAnalyzer) that will provide visual debugging tools).

It would be nice to use factories to enable these Debugging features through some flag.

Implement a function to draw features from a list of 3d points on cv image (same behaviour as cv::drawMatches)

Would prevent having to carry around cv::DMatch everywhere.

In order to compare the current algorithms to the articles and to visualize results. There should be simple programs using S3D features:

• Feature matching
• Epipolar line visualization
• Rectification (with epipolar lines)

Add a way to easily display feature matches (inliers) that correspond to the model computed by DisparityAnalyzer.

As stated in the article cited below, solving STAN alignment with tiltKeystone and zParallaxDeformation may lead to numerical stability issues since they depend on multiple parameters. These parameters should be removed from the estimation for use with RANSAC since a small error in the inputs may lead to inconsistent large output errors.

Note: it would be nice to control the number of parameters to solve without having to change the StanFundamentalMatrixSolver class.

Zilly, Frederik, et al. "Joint estimation of epipolar geometry and rectification parameters using point correspondences for stereoscopic TV sequences." Proceedings of 3DPVT. 2010.

A list of ImageOperation that can be toggled (on/off).
ImageOperations will have an input image, and an output image that will be the output of all the operation chain (each activated operation in order transform the image).

This will help with #31 since it will be possible the operations will always be done on the input image instead of the output image as of right now.

We should be able to toggle visualisation of:

• Raw features
• Filtered matches

Implement a class to display epipolar lines from fundamental matrix, 2 images and 2 sets of points (1 in each image).

Currently only points in the right image are output to results.

Convert DisparityAnalyzerSTAN::Results to STAN alignment

Dealing with videos, a frame may have a large number of detected features (lets say more than 100) while the next frame may have not enough features to give a correct answer (not enough inliers for ransac --> not large enough probability for a good answer).

The alignment and rectification parameters should not be updated if there are not enough features found.

It would be useful to be able to display the feature matches after rectification to see if they are on the same image row.

When the video ends, the play button stays in pause mode so the video cannot be restarted.

To prevent having to define (and repeat in each class) all the operations (move, copy) when defining a virtual destructor for an abstract class.

So:

1. Define a virtual destructor (or the destructors of the implementations classes will not be called when deleting them through the base interface)
2. Delete copy operations (construction, assignment) for base class since we don't know if derived class will be copiable.
3. Default move operations (construction, assignment) for base class to prevent the compiler from deleting them (move operations can be good optimizations).
4. Define default constructor (that would be otherwise deleted by deleting construction operations).
5. Define virtual clone operation to replace copy (owner pointer to use covariant return types that can't be used with smart pointers. States clearly that the returned value will be owned by the caller.

Interfaces should inherit privately this base class to use the features of this base class without any conceptual relationship between the 2 classes.

Quote from Scott Meyers' book :

• "Private inheritance is most likely to be a legitimate design strategy when you're dealing with two classes not related by is-a where one either needs access to the protected members of another or needs to redefine one or more its virtual functions."
• "Private inheritance means is-implemented-in-terms-of. It's usually inferior to composition"
  "If you make a class D privately inherit from a class B, you do so because you are interested in taking advantage of some of the features available in class B, not because there is any conceptual relationship between objects of types B and D."
• "Private inheritance means nothing during software design, only during software implementation."

Example:

#include <iostream>
#include <memory>
using namespace std;

class Empty {};

// defines virtual destructor for interface (pure abstract) class
// deletes copy (don't know if derived will contain incopiable types)
// allows move (everything is movable)

template <class B = Empty>
class rule_of_five_interface : B {
public:
  virtual ~rule_of_five_interface() = default;
  rule_of_five_interface(const rule_of_five_interface&) = delete;
  rule_of_five_interface& operator=(const rule_of_five_interface&) = delete;
  rule_of_five_interface(rule_of_five_interface&&) = default;
  rule_of_five_interface& operator=(rule_of_five_interface&&) = default;
  virtual gsl::owner<rule_of_five_interface*> clone() = 0;
protected:
  constexpr rule_of_five_interface() = default;
};

class Interface : rule_of_five_interface<> {
public:
   virtual int next() = 0;
};

class Impl : public Interface {
public:
  Impl() {}
  ~Impl() override = default;
  
  int next() override { return 1; }    
};

int main() {
  std::unique_ptr<Interface> p = std::make_unique<Impl>();
  cout << p->next() << endl;

  return 0;
}

References:

• Delete copy operations on abstract class
• noexcept move operations