Comments (3)
the inverse warp is both dependant on depth and pose. While ground truth depth and pose will generate the correct warping, every variation in the same form but with depth and pose translation multiplied by a scale factor will also generate the correct warping.
Here, since we learn both by inverse warping, we end up with that scale factor, which can then be determined with pose comparison with groundtruth, i.e. vehicle speed.
from sfmlearner-pytorch.
That’s quite clear thank you:) So I wonder if it is possible to produce true depth since KITTI dataset provides calibration ? For example use depth = focal length * baseline / disparity instead of depth = 1 / disparity?
from sfmlearner-pytorch.
The baseline is never used here, since we only work with monocular cameras. The key is to know the ground truth displacement magnitude. In the case of learning inverse warping from stereo, displacement is indeed only the baseline which makes the depth very easy to learn with the right scale factor.
Here, since we compute the inverse warp according to the actual displacement of the car, you need to 1) figure out the displacement magnitude of the car. From GPS values or even the wheels speed, it's not very hard fgure out, with the frames timings.
2) compare it to the translation magnitude estimation from posenet and figure out the scale factor so that the depth is rescaled accordingly
This is in essence what's done in the test_disp script that I provide. The obvious drawback is that you need to know speed during training. Or you will have to run posenet during evaluation.
from sfmlearner-pytorch.
Related Issues (20)
- What happens if I use 3 or more frames? HOT 1
- train with my own video HOT 1
- what's the minimal files required to train depth only model HOT 1
- Query regarding depth map. HOT 2
- Large Errors on Pose Prediction Network HOT 3
- why the gpu memory cost of tensorflow version is larger than pytorch version HOT 2
- Weird results from pretrained model on KITTI images HOT 4
- Question about using oxts data HOT 1
- Cannot run `train.py` with nohup HOT 2
- imread during inference load the image as uint8 HOT 4
- How About the Flops, fps and parameter of this model? HOT 1
- regarding the predicted depth map during inverse warp HOT 2
- How to visualize the warped image (ref_img_wapred) HOT 2
- regarding inverse_warping HOT 10
- Is the image input of depth network fixed? HOT 2
- Question about diff
- How to load training dataset
- Regarding the depth used for generating target image HOT 5
- Question about the poses predicted by the posenet HOT 2
- about the pose scale HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sfmlearner-pytorch.