Comments (3)
Hello.
We regularly use data with ARKit/ARCore poses (see the Wayspots dataset, for example), and we had generally good results even though the poses, as you mentioned, are not perfect.
I have a couple suggestions for you to try and investigate the issue.
First, can you try creating a visualization of the training and testing process? You can use the --render_visualization
flag in the train/test scripts (https://github.com/nianticlabs/ace#ace-training) to save a video showing the evolution of the trained network over the course of the training, and the predicted poses during testing.
That should:
- tell you whether the network is learning anything useful (the point cloud will rougly look like the scene) or it's just random.
- allow you to visualize the camera poses relative to the point cloud, to make sure your ground truth is good enough and there isn't a coordinate system error (flipped axis or similar issue).
If the output from the above suggestion seems reasonable, you could look into the inlier count for each predicted camera pose.
The metrics for the datasets we experimented with (7Scenes/12Scenes/etc...) measure the error over all the predicted pose, there isn't a concept of "recall" but, in practice, you can filter results by inlier count (see the replies to this thread) and return a failure instead of a pose which is grossly incorrect, since that's probably preferable.
As Eric said, estimates with inlier count below 100 are rarely trustworthy, whereas if the inlier count is above 400 then the results are typically pretty good.
from ace.
Managed to catch an issue of inverted axis by running the visualizer that was messing with the results. Works perfectly now, thank you so much for the awesome model @tcavallari. Our rotation error now hovers around 0-2 degree and translation hovers around 5cm which is wonderful!
The point cloud looks strikingly similar to the scene. When looking at a table, it's even possible to make out objects like laptops and backpacks :)
from ace.
Summary results when we fed posed ARKit images walking around a table with some objects on it:
INFO:__main__:===================================================
INFO:__main__:Test complete.
INFO:__main__:Accuracy:
INFO:__main__: 10cm/5deg: 98.0%
INFO:__main__: 5cm/5deg: 78.4%
INFO:__main__: 2cm/2deg: 20.3%
INFO:__main__: 1cm/1deg: 3.4%
INFO:__main__:Median Error: 0.7deg, 3.7cm
from ace.
Related Issues (20)
- Cannot download encoder weights HOT 1
- Consistency in pose format for train/validation HOT 1
- Support principle point intrinsic parameters in training HOT 3
- Wayspot Datasets Visualization HOT 1
- Running inference on mobile HOT 4
- Peformance vs hloc HOT 3
- Need some clarification on what pose file mean HOT 4
- Does ACE not support large dataset? HOT 4
- 3D models for the wayspot dataset HOT 2
- fx and fy in test_ace.py HOT 2
- Confusion about ACE coordinate frame
- Where does ACE freezes the backbone when training the head HOT 1
- Pretrain Feature Extractor Backbone Network from ScanNet HOT 2
- About 7_scenes rgb calibration HOT 2
- About _convert_cv_to_gl(pose) HOT 3
- Reprojection Error HOT 3
- Encoder Training
- Encoder Training HOT 1
- Question with regards to results on Indoor6 dataset HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ace.