Comments (3)
Thanks! I think I got it working now. I was interested in using just the OccAnt model, not the mapper, so using just RGB, Depth and ego map as inputs worked.
from occupancyanticipation.
Hi @jaidevshriram,
The model currently depends on having RGB-D + odometry inputs. I don't see a way to get around having no pose inputs whatsoever. The pre-trained models are trained to handle a specific noise model in the odometer sensor (see code here). Assuming that you have perfect pose estimates or noise similar to the above model, you would just have to ensure that the inputs are in the expected format to the mapper (see here). This specific function might offer the best insights for what you're trying to achieve. The interface to the mapper module was designed to be easy to understand. In your case, you would have to input the following variables to update the map at time t:
"rgb_at_t_1" - RGB image at t-1
"depth_at_t_1" - Depth image at t-1
"ego_map_gt_at_t_1" - Egocentric local map at t-1
"pose_at_t_1" - Estimated agent world pose at t-1 (relative to starting position) -- this may have noise in it
"pose_hat_at_t_1" - Agent corrected pose at t-1 --- the agent corrects for the noise (if trained to do so)
"map_at_t_1" - Full world map from 0 to t-1 (agent starts at the center of this map facing north)
"rgb_at_t" - RGB image at t
"depth_at_t" - Depth image at t
"ego_map_gt_at_t" - Egocentric local map at t
"pose_at_t" - Estimated agent world pose at t --- this may have noise in it
This should be a good starting point. I hope you can develop the code based on this. Feel free to bring up any specific questions you have here.
from occupancyanticipation.
Glad to hear that @jaidevshriram . I'm closing this for now. Please feel free to open it in case you have further questions.
from occupancyanticipation.
Related Issues (20)
- actuation noise HOT 4
- Mapper training during policy training HOT 2
- Not able to load trained models HOT 9
- How is "done" variable being defined in exploration task? HOT 1
- What's does height_thresh means when generating map? HOT 2
- RuntimeError when running run.py for training HOT 1
- How to set appropriate values in .yaml file to avoid low GPU-uti but high GPU Memory-usage? HOT 3
- Normalization of image in Mapper HOT 2
- How to generate episode datasets for exploration task after changing the height and radius of agent? HOT 2
- No module named 'habitat_baselines.common.env_utils' HOT 3
- Run OccupancyAnticipation in Colab HOT 1
- What's the behind meaning of "GT_TYPE: wall_occupancy" in the configuration .yaml file? HOT 2
- Unexpected results for pre-trained model HOT 1
- RuntimeError: inverse_cuda: For batch 0: U(163642880,163642880) is zero, singular U HOT 1
- mp3d configs HOT 2
- RuntimeError: Error(s) in loading state_dict for Mapper: HOT 1
- What does map_size mean? HOT 1
- How to use AR? HOT 3
- Question about local masks
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from occupancyanticipation.