Comments (4)
I guess you are right, the speed of io is too slow. One thing you can try to verify this guess is to iterate the dataloader without training and check the speed.
from giga.
yes,I found that loading the data took about the same amount of time as before with training.
from giga.
Do you get a similar speed when training giga-aff? If not, then the IO bottleneck is the occupancy data loading. The default size of each occupancy data file is 100K points, but we only sample 2048 points. So one thing you can try is to decrease the size and increase the number of occupancy files when generating data:
python scripts/save_occ_data_parallel.py /path/to/raw/data {number_of_points_per_file} {number_of_files} --num-proc 40
from giga.
Do you get a similar speed when training giga-aff? If not, then the IO bottleneck is the occupancy data loading. The default size of each occupancy data file is 100K points, but we only sample 2048 points. So one thing you can try is to decrease the size and increase the number of occupancy files when generating data:
python scripts/save_occ_data_parallel.py /path/to/raw/data {number_of_points_per_file} {number_of_files} --num-proc 40
Thank you for your reply. I found a more powerful computer, and now the training speed is normal!
from giga.
Related Issues (20)
- New dataset HOT 3
- Visualizing on custom dataset HOT 9
- Visualization of data generation HOT 2
- visual HOT 1
- How to use GIGA on real robot? HOT 2
- how to query at a higher resolution of 60×60×60 HOT 4
- Installation error: " LINK : fatal error LNK1181: can not open input file“m.lib” " HOT 4
- About NVIDIA Driver in WSL2 HOT 5
- Scene Descriptor HOT 2
- [Question] How to get the 3d reconstruction at inference time? HOT 6
- No module named vgn HOT 2
- libmesh failed! HOT 2
- question about GIGA(HR) HOT 1
- Train GIGA HOT 1
- Can't log _aff.obj when running sim_grasp_multiple.py HOT 1
- Some confusion in the paper HOT 2
- The program that generates data gets stuck in the first loop HOT 2
- The time to generate the training set HOT 1
- Re-implentation in real world HOT 6
- Did you consider trying to avoid using the grasp data on the wrong voxels? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from giga.