Comments (1)
You can set cpu_offload=True
. See https://colossalai.org/docs/basics/booster_plugins#low-level-zero-plugin
For nvme offload, see https://colossalai.org/docs/features/nvme_offload
You can try full parameter training without offloading if your model and videos are small.
We only provide pretraining script now. Finetuning script may be provided in the future.
from open-sora.
Related Issues (20)
- torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 0 (pid: 28171) of binary: /usr/local/bin/python3 HOT 2
- about the usage of computational power from colossal cloud HOT 3
- my colossal cloud account cannot sign in! HOT 3
- how to use the four-grid midjourney images HOT 1
- for masked video generation, what is the meaning of edit ratio? HOT 2
- Was a pre-trained model used for opensora-v1-1 training? HOT 2
- batch_size大小 HOT 2
- Training loss problem
- A spatial-temporal mask is required HOT 1
- Problems about the pre-trained model for open-sora1.1 HOT 2
- OCR speed HOT 3
- aspect_ratio code error in this repository
- data_process中,使用load_4bit加载liuhaotian/llava-v1.6-34b模型报错 HOT 2
- Out-of-memory for default config. HOT 4
- 这个生成的视频prompt在哪里? HOT 2
- How to train a image to video model HOT 2
- The loss curve fluctuates greatly HOT 2
- why T5 must use fp32 instead of fp16 or bf16? HOT 2
- The memory usage during multi-GPU inference is the same as with a single GPU HOT 4
- use parallel_apply in sampler get same result in each epoch HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from open-sora.