Comments (3)
Installing 0.7.6
and 0.0.4
did not work for me.
Required the above PR to fix, installing deepseed
from source fixes this. Just waiting on PYPI release.
pip install git+https://github.com/microsoft/DeepSpeed.git
Then via ds_report
:
deepspeed info ................... 0.7.7+2076bf23, 2076bf23, master
After that, all worked!
from deepspeed-mii.
+1, I'm using EC2 g5.4xlarge (64GB RAM)
Deep learning AMI with pytorch 1.12.1 and CUDA 11.6
I am able to run the baseline script successfully
from deepspeed-mii.
This was resolved in microsoft/DeepSpeed#2536
Just tested and it's working. Please update DeepSpeed to 0.7.6
and installing deepspeed-mii from source or install version 0.0.4
when we release it later today.
from deepspeed-mii.
Related Issues (20)
- Base model support HOT 1
- Support for Codellama Model in deepspeed-fastgen HOT 2
- Invalid parameter bricks service HOT 1
- Deadlock detected HOT 12
- AttributeError: 'NoneType' object has no attribute 'value' HOT 2
- RuntimeError: There is no current event loop in thread 'Thread-1'. HOT 2
- support chatglm3 HOT 1
- Loading Bloom-3b in persisten mode fails HOT 1
- restful api host need configuration HOT 1
- Can deepspeed-MII run on AMD GPU? HOT 2
- The choice of the split size for splitAndFuse HOT 4
- How to stream tokens? HOT 1
- Error messages spilled from persistent deployment for every request HOT 6
- Can you support DeepSeek's inference acceleration? Thank you very much. HOT 4
- mixtral support HOT 2
- Deployment in kubernetes
- Reproduced readme results HOT 8
- Can MII support quanted Llama2 of AWQ?
- Problem while running facebook/opt-125m with MII HOT 2
- When running mii.serv, it keeps print waiting for server to start. HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deepspeed-mii.