Comments (6)
Thanks for trying this @Thytu. Small clarification here, the min version for SD is 0.7.4 which is the latest released to PyPI (https://pypi.org/project/deepspeed/0.7.4/). I'll update the min requirements for MII in an upcoming PR.
from deepspeed-mii.
Note : This issue still occurs when using the updated code from this PR.
from deepspeed-mii.
Hi @Thytu, I just tested the example and it's working for me. It looks like you have DeepSpeed v0.7.3 installed. Stable Diffusion injection policies were only recently added, and you will need to install v0.7.5 v0.7.4 or latest source:
pip install deepspeed==0.7.5
pip install deepspeed==0.7.4
or pip install git+https://github.com/microsoft/deepspeed.git
Let me know if that resolves the error.
@jeffra thanks for the correction
from deepspeed-mii.
Still encountering error with the requirement updated. To avoid further errors related to any dependencies, I suggested this feature request.
from deepspeed-mii.
@Thytu what error are you seeing after updating DeepSpeed? Also, are you providing the hf_auth_token
in the mii_config
from the example?
from deepspeed-mii.
@Thytu what error are you seeing after updating DeepSpeed?
My bad I answered here.
Also, are you providing the hf_auth_token in the mii_config from the example?
Yep 👌
from deepspeed-mii.
Related Issues (20)
- RuntimeError: server crashed for some reason, unable to proceed HOT 2
- The inference result is inconsistent with hf HOT 1
- TypeError: expected Tensor as element 0 in argument 0, but got bool HOT 1
- How to generate multiple responses in one time? HOT 1
- Is the DeepSpeed-MII will support habana (HPU) hardware? HOT 2
- How does GPT2/Bert models utilize continuous batching feature in MII? HOT 1
- Use of dtype in the mii fastgen HOT 1
- Fp6 eta HOT 2
- How to set trust_remote_code=True in pipeline HOT 2
- why all-reduce takes lots of time for mixtral which is quite larger than that of vllm and tensorrt-llm
- When I start server, after loading model, I got an error of 'grpc.aio._call.AioRpcError' HOT 5
- Add support for Gemma models HOT 1
- Speeding up loading in inference checkpoints HOT 2
- Requests.exceptions.ConnectionError: HOT 2
- How to use DeepSpeed-MII to deploy a LLM model from DeepSpeed/Megatron-DeepSpeed trained checkpoints? HOT 2
- MII Example shows that mii is "Slower" than Baseline!
- ValueError: Unsupported model type roberta HOT 2
- Can DeepSpeed-MII inference on multi gpus with only 1 replica? HOT 2
- Kernel execution error with long context length
- Workarounds for pre-Ampere devices HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deepspeed-mii.