Errors running Zero-Inference text generation example about deepspeed-mii

[mrwyattii]

Hi @joaopcm1996 and thanks for trying out MII. Looks like we had a bug in the DeepSpeed config for text-generation-zero-example.py. I've fixed that in #88

With the updated script, I was able to run the example with:

python3 text-generation-zero-example.py
python3 text-generation-query-example.py -d "gpt2-xl_deployment"

The error you see when querying may be due to another MII server running in the background, but I'm not certain. You can shut down a server with:

from mii import terminate
terminate("gpt2-xl_deployment") #or replace with whatever the deployment name is

[jeffra]

@joaopcm1996, is this now resolved on your side? Please reopen if you still have an issue here.

[joaopcm1996]

I'll test it out with your recommendation and get back to you if I run into any issues. Appreciate the help!