trainy-ai / llm-atc Goto Github PK

Hey there, I've been trying to follow the instruction, when I use this command it tells me --envs not found and also in llm-atc serve --help don't show this option

llm-atc serve --name meta-llama/Llama-2-7b-chat-hf --accelerator V100:1 -c servecluster --cloud aws --region us-east-2 --envs "HF_TOKEN=token"

Usage: llm-atc serve [OPTIONS]
Try 'llm-atc serve --help' for help.

Error: No such option: --envs

Apparently Skypilot uses spot instances. Is there currently a way to activate this option when using llm-atc?

I was attempting to follow the instructions provided for replacing the OpenAI API using llm-atc and I received the following error message when running the serve command that is provided in the
documentation:

I 09-15 11:11:41 optimizer.py:821] 
E 09-15 11:11:41 storage.py:762] Could not get StoreType.S3 store with name llm-atc.
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadBucket operation: Forbidden

The above exception was the direct cause of the following exception:

sky.exceptions.StorageBucketGetError: Failed to access existing bucket 'llm-atc'. This is likely because it is a private bucket you do not have access to.
To fix: 
  1. If you are trying to create a new bucket: use a different name.
  2. If you are trying to connect to an existing bucket: make sure your cloud credentials have access to it. To debug, consider running `aws s3 ls llm-atc`.

Is this because the s3 bucket llm-atc is hardcoded in the serve.yml file_mounts property? Should this be a command line parameter?