[Bug]: Failing to find LoRA adapter for MultiLoRA Inference about vllm

[RonanKMcGovern]

Can someone assist on this bug? Thanks

[LattaruoloAndrea]

Hello I do have the same issue but from loading the adapter from a local directory

docker run  -v  /.cache/huggingface:/root/.cache/huggingface --env "HUGGING_FACE_HUB_TOKEN=..." -p 8000:8000 --ipc=host vllm/vllm-openai:latest --model meta-llama/Llama-2-7b-hf --enable-lora --lora-modules sql-my-test-lora=./FINE_TUNED_MODELS_TEXT_TO_SQL/checkpoint-500/

And I get the following error

    lora = self._lora_model_cls.from_local_checkpoint(
  File "/usr/local/lib/python3.10/dist-packages/vllm/lora/models.py", line 214, in from_local_checkpoint
    with open(lora_config_path) as f:
FileNotFoundError: [Errno 2] No such file or directory: './FINE_TUNED_MODELS_TEXT_TO_SQL/checkpoint-500/adapter_config.json'

but inside the directory there is the adapter_config.json (notice I also display the models)

curl http://localhost:8892/v1/models  
{"object":"list","data":[{"id":"meta-llama/Llama-2-7b-hf","object":"model","created":1716542398,"owned_by":"vllm","root":"meta-llama/Llama-2-7b-hf","parent":null,"permission":[{"id":"modelperm-9dcdb84d92de4bc7ba960fa844213025","object":"model_permission","created":1716542398,"allow_create_engine":false,"allow_sampling":true,"allow_logprobs":true,"allow_search_indices":false,"allow_view":true,"allow_fine_tuning":false,"organization":"*","group":null,"is_blocking":false}]},{"id":"sql-my-test-lora","object":"model","created":1716542398,"owned_by":"vllm","root":"meta-llama/Llama-2-7b-hf","parent":null,"permission":[{"id":"modelperm-403c7eca17474cf795a88605cd75fa47","object":"model_permission","created":1716542398,"allow_create_engine":false,"allow_sampling":true,"allow_logprobs":true,"allow_search_indices":false,"allow_view":true,"allow_fine_tuning":false,"organization":"*","group":null,"is_blocking":false}]}]}

@RonanKMcGovern did you try to load adapters from a local directory?

Thanks