[Misc]: RuntimeError: Cannot find any model weights [vllm=0.4.0] about vllm

[vishwa27yvs]

Follow up:

the code seems to work when I commented the load_format, this defaults to Using model weights format ['*.safetensors'], is this expected?

model = LLM(
        model=model_name_or_path,
        tensor_parallel_size=num_gpus,
        trust_remote_code=True,
        download_dir=HUGGINGFACE_CACHE,
        # load_format="pt",
        max_num_batched_tokens = max_model_len,
        max_model_len = max_model_len,
        swap_space = 1
        # max_num_batched_tokens = 8192 # max_prompt_length
    )