ahnsun / merlin Goto Github PK

what's the difference between Merlin and Merlin-Chat models?
How long time does it cost to train the model?
how many GPUs does the model need to train?

Hello,

Your project seems really interesting.
I have a question regarding the execution of sh playground/merlin/clip-large+conv+vicuna-v15-7b/pretrain.sh.
In the file, it says --model_name_or_path /path/models--lmsys--vicuna-7b-v15 \.
If I want to use lmsys/vicuna-7b-v15 as the text decoder, do I need to download the models manually, place them in a specific path, and modify the path accordingly?
Should I download all the files and place them in a specific path as shown in the picture below?

I would appreciate it if you could provide a guide on how to set up the vicuna-7b-v15 model.

And could you please let me know the required CUDA version to run this file?
I encountered an error stating that the libcudart.so.12 file is missing, so I set up the environment with CUDA 12.0. However, I got an error indicating that the version does not match with flash-attn. When I switched to CUDA 11.7, I again encountered the missing libcudart.so.12 file error.

Thank you.

I tried
python -m mmgpt.engine.serve.cli --model-path ./mmgpt/engine/serve/merlin --image-file ./mmgpt/engine/serve/examples/waterview.jpg

but there is error

File "/merlin-main/mmgpt/engine/serve/cli.py", line 53, in main
image_tensor = image_processor.preprocess(image, return_tensors='pt')['pixel_values'].half().cuda()
AttributeError: 'NoneType' object has no attribute 'preprocess'

Hi,
I'm so fascinated with your work, Merlin!
Do you have any plan to release the reformatted datasets used during the pretraining/SFT?

Thank you in advance!

Hello,
Your project is interesting.
But the link you gave in the readme is for clip-vit-large-patch14-224, and I can't find clip-vit-large-patch14-448 on the huggingface, can you updata the link for the clip-vit-large-patch14-448?