cohere model support request about auto-round

[wenhuach21]

Hello, it seems that the CoHere model is mainly tailored for RAG. Although the LLM component is expected to be supported, it's worth noting that the embedding model tends to be relatively small and computation-bound, hence it may not yield significant benefits from weight-only quantization.
Please feel free to correct me. We'll prioritize exploring this if it could bring large benefits to the community.

[wenhuach21]

Apologies for any confusion, just to clarify, are you referring to https://huggingface.co/CohereForAI/c4ai-command-r-v01 instead of https://huggingface.co/Cohere? We will have a test of https://huggingface.co/CohereForAI/c4ai-command-r-v01

[MichoChan]

Apologies for any confusion, just to clarify, are you referring to https://huggingface.co/CohereForAI/c4ai-command-r-v01 instead of https://huggingface.co/Cohere? We will have a test of https://huggingface.co/CohereForAI/c4ai-command-r-v01

yes，i mean https://huggingface.co/CohereForAI/c4ai-command-r-v01 and https://huggingface.co/CohereForAI/c4ai-command-r-plus

[wenhuach21]

yes，i mean https://huggingface.co/CohereForAI/c4ai-command-r-v01 and https://huggingface.co/CohereForAI/c4ai-command-r-plus

Ok, have you already met some issues? If the LLM is "typical", AutoRound should have already supported without requiring any additional code or configuration.
Anyway, we will have a test.

[MichoChan]

yes，i mean https://huggingface.co/CohereForAI/c4ai-command-r-v01 and https://huggingface.co/CohereForAI/c4ai-command-r-plus

Ok, have you already met some issues? If the LLM is "typical", AutoRound should have already supported without requiring any additional code or configuration. Anyway, we will have a test.

Not yet，i will have a try

[wenhuach21]

I've conducted a preliminary test on https://huggingface.co/CohereForAI/c4ai-command-r-v01 and it appears that AutoRound works well. However, given our current prioritized tasks, we plan to postpone reporting the recipe until we have available compute resources.

[MichoChan]

thanks a lot

[wenhuach21]

close this as there is no issue, feel free to reopen it.