[model] StableLM 2 Zephyr 1.6b about web-llm

[CharlieFRuan]

Thanks for the list! WizardMath and OpenHermes can reuse the wasm of Mistral (as shown in prebuiltAppConfig in src/config.ts); CodeLlama should be able to reuse that of Llama-2, as long as they share the same quantization (e.g. q4f16_1) and number of params (e.g. 7B or 13B).

[flatsiedatsie]

I think I just found the files:

https://huggingface.co/OO8/1_6B_dev/tree/main

[flatsiedatsie]

Got stuck on:

[FATAL] /workspace/mlc-llm/3rdparty/tvm/include/tvm/runtime/packed_func.h:1908: Function tvmjs.array.decode_storage(0: runtime.NDArray, 1: basic_string<char>, 2: basic_string<char>, 3: basic_string<char>) -> void expects 4 arguments, but 3 were provided.
put_char @ web-llm.bundle.mjs:3421

[CharlieFRuan]

This is likely due to an old version of the web-llm npm (if you are not building from source). If you are building from source, this is likely due to the repo not up to date; try pull the recent changes

[flatsiedatsie]

It would be fantastic if this model could become part of the default supported models.

The multi-language ability is fantastic. I'm very impressed with it, especially for its size.

[flatsiedatsie]

Awesome, it seems the model has already become available in the Huggingface repo. The chunks exist:

https://huggingface.co/mlc-ai

However, the .wasm files are missing from binary-mlc-llm-libs. I've created an issue about that.

mlc-ai/binary-mlc-llm-libs#111

[CharlieFRuan]

Thanks for the request! We should be able to add the prebuilt wasm files in shortly. cc @YiyanZhai

[flatsiedatsie]

Fantastic! Thank you!

[flatsiedatsie]

For the record, I think there are more models for which the shards are available, but the wasm files are not (yet).

• Music
• WizardMath
• Gorilla
• Gemma 7B
• CodeLlama
• OpenHermes