Excessive Memory consumption. about transformers.js

[flatsiedatsie]

WebGPU currently only supports 16 and 32bit mode.

[xenova]

5.31 Gb for Renderer process

Can you confirm you do not have any other tabs open? I can't seem to understand how this can be related to the application (not much is being rendered).

4.16 Gb for GPU process

This makes sense since it's (most likely) running in fp16 mode.

[iSuslov]

@xenova can confirm:

• no opened tabs
• no browser extensions

One empty tab opened:

Navigated to https://huggingface.co/spaces/Xenova/experimental-phi3-webgpu and loaded the model:

---

This makes sense since it's (most likely) running in fp16 mode.

Can we make it run in lower precision if we run q_4 quantization? (We can't, ONNX doesn't support 4q yet)

To avoid any confusion this is the downloaded model:
/Xenova/Phi-3-mini-4k-instruct_fp16/resolve/main/onnx/model_q4.onnx 838 mb
/Xenova/Phi-3-mini-4k-instruct_fp16/resolve/main/onnx/model_q4.onnx_data 1454mb

[everythinginjs]

same here