Comments (7)
The problem is related on how Chrome manages quota and how much can be stored.
The maximum data allowed to store is severely restricted depending on opaque responses, incognito sessions and a very common setting that seems to be found enabled in MacOS by default:
If the user has enabled "Clear cookies and site data when you close all windows" in Chrome, the storage quota is significantly reduced to a maximum of approximately 300MB.
To avoid the restriction and be able to have bigger cache just change this setting on Chrome. To my mind this is a bump. Also que can confirm that IndexedDB is far better that caches as the limit is much higher.
from web-llm.
Really appreciate the findings! Will update the doc
from web-llm.
I think this information would be very valuable in docs.
from web-llm.
For users who can't change the "Clear cookies and site data when you close all windows" settings in Chrome (e.g. when you use a company laptop with organisation-managed settings), the only option that seems to work is to pack code into a Chrome extension with unlimitedStorage
permission, based on my tests (on M1 Mac & Chrome 125.0.6422.77 ).
e.g. the chrome-extension-webgpu-service-worker example, you will need to change the permissions
in manifest.json
to:
"permissions": ["unlimitedStorage", "tabs", "webNavigation"]
from web-llm.
Its related to the cache quota. For some reason in this OS and Chrome is 345MB
from web-llm.
I can it force to be bigger and it works 🤔
from web-llm.
Interestingly enough if I open a new tab without any URL the inspector shows a much larger limit
from web-llm.
Related Issues (20)
- model request: Llama-3-8B-Web
- How to actually use WebLLM HOT 3
- wasm optimization? HOT 1
- Microsoft just released a more capable new version over Phi 3 Mini
- Example for using web worker with next js HOT 1
- Error: Failed to execute 'mapAsync' on 'GPUBuffer'
- How to let the user cancel loading the model and stop it from fetching params HOT 3
- Which LLM models can run on 6GB RTX 4050? HOT 2
- [Bug] Converted model outputs gibberish text HOT 1
- TOO SLOW in downloading models from huggingface when running 'mlc_llm package'
- Can I initialize existing model with random weights?
- Deply llama 3 40 billion parameters model HOT 4
- Sending raw text to the model HOT 4
- Deploy small LLM in a chrome extension HOT 2
- Runing LLM in a webworker fails due to loglevel dependency HOT 1
- support concurrent inference from multiple models HOT 4
- anyone tried to run web-llm in tauri?
- Request: Allow deletion of individual cached models. HOT 3
- LLama 3.1 Error: Device was lost during reload. This can happen due to insufficient memory or other GPU constraints. Detailed error: [object GPUDeviceLostInfo]. Please try to reload WebLLM with a less resource-intensive model. HOT 4
- Custom model outputs garbage in firefox nightly, works fine in chrome.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from web-llm.