Run a 125-billion-parameter AI model on a normal gaming PC
one NVIDIA card (12-24 GB) + 64 GB of RAM · Windows or Linux · one click to install

_{A voxel pagoda garden, 1 shot prompt running on an RTX 5070 with Strata (IQ3_S, 128K context) · full video (49 s)}

Strata runs Qwen3.8-Flash-Next - a large, smart AI model that normally needs a server - on your own PC. It writes its answers at 60-95 tokens per second (a token is about ¾ of a word): faster than you can read.

• Free and open source.

Jump to: How fast? · Which model? · Install · Using it · Problems? · How it works · All the details

Measured on an RTX 5070 (12 GB), a Ryzen 5 7600 and 64 GB of RAM:

• Writes answers = how fast the reply appears (tokens per second).
• Reads your prompt = how fast it takes in what you send (long documents, code, chat history).

A card with more VRAM is faster, because more of the model fits on the GPU: an RTX 3090 (24 GB) should do roughly 100-140 tokens per second. All measurements, long-context numbers and estimates for other cards are in the details.

The size (the same model, compressed more or less):

Will it fit? Shard 1 is the part of the model that gets loaded when it starts: its experts go into your RAM, the rest onto your graphics card (the second shard, a 29 GB lookup table, stays on the SSD). So it fits when your RAM is at least shard 1 + about 10 GB for Windows and your other programs. With 64 GB of RAM every size fits (IQ3_S with little else open); with 48 GB, Q2_0 and IQ2_XS. A bigger graphics card makes it faster, but it doesn't lower the RAM needed.

The version:

• Qwen3.8-Flash-Next - the original.
• Swift 1.5 - a fine-tune by UkisAI that thinks much shorter before answering, so you get the answer sooner, with about the same quality. Same speed per token, and about the same RAM as the same size of the original (no IQ3_S). Its own license applies (see its page).

Not sure? Take IQ2_XS. You can add another one later with START-HERE.bat --setup.

You need: an NVIDIA RTX 30, 40 or 50 card with 12 GB of VRAM or more, enough RAM for the size you pick (above), ~80 GB of free disk space (an SSD makes the first start much faster), and Windows 10/11 or Linux. The only thing you install yourself is a current NVIDIA driver (nvidia.com/drivers or the NVIDIA App). Everything else - Python, the engine, the model - is set up for you.

Windows

1. Download this project and unzip it (or git clone it).
2. Double-click START-HERE.bat.
3. Answer a few questions - or just press Enter each time for the recommended choice:
   • Which model and size? The original or Swift 1.5, and Q2_0, IQ2_XS, IQ3_XXS or IQ3_S - see above
   • How much context? How much text it can keep in mind at once (it suggests one for your card)
   • Images? Whether it should also read pictures
   • Experimental speed projection? Off unless you say yes - read what it does first

Then it downloads everything (the model is ~70 GB, so the first time takes a while - you can stop and it picks up where it left off) and starts the model. Your browser opens the Strata app at http://127.0.0.1:8080.

Next time, just double-click START-HERE.bat again: it starts right away, nothing is downloaded twice. Close its window to stop the model.

Linux: run ./setup.sh - same questions, same result.

_{The Strata app's Monitor (left) while a coding agent writes the pagoda garden from the video (right)}

• In the browser: http://127.0.0.1:8080 - the Strata app (it opens by itself when the model starts): Chat, a live Monitor of the model and your GPU/CPU/RAM, and About with the settings and addresses.
• Chat in the terminal: .venv\Scripts\python chat.py
• Your apps and coding agents: add it as an "OpenAI-compatible" provider with base URL http://127.0.0.1:8080/v1, any API key and any model name. Apps that use Anthropic's API: http://127.0.0.1:8080/v1/messages.
• Thinking: the model thinks before it answers. Choose off, low, medium or high - in the chat page menu, with /think low in chat.py, or with your app's "reasoning effort" setting. Off is fastest; high is best for hard questions.
• Pictures: in the chat page click Picture; in chat.py type /image <path>; in apps just attach them.
• From your phone or another PC: START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>, then open the address the server window prints; see the details.
• Experimental speed projection (off by default): an experimental control vector that setup can turn on; it changes how the model answers - read what it does first.

Good to know: it answers one request at a time. The first message of a chat is read in full (about 1 minute per 30,000 tokens); after that it keeps the conversation and reads only what is new, so follow-ups start in seconds.

My PC froze, or got very slow, the first time Strata started. That's normal the first time. Strata loads 35-55 GB into your RAM, locks part of it for the graphics card, and works out how much of the model fits on your GPU. The mouse can freeze for a few minutes. Wait, and don't close the window. The next starts are much faster. Still frozen after 10 minutes? Restart the PC, close other programs (browsers use a lot of RAM) and try again. If it keeps happening, pick a smaller size (Q2_0 or IQ2_XS).

It stopped while downloading or installing. Run START-HERE.bat again. It continues where it stopped.

It says the NVIDIA driver is too old. Update it (NVIDIA App or nvidia.com/drivers), restart the PC, and run START-HERE.bat again.

It says port 8080 is already in use. Strata is already running. Look for its window.

It's very slow and the disk light keeps blinking. Your PC is out of free RAM. Close other programs, or pick a smaller size (Q2_0 or IQ2_XS).

An answer stopped with "the engine stopped unexpectedly". Usually not enough RAM (on Linux the system then stops the engine). Just send your message again: Strata starts the engine by itself. If it keeps happening, close other programs or pick a smaller size.

It says the prompt exceeds the context. The conversation is longer than the context you chose. Start a new chat, or run START-HERE.bat --setup and pick more context.

Still stuck? Look in the full troubleshooting table, or open an issue and attach strata-<model>.log from the Strata folder.

A model this big doesn't fit on a gaming graphics card. Strata splits the work between the parts of your PC:

• The GPU runs the part of the model that is used for every word, plus the "experts" it needs most often.
• The RAM holds all 24,576 experts, and the CPU computes the few the GPU doesn't have - at the same time as the GPU.
• The SSD holds a big lookup table; the model reads a few rows of it per word.
• A small helper inside the model guesses the next words, and Strata checks several guesses at once. That makes it 1.6-1.8x faster than going word by word - and the answer is exactly the same.

The full story is in the paper and the details.

• Model: Qwen3.8-Flash-Next by the Qwen team; compressed versions by ISTA-DASLab; Swift 1.5 by UkisAI. Their licenses apply to the model files.
• Built with parts of llama.cpp / ggml (MIT). Ideas from Splash, ninfer and HyperQwen. More in the details.