Basalt on one RTX 5090: IQ3_S writes code 22 to 25% slower than IQ3_XXS with nothing left on the SSD

Question

On one RTX 5090 running Qwen3.8-Flash-Next with the Basalt engine, how much slower does a coding agent’s reply come with the IQ3_S pack than with IQ3_XXS, and what does it cost when some of the model’s experts have to be read from the SSD? The prompts are 6,756 to 8,576 tokens long; the engine’s context window is set to 262,144.

Qwen3.8-Flash-Next is a Mixture of experts A model whose layers hold many small sub-networks, the experts, of which only a few work on each token. The whole model must be stored, but each token uses only a small part of it, and the experts in use change from one token to the next.. Its two 3-bit packs use GSQ-RCO, a quantisation method from IST Austria, and differ in size and fidelity: IQ3_S has 46.8 GiB of experts and, against the 8-bit version of the model (the reference), a KL divergence A measure of how far one set of probabilities is from another. To judge a compressed model, its predicted probabilities for the next token are compared with those of a reference version over a test text: 0 means the same predictions, and lower is closer. of 0.155 and the same next token 87.3% of the time; IQ3_XXS has 40.0 GiB, 0.243 and 84.0% (the author’s figures, on the packs’ page). Neither fits on a 32 GB card: Basalt keeps about 29 GiB of weights on it, copies into RAM as many of the other experts as a limit we set allows, and reads the rest from the model file on the SSD while it writes.

Results

Three runs, with the RAM copy limited to 24, 17 and 11 GiB, and four starts of each pack in each, called draws; how it was measured is in the last block.

IQ3_S wrote slower than IQ3_XXS on every kind of output in every run, its four draws all below the four of IQ3_XXS except for prose with nothing left in the file (6.0% slower, draws mixed: not established). Code in the reply was 22.3% slower with nothing left in the file (24 GiB), 47.0% slower with 11% of IQ3_S’s experts left in the file (17 GiB) and 49.5% slower with a quarter left (11 GiB). The card also drew less power with IQ3_S each time.

Copy limit (GiB)	Pack	Prose (t/s)	Code in the reply (t/s)	Code in a Write call (t/s)	Reasoning (t/s)	Power (W)
24	IQ3_XXS	368.0	469.1	514.3	357.8	451
24	IQ3_S	345.8	364.5	387.3	312.9	402
24	change	−6.0%	−22.3%	−24.7%	−12.5%	−49 W
17	IQ3_XXS	355.2	472.9	528.5	350.2	454
17	IQ3_S	302.9	250.8	295.8	261.9	330
17	change	−14.7%	−47.0%	−44.0%	−25.2%	−124 W
11	IQ3_XXS	352.1	365.7	414.6	320.1	395
11	IQ3_S	271.4	184.5	198.2	197.3	252
11	change	−22.9%	−49.5%	−52.2%	−38.4%	−143 W

Medians of four draws per pack; t/s is tokens per second; change is IQ3_S against IQ3_XXS. Prompts of 6,756 to 8,576 tokens, context window 262,144. 24 GiB: 24 October 2026, 23:40 to 23:53 UTC; 17 GiB: 25 October, 11:52 to 12:06 UTC; 11 GiB: 24 October, 21:52 to 22:11 UTC.

What staying in the file costs

Code in the reply, against the same pack’s speed in the 24 GiB run, where nothing stayed in the file:

Pack	Left in the file (GiB)	Speed (t/s)	Of its full speed (%)
IQ3_XXS	3.9 (10% of its experts)	365.7	78
IQ3_S	5.3 (11% of its experts)	250.8	69
IQ3_S	11.3 (24% of its experts)	184.5	51

Code in the reply, as a share of the same pack's speed in the 24 GiB run; the points at 0 are that run itself. Dashed: each GiB left in the file takes 6% off the speed.

That is about 6% of the speed lost per GiB left in the file, if each GiB takes the same share of what remains (6.2%, 6.8% and 5.8% from the three points), for both packs. Three points from separate runs are a hint, not a law.

What it suggests

IQ3_S’s experts are 17% larger (46.8 ÷ 40.0), which would explain a slowdown of at most about 17%: that held for prose and reasoning with nothing left in the file, not for code. On code IQ3_S also went to RAM or to the file for experts nearly twice as often (the card held 94.3 to 94.8% of the experts it needed, against 96.9 to 97.2% for IQ3_XXS; on prose both held 97.9 to 99.0%, and prose lost the least), it kept slightly fewer drafts (68.8 to 71.3% against 72.1 to 74.0%), and the card drew less power, which is what waiting for data looks like. We did not profile the engine’s reads, which is what would settle it: this is a lead, not a proof.

What this does not say

Four draws per pack: the direction, not the size. Every IQ3_S draw was slower than every IQ3_XXS draw in code and reasoning, in all three runs (four against four is at best a 1-in-35 coincidence); prose at 24 GiB mixed, so 6.0% is not established.
One unexplained slow draw: IQ3_S, 24 GiB, Write call: 251 t/s against about 390 in the other three, while the card drew 135 to 330 W for about 30 seconds; it stays in the medians.
Runs not interleaved: the packs alternated within a run only, so comparing runs is a hint (another program held 12 GiB of RAM during the 11 GiB run only; IQ3_XXS sped up from 365.7 t/s at 11 GiB to about 470, with 10% of its experts in the file at 11 GiB).
“In the file” is not always “from the SSD”: Basalt reads through the operating system’s file cache; we did not record how much came from the SSD, nor empty the cache between draws.
One card, engine version, model and use: at 460 W the power limit held IQ3_XXS back for 55 to 62 of its 64 to 69 busy seconds per draw (24 and 17 GiB runs), so the gap may be larger at 575 or 600 W; long contexts, several conversations and the wait for the first token were not tested.
The fidelity figures are the author’s; whether IQ3_S writes better code is unchecked.

How we measured: setup and protocol

Setup

Card: one MSI RTX 5090 Gaming Trio OC (32 GB, air-cooled, PCIe 5.0 x16) limited to 460 W (575 W by default); the second card unused.
Machine: Ryzen 9 9900X, 60 GiB of RAM, Samsung 990 EVO Plus 4 TB SSD; Ubuntu 24.04, kernel 7.0.0-38, driver 610.43.02 on both test days (24 and 25 October 2026).
Engine: Basalt at commit 588a4776ed (23 October 2026), unchanged.
Model: Qwen3.8-Flash-Next in the two GSQ-RCO packs of the packs’ page, Quantisation Storing a model’s numbers with fewer bits, for example 4 instead of 16. The model takes less memory and runs faster, for some loss of precision that a test has to measure. by ISTA-DASLab: IQ3_XXS (74.4 GiB file) and IQ3_S (81.0 GiB), both with the drafter for MTP (multi-token prediction) A small extra part of the model drafts the next few tokens, and the model checks them in one step. When the drafts are right, the answer is written faster; drafting itself costs time and memory..
Settings: 262,144 tokens of context, KV cache The memory where a model keeps what it has already worked out for each token it has read (its keys and values), so that writing the next token does not mean reading everything again. It grows with the length of the conversation, and on a graphics card it competes with the model itself for memory. in 8-bit integers, image input on the processor, Basalt’s default drafting (each step checks up to 8 tokens: the last one kept and up to 7 drafted) and sampling (temperature 1.0, top-p 0.95, top-k 20). Engine options: resident experts on (the RAM copy), RAM copy capped at 11, 24 and 17 GiB in the first, second and third run (the other experts stay in the file), 700 MiB of VRAM kept in reserve, a conversation cache of 4,096 MiB in 4 slots with 6,144 MiB kept free.
Run: each draw in a fresh container of the engine image, the model file mounted read-only.

Where the engine put the experts the card did not hold, from its start-up lines, the same in every draw:

Pack	RAM copy limit (GiB)	Copied to RAM (GiB)	Left in the file (GiB)
IQ3_XXS	11	11.0	3.9 (10% of its experts)
IQ3_S	11	11.0	11.3 (24% of its experts)
IQ3_XXS	17	14.9	none
IQ3_S	17	17.0	5.3 (11% of its experts)
IQ3_XXS	24	14.9	none
IQ3_S	24	22.3	none

The engine prints what stays in the file only when something does, so IQ3_S’s 22.3 GiB at 24 GiB is the sum of the other runs’ lines. The card holds the same in every run (25.1 GiB of IQ3_XXS’s experts, 24.5 GiB of IQ3_S’s). IQ3_XXS needs only 14.9 GiB of RAM copy, so the 17 and 24 GiB limits do not change its placement.

Protocol

Each pack was started four times in each of three runs, in the order IQ3_XXS, IQ3_S, IQ3_S, IQ3_XXS, IQ3_S, IQ3_XXS, IQ3_XXS, IQ3_S. One start is a draw: a fresh container, one warm-up request we discard, then nine requests, one at a time. No request failed. Between runs the RAM copy limit changed (11, 24, then 17 GiB) and, for the last two, we also stopped another program that held about 12 GiB of RAM.

The requests look like a coding agent’s: a system prompt for an agent with four tools (Read, Write, Edit, Bash), one source file in the message, and one of three tasks. The files are rich’s markdown.py (Python), TanStack Query’s queryObserver.ts (TypeScript) and gin’s tree.go (Go), all under the MIT licence: prompts of 6,756 to 8,576 Token The unit a language model reads and writes: a word or a piece of a word. In English text, 1,000 tokens are roughly 750 words.. The tasks: explain the module in about 600 words of prose; write a test file of about 120 lines as code in the reply; write the same test file through one call of the Write tool. Reasoning is on with a budget of 1,024 tokens; a reply may run to 8,192 (they ran 1,100 to 4,200). Each prompt starts with a random number and the request’s index, so the engine served no prompt token from its cache, and every reply was of the kind asked.

What we measure:

Writing speed: for each part of a reply (reasoning, text, tool-call arguments), its tokens, counted with the model’s tokenizer, ÷ (arrival of its last streamed piece − arrival of its first), read by a script on the same machine; parts under 200 tokens are left out (one reasoning part per run); the wait for the first token is outside. Per draw, the median of each kind of part.
Experts served by the card: the share of the experts a request needed that the card already held, as the engine counts it; per run, pack and kind of output, the median of its twelve requests.
Drafts kept: drafted tokens the model kept ÷ drafted tokens, per request, from the engine.
Power: the card’s draw each second from nvidia-smi, with its clock and whether the power limit held it back; the median over the seconds the card was at least half busy. Per draw, the median clock was 2,782 to 2,872 MHz, the card never passed 76 °C and free RAM never fell below 13.2 GB.