Modern RPG Solutions LabAnthony Windey

An RTX 5090 at 460 W instead of 600 W: prompts take 9 to 16% longer to read, replies are written 3 to 8% slower

We had capped an RTX 5090 at 460 W to keep a home server cool and quiet, then measured what that costs a 27B model against the card's 600 W maximum. Reading a prompt took 9 to 16% longer. Writing was 3 to 4% slower without multi-token prediction, and about 8% slower with it for the same share of drafted tokens kept. In August, on another engine without drafting, the same cap had not slowed writing.

Question

On one RTX 5090 serving a 27B model, how much slower does the card serve when its Power limit The most electrical power a graphics card may draw, set through its driver. When the card reaches it, it lowers its clock speed to stay under. A lower limit means less heat and noise, and sometimes less speed. is 460 W instead of 600 W: reading the prompt, and writing the reply, with MTP (multi-token prediction) A small extra part of the model drafts the next few tokens, and the model checks them in one step. When the drafts are right, the answer is written faster; drafting itself costs time and memory. and without?

The usual rule says: writing, hardly at all; reading, somewhat. Writing a reply, one Token The unit a language model reads and writes: a word or a piece of a word. In English text, 1,000 tokens are roughly 750 words. after another, waits on how fast the card reads the model from its memory, not on how fast it computes; reading the prompt (the Prefill The first phase of a request: the model reads the whole prompt before it writes the first word of its answer. On a long prompt, this is the wait before anything appears.) is computation, and computation is what a lower limit slows. That held for us in August, on this same card, under vLLM and without multi-token prediction: one conversation wrote 81.8 tokens per second at 460 W and 81.9 at 575 W. Reading a 112,000-token prompt, on the other hand, went 11.7% faster at 575 W. If the rule holds here, writing barely changes between the two limits, and reading does.

Setup

  • Card: one MSI GeForce RTX 5090 32G Gaming Trio OC (32 GB, air-cooled, PCIe 5.0 x16). Its default power limit is 575 W, and 600 W is the highest it accepts. The machine’s second card took no part in the test.
  • Machine: AMD Ryzen 9 9900X, 60 GiB of RAM; Ubuntu 24.04, kernel 7.0.0-38-generic, NVIDIA driver 610.43.02 (open kernel module), as they were on the day of the test, 22 October 2026.
  • Engine: NInfer at commit 81c8ce093b (7 October 2026), built in a container on NVIDIA’s CUDA 13.1.2 image, with one patch of ours that changes the separators of its JSON output and nothing in the computation.
  • Model: Qwen3.8-27B in the NVFP4 Quantisation Storing a model’s numbers with fewer bits, for example 4 instead of 16. The model takes less memory and runs faster, for some loss of precision that a test has to measure. we published on Hugging Face, converted to NInfer’s format by its converter, with the output head and the embedding (the model’s last and first layers, which turn numbers back into tokens and tokens into numbers) in FP8, an 8-bit number format: 16.2 GiB of weights.
  • Settings: the ones we use every day. Multi-token prediction with 3 drafted tokens, up to 8 conversations at once, 262,144 tokens of context, the KV cache The memory where a model keeps what it has already worked out for each token it has read (its keys and values), so that writing the next token does not mean reading everything again. It grows with the length of the conversation, and on a graphics card it competes with the model itself for memory. in NVFP4 A 4-bit number format from NVIDIA. Values are stored on 4 bits in blocks of 16, and each block carries its own 8-bit scale: about 4.5 bits per value in all., image input on. The runs without multi-token prediction drop its three options and change nothing else. At start-up the engine reported room for 603,328 tokens in its cache with multi-token prediction and 720,768 without, the same in every draw.
# The power limit, set before each draw
sudo nvidia-smi -i 0 -pl 460        # or: -pl 600

# The engine, in a fresh container for each draw
ninfer-serve /models/qwen3_8_27b_mrs_nvfp4.ninfer \
  --max-context 262144 --kv-capacity auto --max-concurrency 8 --kv-dtype nvfp4 \
  --spec mtp --draft-tokens 3 --lm-head-draft \
  --preserve-thinking --vision --device-state-slots 0 \
  --chat-template /templates/qwen3.8.chat_template.jinja --pending-timeout-ms 300000
# Without multi-token prediction: the same line, without "--spec mtp --draft-tokens 3 --lm-head-draft"

Protocol

  • Two things change, in every combination: the power limit (460 W or 600 W) and multi-token prediction (on or off). Engine, model, settings and requests are otherwise the same.
  • Twelve draws, three per combination, fixed before the run, in three blocks where each combination comes once and the order turns from one block to the next: off 460, off 600, on 600, on 460; on 460, on 600, off 600, off 460; off 600, off 460, on 460, on 600 (on and off refer to multi-token prediction). On 22 October 2026 from 19:28 to 19:41 UTC; all twelve completed, with no failed request.
  • Each draw starts the engine in a fresh container, so nothing cached survives from one draw to the next, and sends one short warm-up request that is not counted. Then four measures, in this order: one conversation with a short prompt (1,434 to 1,506 tokens), eight at once with short prompts, one with a long prompt (33,431 to 34,516 tokens), eight at once with long prompts.
  • Every request asks for a technical essay, at temperature 0.5, and is stopped after 512 written tokens. Its prompt starts with a random number and is padded to length with a numbered paragraph repeated, so no two prompts share more than their first few tokens; the engine’s log shows that none of the 228 requests (nineteen per draw, warm-up included) found any part of its prompt already in the engine’s cache. In every request, the 512 tokens were the model’s reasoning, before any answer.
  • What is measured:
    • time per Decoding step One pass of the model while it writes. Without multi-token prediction, a step writes one token. With it, a step checks the drafted tokens and keeps the right ones, so it writes one to four tokens.: from the engine’s own log, for each request with one conversation. Each step writes one token plus the drafted tokens it keeps, so the 512 tokens took 512 − (drafted tokens kept) steps, and the time per step is 512 ÷ the decoding speed ÷ that number of steps. Without multi-token prediction a step writes one token, so there it is simply the time per token;
    • writing speed: tokens written ÷ (duration of the request − wait for the first token), read by a script on the same machine as the replies stream in;
    • wait for the first token: from sending a request to its first token, almost all of it spent reading the prompt; with eight conversations, the median of the eight;
    • total output: with eight conversations, all tokens written ÷ the time from the first request sent to the last token received, reading of the prompts included;
    • power: the card’s draw every 500 ms, read with nvidia-smi together with its clock, its temperature and whether the power limit was holding it back; averaged over the samples above 100 W of a draw.

Results

Reading the prompt took longer at 460 W in every combination: with one conversation and a long prompt, the wait for the first token grew by 8.8% with multi-token prediction and by 13.2% without; with eight conversations, by 9.8 to 16.3%. Writing lost less. Without multi-token prediction, it was 3.1% slower with a short prompt and 3.8% slower with a long one. With it, each decoding step took 8.7 to 8.9% longer. In each of these measures, the three draws at 460 W were all slower than the three at 600 W. The card drew 114 to 128 W less.

With multi-token prediction

Measure600 W3 runs460 W3 runs
Writing, short prompt: time per decoding step (ms)13.0914.23
Writing, 34K prompt: time per decoding step (ms)13.8815.12
One conversation, 34K prompt: wait for the first token (s)3.413.71
Eight conversations, 34K prompts: wait for the first token (s)27.6230.32
Eight conversations, 34K prompts: total output (tokens/s)125.5113.6
Power drawn (W)583455
Median of three draws per setting, one RTX 5090, measured on 22 October 2026. Time per step from the engine's log; power: the mean of the samples above 100 W of each draw.

Without multi-token prediction

Measure600 W3 runs460 W3 runs
Writing, short prompt: time per decoding step (ms)10.9511.30
Writing, 34K prompt: time per decoding step (ms)11.4311.88
One conversation, 34K prompt: wait for the first token (s)3.263.69
Eight conversations, 34K prompts: wait for the first token (s)26.9129.88
Eight conversations, 34K prompts: total output (tokens/s)115.2105.8
Power drawn (W)571457
Median of three draws per setting, same card, same night. A decoding step writes one token here.

With multi-token prediction, the writing speed a reader sees depends on the step and on how many drafted tokens the model keeps, which depends on the text it is writing. From one reply to the next the share kept went from 36.6% to 51.3%, and the speed with it: with a short prompt, 160.0 to 186.5 tokens per second at 600 W and 163.1 to 176.7 at 460 W. That spread is larger than the effect of the limit, so three replies per setting do not show it in tokens per second (medians 167.9 and 168.4); the time per step leaves the drafts out. At the same share kept, a step 8.7 to 8.9% longer means a reply written about 8% slower.

The earlier run, on 13 October, measured multi-token prediction only, with the same protocol. Its decoding steps took 13.04 to 13.11 ms at 600 W and 13.85 to 14.28 ms at 460 W: a step 8.9% longer at 460 W, as on 22 October.

Every draw, in the order it ran

One conversation, short prompt. “Drafts kept”: the share of drafted tokens the model kept.

Draw Setting Writing (tokens/s) Time per step Drafts kept
1 off, 460 W 90.0 11.14 ms —
2 off, 600 W 91.6 10.94 ms —
3 on, 600 W 160.0 13.09 ms 36.6%
4 on, 460 W 168.4 14.23 ms 46.6%
5 on, 460 W 163.1 14.23 ms 44.2%
6 on, 600 W 186.5 13.09 ms 48.4%
7 off, 600 W 91.5 10.95 ms —
8 off, 460 W 88.6 11.31 ms —
9 off, 600 W 91.4 10.95 ms —
10 off, 460 W 88.7 11.30 ms —
11 on, 460 W 176.7 14.30 ms 51.3%
12 on, 600 W 167.9 13.11 ms 40.1%

One conversation, 34K prompt.

Draw Setting Writing (tokens/s) Time per step Drafts kept First token
1 off, 460 W 86.0 11.66 ms — 3.62 s
2 off, 600 W 87.7 11.42 ms — 3.37 s
3 on, 600 W 172.0 13.87 ms 46.3% 3.41 s
4 on, 460 W 158.6 15.11 ms 46.8% 3.71 s
5 on, 460 W 156.3 15.12 ms 45.7% 3.71 s
6 on, 600 W 179.4 13.88 ms 50.2% 3.42 s
7 off, 600 W 87.6 11.43 ms — 3.25 s
8 off, 460 W 84.3 11.88 ms — 3.75 s
9 off, 600 W 87.5 11.44 ms — 3.26 s
10 off, 460 W 84.3 11.88 ms — 3.69 s
11 on, 460 W 166.1 15.21 ms 51.3% 3.73 s
12 on, 600 W 164.7 13.89 ms 43.1% 3.37 s

Eight conversations at once; total output in tokens per second, first token as the median of the eight.

Draw Setting Short: total Short: first token 34K: total 34K: first token
1 off, 460 W 529.4 0.92 s 113.0 27.56 s
2 off, 600 W 536.6 0.82 s 114.7 27.04 s
3 on, 600 W 802.9 0.86 s 127.8 27.06 s
4 on, 460 W 785.9 1.00 s 113.6 30.32 s
5 on, 460 W 809.7 1.00 s 113.1 30.64 s
6 on, 600 W 825.2 0.86 s 125.5 27.62 s
7 off, 600 W 535.1 0.83 s 115.2 26.91 s
8 off, 460 W 521.4 0.96 s 105.8 29.88 s
9 off, 600 W 530.0 0.83 s 117.4 26.26 s
10 off, 460 W 521.2 0.96 s 104.0 30.50 s
11 on, 460 W 762.4 1.01 s 114.3 30.30 s
12 on, 600 W 829.5 0.87 s 124.8 27.69 s

The card during each draw

Over the samples above 100 W of each draw.

Draw Setting Power, mean Clock, mean Hottest Held back by the limit
1 off, 460 W 456 W 2,574 MHz 67 °C 118 of 119
2 off, 600 W 562 W 2,763 MHz 81 °C 88 of 117
3 on, 600 W 577 W 2,698 MHz 84 °C 89 of 94
4 on, 460 W 455 W 2,387 MHz 78 °C 103 of 103
5 on, 460 W 456 W 2,384 MHz 78 °C 103 of 103
6 on, 600 W 584 W 2,683 MHz 86 °C 87 of 93
7 off, 600 W 571 W 2,734 MHz 87 °C 96 of 116
8 off, 460 W 457 W 2,483 MHz 80 °C 125 of 125
9 off, 600 W 572 W 2,727 MHz 87 °C 95 of 115
10 off, 460 W 457 W 2,477 MHz 80 °C 126 of 126
11 on, 460 W 453 W 2,377 MHz 80 °C 101 of 102
12 on, 600 W 583 W 2,678 MHz 87 °C 93 of 94

At 600 W the power limit still held the card back in 75 to 99% of the samples: it would have drawn more if allowed, but 600 W is the highest limit this card accepts. Its clock ran 9 to 11% lower at 460 W.

What this does not say

  • The size of the effect is rough. Three draws per combination is few. In each measure above, the three draws at 460 W were all slower than the three at 600 W, but with three against three a rank test (it looks only at the order of the six draws) cannot put the chance of such an order below one in ten, two-sided. Read the percentages as the size of the effect on this machine, not as precise figures.
  • The time per step was chosen after the run. Before it, we had fixed the writing speed in tokens per second as the main measure; with multi-token prediction, for the reason given above, it could not settle anything. The time per step separates the two limits on both nights without a single overlap, but it was picked after seeing the data.
  • Why a step with drafts loses more. Not measured. Checking several tokens in one step is more computation than writing one, and computation is what a lower limit slows; we did not profile the engine to confirm it.
  • One card, one model, one engine, one kind of text. An air-cooled card, in a room whose temperature was not recorded; one 27B model; a padded, synthetic prompt, and output that was only the model’s reasoning. How many drafts the model keeps depends on the text, so with multi-token prediction the writing speed a reader sees depends on it too.
  • The card’s other limits were not recorded. The sampler logged whether the power limit was holding the card back, not whether heat was. The card reached 87 °C at 600 W.
  • The earlier run had a flaw. On 13 October the request numbers came from the clock, and in each draw one or two of them repeated an earlier request of the same draw, so part of a prompt was already cached. That shortens a wait for the first token, not a decoding step; the 22 October run is free of it.
  • Energy per token. The power is averaged over a whole draw, its four measures mixed; it does not give the energy each measure took.
  • Not a long-term test. It says nothing about running a card at 600 W for months, nor about its power connector.

History

  1. Measured with multi-token prediction only: six draws, 460 W and 600 W alternated.
  2. Measured again, with and without multi-token prediction: twelve draws. Written up for this site.