All posts

    Essay

    Turning down Qwen3.8's reasoning effort made it 42% slower on my laptop

    Lowering Qwen3.8-27B's reasoning effort kept accuracy at 13/13 but cost more time and tokens on my laptop. A small, measured case for running your own evals.

    2 min read
    The common advice for Qwen3.8 is to turn its reasoning effort down from the default, xhigh, to "medium". I measured that on my own laptop in August 2026. Accuracy held at 13 out of 13 at every setting, but medium was 42% slower than the default and used 47% more tokens. The setup was Qwen3.8-27B on a laptop with an 8 GB RTX 4060, run through llama-server against 13 graded questions. A script did the scoring, so nobody judged the answers by eye. Default (xhigh): 13/13 correct, 446 seconds, 1,182 tokens. Low: 13/13 correct, 553 seconds, 1,500 tokens. Medium: 13/13 correct, 632 seconds, 1,736 tokens. Accuracy didn't move, and both of the settings people recommend as cheaper cost more time and more tokens than the default did. My read is that with less room to think, the model pads its final answer instead, so the tokens you save on reasoning come back in the reply. I haven't verified that. It's the explanation that fits the numbers, and it's the next thing I'd test. Getting a clean measurement took a detour. LM Studio 0.4.12, the version I tested, silently ignored the reasoning setting: its outputs came back byte-identical across four different ways of sending it. A tool can accept a parameter and do nothing with it, and you only find out if you compare outputs. To test the knob at all, I had to drive llama-server directly. For anyone shipping AI products, reasoning tokens are now a real cost line, and the knobs that look like savings often aren't. A tip from a forum describes someone else's model, hardware and workload. Teams that measure on their own workload will spend less than teams that copy the tip, and a small graded suite like this one was enough to catch a setting that costs more than it saves. That's the case for evals in one result. It's one model, one machine and 13 questions, so treat it as a prompt to test on your own setup rather than a law. I'd like to know whether other reasoning models behave the same way.
    Local LLMsLLM evalsAI engineeringInference cost