Local LLM Lab

I run open models on my own hardware and measure them instead of trusting leaderboards, with an eval harness that grades the code models write by running it.

llama.cppLM StudioOllamaGGUF quantizationSpeculative decodingPython

P-017 / Register plateRESEARCH

35 tok/s

26B model on a MacBook Pro M5

llama.cppLM StudioEvals

More detail

On a MacBook Pro M5 (24 GB), Gemma 4 26B-A4B runs at about 35 tokens a second for everyday chat and Qwen3.6-35B-A3B at 3-bit handles coding. On an RTX 4060 laptop (8 GB) I serve models through llama.cpp and LM Studio, run Claude Code and OpenCode against them, and benchmark every setting I change. Measured findings include a GPU-offload cliff that silently cuts prompt processing 7 to 10 times, and speculative decoding that sped up generation 29%.