GLM-5.1 Doesn't Quit: Zhipu AI's Model That Outlasts the Competition

There is a quiet lie in the AI coding benchmark game. Models get tested on their first pass. Their opening salvo. The sprint from zero to "good enough." What nobody measures — at least not until now — is what happens after.

GLM-5.1, Zhipu AI's latest flagship model released this week, was benchmarked exactly there. The long game. And the results are a genuine rethinking of what agentic AI is supposed to look like.

The Benchmark Nobody Talks About

SWE-Bench Pro. Terminal-Bench 2.0. KernelBench. These are the established tests for AI coding ability, and GLM-5.1 posts strong numbers across all of them: 58.4 on SWE-Bench Pro (state-of-the-art), 75.1 on Terminal-Bench 2.0 when run through Codex, and competitive results against Claude Opus 4.6, GPT-5.4, and Gemini 3.1 Pro across a wide suite of agentic tasks.

Those numbers are impressive. But the Zhipu team is betting that the more important differentiator isn't peak performance — it's what the model does after the first 100 rounds, after the obvious optimizations are exhausted, after the easy wins are gone.

"Previous models — including GLM-5 — tend to exhaust their repertoire early: they apply familiar techniques for quick initial gains, then plateau. Giving them more time doesn't help."

The claim is specific: GLM-5.1 doesn't plateau the way its predecessors do. It stays effective on agentic tasks over much longer horizons — sustaining optimization over hundreds of rounds and thousands of tool calls.

600 Iterations, 6,000 Tool Calls, and a 6× Jump

The clearest proof is a VectorDBBench experiment that should make any systems engineer pay attention.

VectorDBBench tests whether an AI can build a high-performance approximate nearest neighbor search database in Rust, given only a skeleton with HTTP API stubs. The standard evaluation gives the model a 50-turn tool-call budget. The best result under that setting was 3,547 QPS, achieved by Claude Opus 4.6.

Zhipu reframed the test. They wrapped GLM-5.1 in an outer optimization loop using the Claude Code framework: in each iteration, the model can use as many tool calls as it wants to edit code, compile, test, and profile, then submit a new version to be benchmarked. The model decides autonomously when to stop and what to try next.

Over 600 iterations and 6,000+ tool calls, GLM-5.1 reached 21,500 QPS — roughly 6× the best single-session result.

The optimization trajectory followed a staircase pattern. The model made incremental tuning gains within a fixed strategy, then hit a structural transition — a fundamental rethinking of the approach — that shifted the performance frontier upward. Around iteration 90, it shifted from full-sequence scanning to IVF cluster probing with f16 vector compression, jumping from baseline to 6,400 QPS. Around iteration 240, it introduced a two-stage pipeline — u8 prescoring followed by f16 reranking — reaching 13,400 QPS. Six such structural transitions occurred across the full run, each initiated by the model after analyzing its own benchmark logs and identifying the current bottleneck.

GLM-5, for comparison, improved quickly at first, then leveled off. GLM-5.1 kept going.

KernelBench: The Proof Under the Hood

KernelBench measures whether a model can take a reference PyTorch implementation and produce a faster GPU kernel with identical outputs. This isn't a toy benchmark — it covers 50 problems across fused operator sequences and full-model end-to-end optimization of real architectures like MobileNet, VGG, and MiniGPT. Torch.compile with default settings achieves 1.15× speedup on these problems; with max-autotune, 1.49×.

GLM-5.1 reached 3.6× speedup. More importantly, it was still making progress at the end of the run.

The trajectories in Zhipu's chart tell the story visually. GLM-5 starts strong and flattens early. Claude Opus 4.5 extends a bit further. GLM-5.1 pushes the frontier further still and sustains useful optimization for substantially longer. Claude Opus 4.6 remains the strongest finisher at 4.2× with clear headroom remaining — the gap between GLM-5.1 and the frontier is real, but so is the fact that GLM-5.1 reached it faster and held it longer than GLM-5 did.

8 Hours to Build a Linux Desktop

The most striking demonstration is qualitative rather than quantitative. For the Linux desktop experiment, Zhipu gave GLM-5.1 an ambitious prompt with no starter code, no design mockups, no intermediate guidance: build a Linux-style desktop environment as a web application.

The framing matters: most models — including earlier versions of GLM — give up quickly on open-ended tasks like this. They produce a basic skeleton with a static taskbar and one or two placeholder windows, declare the task complete, and stop. There is no mechanism to step back and ask what else is missing.

Zhipu wrapped GLM-5.1 in a simple harness: after each round of execution, the model reviews its own output, identifies what can be improved, and continues. This loop ran for 8 hours.

Early on, GLM-5.1 delivered something similar to a short session's output — a basic layout with a taskbar and simple window. But it didn't stop there. The system steadily filled out: file browser, terminal emulator, text editor, system monitor, calculator, games. Each new addition was integrated into a coherent UI rather than bolted on as an afterthought. Styling became more polished. Interactions became smoother. Edge cases were handled. By the end, the result was a complete, visually consistent desktop environment running in the browser.

Why This Matters Beyond the Benchmarks

The long-horizon capability isn't a gimmick for demos. It changes what AI coding agents can actually accomplish in practice.

Most real engineering problems — performance optimization, bug hunts across complex codebases, system redesigns — don't resolve in one shot. They require iteration. A model that gives up after the first 50 tool calls is useless for a week-long refactoring project. A model that can sustain useful optimization over hundreds of rounds and thousands of tool calls opens up an entirely different class of tasks.

The Zhipu team is honest about what remains unsolved. Escaping local optima earlier when incremental tuning stops paying off. Maintaining coherence over execution traces that span thousands of tool calls. Developing reliable self-evaluation for tasks where there is no numeric metric to optimize against — like the Linux desktop experiment, where "good" depends on subjective judgment the model has to generate itself.

These are real open problems. But the direction is clear: the next frontier in AI coding isn't a bigger context window or a higher one-shot benchmark score. It's the ability to stay effective over longer time horizons — to be a collaborator that doesn't run out of steam.

GLM-5.1: Open Source and Available Now

GLM-5.1 is released under the MIT License, making it one of the most capable open-source agentic coding models available. Model weights are on HuggingFace and ModelScope. Local inference is supported via vLLM and SGLang. The model is also available via Zhipu's developer APIs at api.z.ai and BigModel.cn, and is compatible with Claude Code, OpenClaw, and other major coding agent frameworks.

For Claude Code users subscribed to the GLM Coding Plan, GLM-5.1 can be enabled by updating the model name to "GLM-5.1" in settings. Usage is billed at 3× quota during peak hours (14:00–18:00 UTC+8) and 2× during off-peak — with an end-of-April promotion pricing it at 1× during off-peak hours.

The Linux desktop, the 21,500 QPS vector search, the GPU kernel optimizations — none of those were in the original spec for "an AI coding model." GLM-5.1 is evidence that the model that wins in 2026 won't be the one that scores highest on a single-pass benchmark. It'll be the one that keeps going.