When Does a Benchmark Become a Substrate for Self-Improvement?

15 Aug, 2026

Benchmarks used to be scoreboards. The interesting ones are starting to become something else: environments where failure is observable, evidence is diagnostic, changes are editable, and improvement can be replayed. That shift may matter more for RSI than any single leaderboard.

The master question: what has to change before evaluation stops being a mirror of capability and starts becoming part of the mechanism that improves it?

TL;DR

A benchmark becomes useful for self-improvement when it does more than produce a score: it exposes a verifiable outcome, preserves a diagnostic trajectory, points to an editable substrate, and supports protected replay. The deeper shift is that evaluation is moving from the end of the pipeline into the learning loop itself.

The old benchmark mental model no longer fits

The traditional benchmark has a clean job: hold the test set fixed, run a frozen system, and report a score. That separation is valuable precisely because the evaluator sits outside training.

But Agent systems violate the quiet assumption underneath that setup. They do not just answer. They call tools, mutate state, branch, recover, spend resources, and sometimes succeed for the wrong reason. A single scalar at the end hides most of what matters.

The anomaly is that the more capable Agents become, the less satisfactory static answer evaluation feels. The object being measured is moving outward—from text, to behavior, to environment state, and eventually to the process that changes the next trajectory.

Benchmarks are becoming active environments

Three changes are colliding. Models are better at tool use. Agent harnesses can preserve traces and intermediate state. And more domains now expose automatic feedback that is cheap enough to run repeatedly.

Shunyu Yao’s The Second Half argues that once the general learning recipe becomes strong enough, progress increasingly depends on defining better problems rather than only finding better methods. Agent evaluation is a concrete version of that claim: the benchmark starts deciding not just who wins, but what kind of behavior counts as useful intelligence.

Karpathy’s idea of verifiability gives a second piece. Tasks become much easier to optimize when success can be checked cheaply and reliably. For RSI, I would add three more requirements: the environment should be diagnostic enough to explain failure, editable enough to turn diagnosis into a change, and replayable enough to test whether the change survives.

A benchmark becomes interesting for RSI when it can turn a failed trajectory into a better next experiment.

Evaluation used to come after capability

In the old view, training produces a model and evaluation measures it. The evaluator deliberately stays passive.

That works for short, mostly deterministic tasks. It works much less well once the system interacts with an environment. A support Agent can produce the perfect sentence and still issue the wrong refund. A browser Agent can explain the right plan and click the wrong account. The final answer is no longer the ground truth.

τ-bench (Yao et al. 2024) is a useful transition point because it evaluates tool-agent-user interaction against final database state and domain policy rather than trusting the final prose. It also introduces pass^k to measure consistency across repeated trials.

Once actions change the outside world, evaluation has to move from answer checking to outcome verification.

Figure 1 — YJ’s note: the final response is only one piece of evidence; the environment state is the witness.

Understanding is not execution

Outcome-based evaluation solves one problem but exposes another. A benchmark can tell us that the task failed without telling us whether the Agent misunderstood the problem, changed the wrong component, introduced a bug, or simply ran a weak experiment.

ISO-Bench (Nangia et al. 2026) makes this gap unusually visible. It reconstructs real performance-engineering tasks from merged vLLM and SGLang pull requests and separates execution outcomes from semantic bottleneck targeting.

Figure 2 — The important distinction is between understanding the intended bottleneck and actually producing a valid optimization. (Image source: Nangia et al. 2026)

The deeper lesson is that several capabilities we casually bundle together under “Agent performance” are different: diagnosis, implementation, validation, and experiment design.

Understanding ≠ execution. And for long-horizon systems, execution ≠ experimentation either.

The benchmark starts producing learning signals

The bottleneck moves from generation to experimentation

Once the model knows many plausible optimization techniques, generating another candidate becomes cheap. The scarce capability shifts toward deciding what to try, how to isolate a variable, when to stop, and which state to preserve.

InferenceBench (Yeon et al. 2026) is almost designed to expose this. Each run gives an Agent a model, one H100, a serving bottleneck, and two hours to submit a valid OpenAI-compatible server. The best current Agent reaches a 9.83× aggregate speedup, while a matched-budget hyperparameter search reaches 11.53×. The benchmark also reports a gap between the best valid configuration discovered during a trajectory and the final one the Agent actually submits.

Figure 3 — InferenceBench fixes the model, hardware, scenario, and budget; the agent iterates on an inference engine, then quality and integrity gates determine whether normalized speedup counts.

The figure matters because the benchmark is a bounded experiment environment: real hardware, fixed time, correctness gates, and only the final submitted server counts. (Image source: Yeon et al. 2026)

Failure becomes useful only when it leaves editable evidence，A benchmark still sits downstream until failure can modify the next system .

Figure 4 — Several experiments produce a useful insight only when their results are compared; the proposed metric is information gain per experiment.

Self-Harness (Zhang et al. 2026) makes that transition explicit. The model weights and evaluator remain fixed, but the harness is editable. The system mines recurring failure patterns from execution traces, proposes minimal harness changes, and keeps them only when regression tests support the edit. Across the three reported base models, held-out pass rates rise from 40.5%→61.9%, 23.8%→38.1%, and 42.9%→57.1%.

Figure 5 — What matters here is the boundary: execution failures can edit the harness, but the evaluator stays outside the editable surface. (Image source: Zhang et al. 2026; reproduced in Weng 2026)

**Failure matters only when it leaves editable evidence. Otherwise even a sophisticated benchmark is still just a more expensive scoreboard. **

Figure 6 — YJ’s note: failure becomes useful when it can be turned into a bounded revision and tested again.

The improvement becomes more interesting when it survives replay

A one-off win is not yet recursive self-improvement. A runtime flag can improve one workload. A lucky checkpoint can improve one seed. A prompt can overfit a visible test set.

The stronger criterion is inheritance: the improvement should survive a fresh run and alter the mechanism that produces future capability.

AI4AI-Bench (Chi et al. 2026) pushes directly on this boundary. The Agent edits the training algorithm in frozen research repositories; the submitted source code is then rerun from scratch in a sealed environment and scored by a hidden evaluator. Across 29 configurations of six systems, the mean normalized score is 0.166 and the best reaches 0.250. The minority of submissions that actually change how the model learns average 0.226, versus 0.126 for the rest.

Figure 7 — AI4AI-Bench covers ten research optimization tasks, each paired with a task-specific held-out evaluation measure.

RSI starts to compound when the improvement is not a better output, but a better mechanism for producing the next system.

Evaluation itself becomes part of system design

The more surfaces an Agent can improve, the more pressure eventually reaches the evaluator.

That is why recent benchmarks increasingly contain hidden tests, integrity audits, sealed replay, and correctness gates. These are not administrative details. They define the trusted boundary of the learning loop.

Once evaluation enters the optimization loop, a weak evaluator does more than mis-rank systems. It teaches the system the wrong lesson.

Figure 8 — YJ’s note: some part of evaluation must remain protected, or optimization pressure eventually reaches the measurement itself.

An RSI-ready benchmark needs four properties

The recent work looks diverse—customer support, inference optimization, harness design, continual skills, training algorithms—but I think it compresses into a small model.

Score → state → evidence → modification → replay → inherited improvement.

This is the point where a benchmark stops being only a mirror of capability. It becomes infrastructure for improvement.

In AI Infra, this ladder is already visible

AI Infra is a useful place to see this transition because every layer exposes a different kind of feedback. At the bottom, the evaluator can ask whether a kernel compiles, is numerically correct, and approaches a hardware limit. Higher up, the evaluator has to reason about real workload distributions, repository-scale changes, serving behavior, live system state, and eventually whether an improvement survives into the next learning procedure.

The progression is therefore not simply “harder coding.” The object being optimized keeps expanding: Kernel → Production Workload → Repository → Long-Horizon System → Serving → Live Ops → Continual Learning → Algorithm Improvement.

The kernel layer already shows why this matters. Atrex-Bench draws 30 operators and 440 shapes directly from production inference traces and weights them by real serving importance; even the best vanilla coding Agent reaches only about 10% of the hardware roofline on these production operators. CANN Bench brings the same idea into the Ascend ecosystem with 53 operators and 1,060 test cases, separating compilation, correctness, and performance while anchoring the performance score to a Hardware-Anchored Performance limit rather than a drifting software baseline.

But the more interesting move is what happens after the kernel. FlashInfer-Bench treats Definition, Workload, Solution, and Evaluation as one portable trace artifact, so a benchmark result can flow back into the serving stack rather than die on a leaderboard. RealisticTritonBench similarly derives tasks from real framework PRs and reintegrates generated kernels into the original frameworks for end-to-end validation. In other words, evaluation starts tracking not only whether code is locally fast, but whether the optimization survives contact with the system that actually needs it.

Figure 9 — FlashInfer-Bench makes the benchmark artifact itself portable across the production lifecycle: workload definition, implementation, and evaluation live in the same trace schema. (Image source: FlashInfer Community, 2026)

The same expansion continues upward. Φ-Bench stretches evaluation from single-file kernel work to 85 tasks across nine infrastructure topics, including long-horizon implementation and end-to-end optimization. InfraBench removes the frozen-repository assumption entirely and asks whether an Agent can recover a live system without violating invariants or leaving behind a degraded state. SkillLearnBench then asks whether experience can be compressed into reusable skills, while AI4AI-Bench finally moves the editable object from the infrastructure stack into the training algorithm itself.

In AI Infra, evaluation is progressively moving from local correctness to inherited improvement.

That is why Infra is useful for thinking about RSI. It gives us a sequence of increasingly demanding evaluators, but it also gives us something more valuable: an observable path from a local failure to a replayable change. The benchmark stops being a finish line and starts looking like part of the machinery that produces the next system.

What changed in my mind

I used to put benchmarks after the model in my mental diagram. Train first, evaluate later. That ordering now feels incomplete.

Infra is the cleanest example because its feedback is unusually measurable and hard to fake, but it is not the final scope. The same pattern is appearing in tool use, browser agents, harness engineering, skill learning, and algorithm design.

Open questions

1. How should we measure experiment quality rather than only final success?

2. How much evaluator drift can a self-improving system tolerate?

3. Which surfaces should remain permanently non-editable?

4. When does stored experience become transferable learning rather than memory?

5. Can the evaluator itself improve without becoming easier to game?

Figure 10 — The proposed self-improvement loop favors experiments that yield clearer signals and an evaluator strong enough to distinguish real progress.

RSI may not begin when an Agent can rewrite itself.

** It may begin when the next experiment becomes more informative than the previous one—and the evaluator is strong enough to know the difference.**

References

1. Yao, Shunyu. “The Second Half.” 2025.

2. Karpathy, Andrej. “Verifiability.” 2025.

3. Yao, Shunyu, et al. “τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains.” arXiv, 2024.

4. Nangia, Ayush, et al. “ISO-Bench: Can Coding Agents Optimize Real-World Inference Workloads?” arXiv, 2026.

5. Yeon, Jehyeok, et al. “InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents.” arXiv, 2026.

6. Zhang, Hangfan, et al. “Self-Harness: Harnesses That Improve Themselves.” arXiv, 2026.

7. Zhong, Shanshan, et al. “SkillLearnBench: Benchmarking Continual Learning Methods for Agent Skill Generation on Real-World Tasks.” arXiv, 2026.

8. Chi, Yizhe, et al. “AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement.” arXiv, 2026.

9. Weng, Lilian. “Harness Engineering for Self-Improvement.” Lil’Log, 2026.

10. Lin, Edward, et al. “SOL-ExecBench: Speed-of-Light Benchmarking for Real-World GPU Kernels Against Hardware Limits.” arXiv, 2026.

11. Gao, Xue-Jian, et al. “CANN Bench: Benchmarking Agent Generated Kernels against Real NPU and Algorithmic Limits.” arXiv, 2026.

12. Yang, Lingyun, et al. “Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent.” arXiv, 2026.

13. FlashInfer Community. “FlashInfer-Bench: Building the Virtuous Cycle for AI-driven LLM Systems.” arXiv, 2026.

14. Huang, Jinjun, et al. “RealisticTritonBench: A Benchmark for Triton-Kernel Generation in Real-World AI Frameworks.” arXiv, 2026.

15. Φ-Bench Contributors. “Φ-Bench: Can Large Language Models Engineer the Infrastructure That Powers Them?” 2026.

16. InfraBench. “InfraBench: Evaluating Infrastructure Agents Across Layers, Lifecycle, and Risk.” HotInfra, 2026.