From Evaluation to RSI: Why Infra Agents May Be the Most Practical Starting Point

15 Sep, 2026

RAG evaluates answers. Agents evaluate tasks and trajectories. Reinforcement learning evaluates whether a policy update actually generalizes. Recursive self-improvement goes one step further: it asks whether the improvement process itself becomes better over time.

!YJ summary: evaluation expands from answer quality to the process that creates the next system.

YJ summary: evaluation expands from answer quality to the process that creates the next system.

1. The evaluation target keeps moving outward

In a conventional RAG pipeline, evaluation is still relatively static. We can separate retrieval from generation, check whether relevant context was found, and judge whether the answer is grounded in that context.

Once the system becomes an Agent, that decomposition is no longer enough. The system plans, calls tools, changes external state, observes results, and re-plans. The final response is merely one artifact in a much longer trajectory.

Key point. The central shift is not “more metrics.” It is a change in what counts as evidence of improvement.

2. Harnesses make self-improvement observable

A practical self-improvement loop needs more than a strong base model. It needs an explicit place where the system can observe failures, propose changes, validate those changes, and preserve working behavior.

This is why harness engineering matters. A harness exposes the model to tools, files, state, evaluators, permissions, and persistent artifacts. Once those components are explicit, they also become measurable and, within bounded surfaces, optimizable.

!*Self-Harness uses weakness mining, bounded harness proposal, and validation to update the harness. (Image source: Zhang et al. 2026)*

Self-Harness uses weakness mining, bounded harness proposal, and validation to update the harness. (Image source: Zhang et al. 2026)

The figure makes an important boundary explicit: the system may edit the harness, but the validation machinery should remain outside the editable surface. Otherwise “self-improvement” quickly turns into “editing the exam.”

3. Why Infra is unusually RSI-friendly

Open-ended knowledge work has a weak-ground-truth problem. It is difficult to decide whether a strategy memo or market analysis is “good enough” without substantial subjective judgment.

Infrastructure work is different. Many tasks already expose machine-checkable feedback:

• Does the kernel compile?
• Is numerical correctness preserved?
• How close is performance to a hardware bound?
• Did TTFT, TPOT, or throughput improve?
• Did the patch preserve tests and invariants?
• Does a training-algorithm change improve a hidden evaluator?

Infra gives us five unusually useful properties: execution is measurable, resettable, observable, reproducible, and falsifiable.

!YJ summary: execution leaves evidence.

YJ summary: execution leaves evidence.

Takeaway. Infra is attractive for RSI because feedback is executable and falsifiable. A change either compiles, preserves correctness, improves the metric, and survives replay — or it does not.

4. L0–L1: From kernel generation to production feedback

At the lowest level, kernel benchmarks ask whether an Agent can produce code that is both correct and fast. Recent work pushes this beyond synthetic operator suites by grounding evaluation in real serving workloads and hardware limits.

L0 — Kernel / Operator

SOL-ExecBench (NVIDIA Research 2026) and CANN Bench (Gao et al. 2026) move the target from “faster than a software baseline” toward “how close are we to the hardware limit?”

That distinction matters because software baselines move; hardware limits are more stable. The evaluator also needs to separate compile success, functional correctness, and performance, otherwise reward hacking becomes trivial.

L1 — Production Workload

Atrex-Bench (Yang et al. 2026), FlashInfer-Bench (FlashInfer Community 2026), and RealisticTritonBench (Huang et al. 2026) introduce production traces, realistic shapes, deployment constraints, and end-to-end reintegration.

FlashInfer-Bench is particularly interesting because it turns the benchmark artifact itself into a reusable interface between the Agent and the production system.

!*FlashInfer Trace packages Definition, Workload, Solution, and Evaluation into one reusable artifact. (Image source: FlashInfer Community 2026)*

FlashInfer Trace packages Definition, Workload, Solution, and Evaluation into one reusable artifact. (Image source: FlashInfer Community 2026)

5. L2: Repository optimization exposes the understanding–execution gap

Repository-level optimization is harder because the Agent must navigate a real codebase, identify the right subsystem, implement a patch, run the benchmark, and preserve correctness.

ISO-Bench (Nangia et al. 2026) reconstructs such tasks from accepted performance PRs in vLLM and SGLang. Its evaluation combines hard execution metrics with softer semantic metrics.

!*ISO-Bench evaluates repository-level inference optimization with execution metrics and semantic bottleneck targeting. (Image source: Nangia et al. 2026)*

ISO-Bench evaluates repository-level inference optimization with execution metrics and semantic bottleneck targeting. (Image source: Nangia et al. 2026)

!YJ summary: localizing a bottleneck and shipping a valid optimization are different capabilities.

YJ summary: localizing a bottleneck and shipping a valid optimization are different capabilities.

Contrast. Understanding ≠ execution. Correctly naming the bottleneck is much easier than producing a patch that is correct, buildable, measurable, and actually faster.

6. L3–L4: Long-horizon systems require disciplined experimentation

Φ-Bench (Φ-Bench 2026) expands the scope from local kernel work to multi-file implementation and whole-repository optimization.

The capability shift is subtle but important. The Agent must now manage an experiment process over time — not as a checklist, but as a sequence of evidence-changing experiments.

!YJ summary: each experiment should narrow the search space.

YJ summary: each experiment should narrow the search space.

!YJ summary: evaluation, product iteration, and learning share the same evidence.

YJ summary: evaluation, product iteration, and learning share the same evidence.

The system now contains three coupled loops:

• Evaluation Loop: online cases update the evaluator and its rubric.
• Product Loop: bad cases drive harness and system fixes.
• Learning Loop: trajectories become training or reusable skills, then face generalization evaluation.

When these loops share the same traces, cases, rubrics, and environments, evaluation is no longer merely a release gate.

It becomes part of the learning system’s control plane.

10. What an RSI-ready Infra dataset should preserve

A useful Agent-for-Infra dataset should store more than problem + answer.

The last layer matters most for RSI.The valuable artifact is no longer only:

“What did the final kernel look like?”

It is also:

How did the Agent use evidence to shrink the search space?

11. Evaluators must evolve too

Once an Agent repeatedly sees a benchmark and optimizes against it, a fixed test set has a short half-life.

A more robust evaluator uses three distinct surfaces rather than one fixed test set.

!YJ summary: public development, hidden generalization, and rolling production serve different purposes.

YJ summary: public development, hidden generalization, and rolling production serve different purposes.

A fixed test set has a short half-life.

The public set supports debugging. The hidden set checks generalization to new shapes, seeds, workloads, and system states. The rolling production set is refreshed from new PRs, current traffic, new models, and new hardware.

In RSI, the evaluator itself becomes part of the security boundary.

12. Final takeaway

The progression can be compressed into one idea: RAG evaluates answers; Agents evaluate tasks and trajectories; RL evaluates policy improvement and generalization; RSI evaluates the improvement process itself.

Conclusion. RSI may not begin when an Agent “thinks again.” It may begin when the next experiment becomes more informative than the previous one.

References

[1] Zhang, et al. “Self-Harness: Harnesses That Improve Themselves.” arXiv preprint arXiv:2606.09498, 2026.

[2] NVIDIA Research. “SOL-ExecBench: Speed-of-Light Benchmarking for Real-World GPU Kernels Against Hardware Limits.” arXiv preprint arXiv:2603.19173, 2026.

[3] Gao, et al. “CANN Bench: Benchmarking Agent Generated Kernels against Real NPU and Algorithmic Limits.” arXiv preprint arXiv:2607.20518, 2026.

[4] Yang, et al. “Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent.” arXiv preprint arXiv:2607.14541, 2026.

[5] FlashInfer Community. “FlashInfer-Bench: Building the Virtuous Cycle for AI-driven LLM Systems.” arXiv preprint arXiv:2601.00227, 2026.

[6] Huang, et al. “RealisticTritonBench: A Benchmark for Triton-Kernel Generation in Real-World AI Frameworks.” arXiv preprint arXiv:2608.12004, 2026.

[7] Nangia, et al. “ISO-Bench: Can Coding Agents Optimize Real-World Inference Workloads?” arXiv preprint arXiv:2602.19594, 2026.

[8] Φ-Bench contributors. “Φ-Bench: Can Large Language Models Engineer the Infrastructure That Powers Them?” Project page, 2026.

[9] Yeon, et al. “InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents.” arXiv preprint arXiv:2607.20468, 2026.

[10] Zhong, et al. “SkillLearnBench: Benchmarking Continual Learning Methods for Agent Skill Generation on Real-World Tasks.” arXiv preprint arXiv:2604.20087, 2026.

[11] Chi, et al. “AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement.” arXiv preprint arXiv:2608.20318, 2026.

[12] Weng, Lilian. “Harness Engineering for Self-Improvement.” Lil’Log, July 2026.