Recursive Language Models expose the untapped capability gap in today’s coding agents

The tension between human expertise and brute-force AI search

The episode opens by framing a clash that has shaped AI development for years: human engineers craft clever prompts and toolchains, while raw model capacity sits idle behind them. Zhang points out that “we’re wrapping increasingly powerful models in primitive systems” and asks how much capability is being left on the table. The discussion is anchored in a concrete example – the generation of GPU kernels – where an AI system can produce code that rivals hand-tuned human kernels when allowed to iterate aggressively.

Our read: The framing suggests that the real bottleneck is not model size or training data, but the surrounding orchestration layer. If the harness can’t expose the model’s full search space, engineers will continue to see diminishing returns from scaling alone.

Recursive language models as a new architectural primitive

Zhang introduces recursive language models (RLMs) as a way to let a model call itself as a sub-routine, effectively creating a programmable loop inside the same neural engine. Unlike traditional agents that issue a single prompt to a static model, an RLM can generate a prompt, feed it back to itself, evaluate the output, and repeat until a convergence criterion is met. The podcast highlights three concrete capabilities demonstrated in the session:

1. GPU Mode – a mode where the model writes and tests CUDA kernels on the fly, using a feedback loop to refine performance.
2. KernelBench – an internal benchmark suite that measures the speed and correctness of AI-written kernels across multiple hardware targets.
3. AI-written kernels –

These examples illustrate that recursion turns a single inference pass into a multi-step optimizer, blurring the line between model inference and algorithmic search.

Our read: By embedding a search process within the model’s own inference graph, RLMs sidestep the latency and context-window limits that plague external toolchains. The approach is reminiscent of program synthesis but keeps the entire loop inside the LLM, potentially reducing integration friction and enabling tighter performance guarantees.

The “harness problem” in today’s coding assistants

A recurring theme is what Zhang calls the harness problem – the mismatch between a model’s raw capability and the static, prompt-driven interfaces that expose it. He walks through a comparison of three popular coding agents: Claude Code, Codex, and Pi. Despite differences in branding and training data, the podcast argues they are “more similar than the…”. All three rely on a single-shot prompt that asks the model to generate code, then hand off the result to an external executor. The lack of an internal recursion or self-evaluation step means they cannot systematically explore alternative implementations the way an RLM can.

Our read: The similarity among these agents is less a product of convergent engineering and more a symptom of a shared architectural constraint. Without a built-in recursion or feedback loop, they cannot exploit the combinatorial explosion of possible code variants, limiting their ability to solve hard optimization problems or to adapt to novel hardware.

Rethinking language models with GEV and multi-agent swarms

Later in the conversation, Zhang touches on GEV (Generalized Evaluation Vector) as a framework for measuring model outputs across multiple dimensions—correctness, efficiency, and resource usage—within the same recursive loop. He also speculates about scaling RLMs into massive multi-agent swarms, where dozens of instances coordinate via shared evaluation vectors to explore a solution space in parallel. While still speculative, the podcast frames these ideas as natural extensions of the recursive paradigm, moving from a single self-calling model to a distributed ensemble that retains the same feedback mechanisms.

Our read: If GEV can be operationalized, it would provide a quantitative backbone for the recursive process, allowing automated pruning of unpromising branches. Coupled with swarm-style parallelism, this could dramatically accelerate tasks that currently require manual iteration, such as kernel tuning or complex algorithm design.

Closing on the practical implications

The final segment circles back to the earlier claim about agent similarity, reinforcing that “Claude Code, Codex, and Pi are so similar” because they all inherit the same primitive harness. Zhang’s argument is that without adopting recursive architectures, the industry will continue to see incremental gains at best, while the true upside—leveraging the full combinatorial power of modern LLMs—remains locked behind a narrow interface. The podcast leaves listeners with a clear technical challenge: redesign the harness to enable recursion, feedback, and multi-dimensional evaluation, or risk ceding the next wave of AI capability to research labs that adopt these patterns.