Agentic Development

What are recursive language models?

Recursive language models (RLMs) treat input context as programmable data rather than static prompt text. By using an execution environment and recursive delegation, they give coding agents a way to work through large codebases and datasets without carrying everything in one prompt.

MMichael SwindellSeptember 6, 202616 min read

Recursive language models, or RLMs, are a way to let a language model work with large inputs as data inside a programmable environment instead of forcing it to read every token at once. An RLM can inspect, filter, compute over, and recursively delegate portions of context to other model calls. This makes it useful for long documents, codebases, logs, spreadsheets, and structured datasets where direct prompting becomes unreliable or expensive. For engineering teams, RLMs point toward a more controlled form of agentic work: define the objective and boundaries, then let models choose how to explore the underlying data. An agent-first control plane can provide the shared project context around that work.

At a Glance

RLMs change long-context work by making context programmable rather than prompt-bound.

Core idea: An RLM treats input context as an object available inside an execution environment, often a Python REPL.
Key advantage: The model can write code to inspect relevant portions of data instead of attending to every token.
Recursive work: A parent model can delegate bounded subtasks to submodels, including another call to itself.
Best fit: Long, dense, decomposable tasks involving documents, code, traces, logs, or data frames.
Operational lesson: Keep the outer contract deterministic while allowing the model flexibility inside the task.

What makes a recursive language model different?

A recursive language model differs from a standard prompted model because it can manipulate context as a programmatic object, execute code against it, and ask submodels to solve scoped parts of a larger task. The main model receives useful results rather than an ever-growing stream of raw context.

Traditional prompting puts source material directly into a context window. The model must locate relevant facts, retain them, reason over them, and produce an answer while processing a large sequence of tokens. This can work well for small or carefully curated inputs, but performance may deteriorate as irrelevant details accumulate.

An RLM puts the source material in an environment the model can interact with. For example, a large document collection, a code repository, or a data frame can be represented as a variable. The model can then write code to search, sample, filter, aggregate, or transform the input before deciding what needs deeper reasoning.

The difference is not simply that an RLM has tools. In ordinary tool calling, a model often sends a structured request, such as JSON, to an external service and receives a textual result. With an RLM approach, the model's logic, execution environment, intermediate values, and final answer are more tightly connected.

How do RLMs reduce long-context failure?

RLMs reduce long-context failure by preventing the parent model from carrying the full input through every reasoning step. Instead, the model selectively inspects data, performs computation outside its token stream, and returns only the evidence or summaries needed for the next decision.

Long context is not the same as useful context. A model may technically accept a large input while still struggling to identify a few critical items buried within it, a degradation often called context rot. A simple task, such as adding a dozen values scattered across tens of thousands of tokens, can become unreliable when the model must manually track each number in natural language.

In a programmable environment, the model can use code for the parts of a task that code handles well. It can search for matching patterns, extract values, validate them, calculate a total, and present the result. The language model still decides what to look for and how to interpret the findings, but it does not need to simulate every deterministic operation in prose.

This is especially valuable when inputs are heterogeneous. A directory of invoices, application logs, retention tables, or security-relevant source code may each require different inspection strategies. An RLM can choose an approach based on the objective and the available structure.

The broader point is practical: context management shifts from manually chunking everything in advance toward allowing the model to access and analyze context selectively.

How are RLMs different from RAG, agents, and tool calling?

RLMs overlap with retrieval, agents, and tool calling, but they place context, computation, and recursive delegation in one working environment. RAG retrieves text, while standard agents often exchange strings with tools. RLMs let models operate on data structures directly and return compact intermediate results.

Retrieval-augmented generation, or RAG, typically finds relevant passages and adds them to a prompt. It remains useful when the main challenge is locating a small number of documents or passages. However, RAG still depends on the model reasoning over retrieved text in its context window.

Tool-using agents extend a model with APIs, search systems, databases, and execution tools. They can complete multi-step work, but their intermediate state is often passed between components as strings or structured payloads. That can create overhead, context bloat, and a looser connection between a tool result and the computation that produced it.

RLMs are designed around an execution loop. The model can inspect a variable, write a function, test an assumption, call a submodel, and use the returned result in subsequent computation. This is closer to working in a notebook than composing a long chain of chat messages.

There is no need to treat these approaches as mutually exclusive. An RLM may use retrieval to locate candidate material, use tools to access systems, and use recursive subcalls for expensive or specialized reasoning. The architectural question is where the working context lives and how the system controls it.

Modern agent design increasingly favors this kind of bounded execution. Anthropic's guidance on building effective agents recommends starting with simple, composable patterns and adding complexity only when the task requires it. RLMs fit that principle when large context and decomposition are the actual bottlenecks.

When should you use a recursive language model?

Use an RLM when the input is too large or complex for dependable direct prompting and the task can be broken into inspectable subtasks. Avoid it for small, latency-sensitive requests, simple transformations, or tasks where the base model already performs well without added execution overhead.

RLMs are a strong candidate for workloads with three characteristics:

Large or dense source material: The model needs to reason over many documents, rows, files, traces, or records.
Useful structure: The data can be searched, grouped, filtered, parsed, calculated, or otherwise manipulated with code.
Decomposable questions: The overall objective can be divided into smaller questions with clear outputs.

Examples include consolidating information from lengthy invoices, analyzing customer cohort data across multiple data frames, extracting operational patterns from logs, reviewing large codebases, and inspecting agent traces for recurring failure modes.

RLMs are less attractive when an answer already fits comfortably in context, when every millisecond matters, or when recursive calls cost more than they save. They also do not remove the need for careful evaluation. A model may still choose an incomplete search strategy, write incorrect analysis code, or reach an unsupported conclusion from partial evidence.

A useful design rule is to begin with a direct implementation. Add RLM-style recursion when the task repeatedly fails due to input size, context quality, repetitive tool exchanges, or expensive manual context engineering.

How should you design an RLM workflow for production?

A production RLM workflow should use a stable outer contract, explicit inputs and outputs, bounded iteration, and observable intermediate work. Let the model determine its analysis strategy within those limits, but do not leave scope, permissions, validation, or completion criteria undefined.

The outer layer should be intentionally boring. Define the objective, the input types, the expected output schema, the allowed tools, and the maximum effort the system may spend. Then allow the model to choose whether it should filter data, write code, delegate analysis, or submit a final answer.

1. Define the task contract

Specify what goes in and what must come out. A cohort analysis might accept three data frames and return typed findings, risks, and recommendations. A code analysis workflow might accept a repository path and return a structured security report with evidence linked to specific files.

Typed outputs matter because they make downstream systems easier to validate and maintain. They also make submodel delegation more precise. Instead of asking a submodel for a vague summary, request a defined object such as a list of anomalies, extracted invoice fields, or candidate files requiring review.

2. Give the model a constrained working environment

The model should be able to inspect data and execute appropriate analysis, but it should not receive unlimited access by default. Separate read-only exploration from write-capable actions. Set resource limits, control which files or systems can be accessed, and keep the environment observable.

OpenAI's function calling documentation similarly emphasizes clear tool definitions and structured arguments. RLM workflows benefit from the same discipline, even when computation happens locally in a REPL rather than through a remote API.

3. Delegate only bounded subtasks

Recursive delegation works best when a parent model can frame a subproblem clearly. For example, the parent may identify a subset of log entries, pass them to a submodel for root-cause grouping, and receive a small structured result. The parent then compares those findings with other evidence.

Do not use submodels as an excuse to create unconstrained loops. Each subcall should have a defined purpose, expected output, cost limit, and stopping condition.

4. Treat model completion as a controlled decision

RLMs can allow a model to decide when it has finished exploring. That is useful, but production systems should still impose a maximum number of iterations, token budget, execution time, and recursion depth. A model should be able to stop early, but it should not be able to run indefinitely.

5. Preserve traces and verify final outputs

Store the task objective, code generated, data slices inspected, submodel calls, intermediate conclusions, and final submission. This helps developers debug incorrect results and helps engineering managers assess whether the agent used a sensible strategy.

One way to organize this work in Levr's agent-first project environment is to attach the task's acceptance criteria, tests, agent actions, and review history to the same project object. That makes the plan, execution evidence, and human approval easier to inspect together.

Give coding agents context that survives the prompt.

Levr connects issues, acceptance criteria, test results, and agent activity so teams can review the work behind an agent-produced result.

Explore agent-first project workflows

How can engineering teams apply RLM thinking to coding agents?

Engineering teams can apply RLM thinking by giving coding agents structured project context, bounded access to repositories and tools, explicit verification gates, and durable records of intermediate work. The goal is not maximum autonomy. It is dependable decomposition with evidence for each important decision.

A coding agent working only from a prompt has to infer requirements, discover repository conventions, choose files, run checks, and explain its changes within a transient interaction. This becomes fragile as tasks span multiple repositories, longer development cycles, or multiple agents, a common challenge in agentic software development.

An RLM-inspired workflow separates the durable project state from the agent's immediate prompt, which makes it important to decide where agent context lives. The issue defines intent. Acceptance criteria define the contract. Repository and test results provide evidence. The agent reads relevant context, performs bounded work, and updates the shared state with what it changed and why.

For example, a team investigating a large codebase could split work into independent areas: authentication, data access, dependency usage, and configuration. Each coding agent or submodel can inspect a scoped area, run approved checks, and return structured findings. A parent process can then prioritize results and prepare a report for human review.

Levr supports this pattern through a shared control plane where coding agents can work against the same issues, tests, workflow gates, and history as human contributors. Its agentic workflow model centers on defining intent, assigning work, recording verification, and approving completion rather than relying on a prompt alone.

Keep Intermediate Agent Work on the Record

Long-context work produces many intermediate steps. The Levr Dashboard keeps each issue, its acceptance criteria, linked tests, workflow gate status, and attributed agent activity in one place, so developers can review what an agent did and what was verified before work moves forward.

Agentic Software Development | Control Plane | Manual & Automatic Verification Issue Tracking |Levr

Every gate, test result, and agent action in one place.

What mistakes make RLM workflows unreliable?

RLM workflows become unreliable when teams mistake programmable context for guaranteed correctness. The main risks are vague objectives, unrestricted environments, unbounded recursion, weak output validation, and missing traces. RLMs improve how a model accesses information, but they do not replace system design or engineering judgment.

Using recursion for simple work: Extra model calls and execution steps add latency, cost, and failure modes. Use the simplest approach that meets the requirement.
Giving unrestricted data access: A model should not inspect every connected system merely because a task might benefit from more context.
Trusting generated code without checks: The model may write faulty parsing, filtering, or analysis logic. Test critical calculations and require evidence.
Allowing unlimited exploration: Maximum iterations, recursion depth, time limits, and budget limits are required controls.
Returning unstructured prose: Typed, verifiable outputs make it easier to detect missing fields and unsupported conclusions.
Skipping human review for consequential actions: High-impact changes, security findings, and production writes need explicit approval boundaries.

The practical opportunity is not to stop designing workflows. It is to move design effort upward. Instead of hand-authoring every retrieval rule, chunking policy, and intermediate prompt, define the outcome, boundaries, tools, and quality bar that guide the model's exploration.

Technical Deep Dive FAQ

What is a recursive language model?

A recursive language model, or RLM, is a language model workflow in which source context is held in a programmable environment and the model can inspect it through computation. Rather than placing all material directly into a prompt, an RLM can filter data, write code, create intermediate values, and delegate subproblems to other model calls. The recursive part comes from allowing those submodels to perform similar reasoning or analysis on smaller portions of the overall task.

What problem do recursive language models solve?

RLMs address tasks where direct prompting becomes unreliable because the input is long, dense, or structurally complex. A model may struggle to find facts buried in thousands of tokens, track many values accurately, or reason across a large repository. By making context accessible as data, an RLM can search and compute over the source material, then bring only relevant findings into the reasoning process. This reduces dependence on stuffing a growing amount of text into the context window.

Do RLMs replace retrieval-augmented generation?

No. RLMs and retrieval-augmented generation solve related but different problems. RAG retrieves potentially relevant passages and provides them to a model as prompt context. An RLM gives the model an environment where it can inspect and manipulate a larger body of data. A useful system may use RAG to locate candidate documents, then use an RLM-style process to compare records, calculate values, validate evidence, or recursively analyze large subsets of the retrieved material.

How do RLMs use a Python REPL?

A Python read-eval-print loop, or REPL, provides an interactive environment where an RLM can work with data as variables. The model may inspect columns in a data frame, run a regular expression over text, count occurrences, calculate a metric, or create a subset for deeper analysis. The important property is that the data remains available as an object in the environment. The model can use code against it instead of repeatedly receiving the entire dataset as text.

Can an RLM call itself recursively?

Yes. An RLM can delegate a task to another language model call, which may use the same model or a different one. The parent model can define the subtask, pass the relevant data or environment state, and request a structured response. This enables problem decomposition, such as assigning separate source-code areas for review or asking submodels to analyze individual document groups. Production systems should still enforce limits on recursion depth, runtime, token use, and total cost.

When are RLMs better than coding agents?

RLMs are not necessarily better than coding agents. They are a useful design pattern for coding-agent workflows that require extensive context inspection, deterministic computation, or recursive analysis. A basic coding agent may use tool calls and prompts effectively for a contained task. An RLM approach becomes more compelling when the agent must inspect a large codebase, analyze extensive traces, work with data frames, or repeatedly exchange oversized intermediate results that would otherwise bloat context.

How do you evaluate a recursive language model?

Evaluate an RLM at both the output and process levels. Measure whether the final answer is correct, complete, structured, and supported by evidence. Then inspect whether the model chose appropriate data slices, wrote valid code, used submodels efficiently, and stopped at a sensible point. Compare it with a direct-prompt baseline and a conventional agent workflow on representative tasks. Track accuracy, cost, latency, failed executions, recursion depth, and human correction rate.

What controls should production RLM systems have?

Production RLM systems need explicit permissions, sandboxed execution, validated tool interfaces, resource budgets, output schemas, and complete traces. Separate read-only analysis from actions that modify repositories, data, or infrastructure. Limit which datasets and commands the model can access. Require automated checks for structured outputs and human approval for consequential actions. Capturing intermediate code, submodel requests, tool results, and final decisions is essential for debugging and for maintaining accountability across human and agent work.

Can RLMs analyze documents, logs, and codebases?

Yes. RLMs are well suited to large documents, invoices, contracts, spreadsheets, log collections, agent traces, data frames, and source-code repositories. Their value comes from letting the model choose programmatic operations that fit the material, such as parsing, filtering, grouping, aggregation, or targeted search. They work best when the analysis objective is specific and the environment exposes useful structure. They should not be treated as a substitute for validation when findings affect security, finance, compliance, or production systems.

How does a control plane help teams run recursive workflows?

A control plane gives recursive and multi-agent workflows durable context outside an individual model session. It can connect the work request, acceptance criteria, execution history, test evidence, and review status. This reduces the risk that important decisions remain trapped in prompts or isolated logs. In an agent-first setup, teams can apply the same workflow gates to human and coding-agent contributions, keeping task state and proof of completion visible as agents decompose and execute work.

Key Takeaways

Recursive language models make long-context work more programmable, inspectable, and decomposable.

The important shift is from managing every token to governing the model's access to data and computation.

Context as data: RLMs work with inputs as variables and objects instead of only as prompt text.
Code where code fits: Models can use deterministic operations for extraction, filtering, and calculation.
Recursive delegation: Submodels can analyze bounded portions of a task and return focused results.
Strong boundaries: Typed outputs, limited execution, verification, and traces remain essential.
Engineering impact: Teams can spend less effort hand-managing context and more effort defining objectives and quality gates.

The best RLM workflow is not the most autonomous one. It is the one that produces a correct result with a clear, reviewable path to that result.