Gao Dalie (Ilyass)

Posted on Jan 13

RLM: The Ultimate Evolution of AI? Recursive Language Models

#machinelearning #ai #programming #datascience

During the weekend, I scrolled through Twitter to see what was happening in the AI community. MIT has just released a groundbreaking paper that addresses a significant issue with large language models.

It sounds very academic, but here’s the simple version: essentially, if you have AI act a second time, the results can be remarkable.

Over the past two years, almost all mainstream large-scale models have been racing to expand their context windows. Gemini has increased its window size to the millions, the GPT series continues to increase its investment, and Llama has even proclaimed a goal of tens of millions of tokens.

On the surface, this is an arms race of “who can fill the most space.” But the problem is that increasing the context window does not mean that the model can actually “read in and remember” all the content.

Another popular approach is Retrieval-Augmented Generation (RAG), which first segments long documents into chunks and stores them in a vector database, then retrieves relevant segments based on the question and feeds them to the model.

This avoids having the model consume the entire long document at once, but its effectiveness is highly dependent on the quality of the retrieval, and it often struggles with questions that require comprehensive information from the entire text.

However, these methods all share a common problem: they assume that the model is passive. The model can only wait for humans to organize, segment, and feed it information. True intelligence shouldn’t be like this.

MIT have proposed a disruptive idea: why not let the model read itself? Look it up itself? Slice it itself? Call itself?

Thus, Recursive Language Models (RLM) were born.

RLM’s core insight is very simple, yet revolutionary: it transforms the context from “input” to “environment”.

The model no longer receives a long string of tokens, but instead, like a program, treats the entire context as a variable within a REPL (Read-Eval-Print Loop) environment, allowing it to view, slice, search, filter, and recursively call itself at any time. It is no longer “fed information,” but rather “actively explores information.”

It’s like going from “Here’s a book for you to read” to “Here’s a library for you to search, dissect, summarise, and use your own assistants.”

This not only bypasses the context constraints of Transformer, but also gives the model the ability to “procedurally access the world” for the first time.

So, let me give you a quick demo of a live chatbot to show you what I mean.

Check a video

We're going to ask a question: “ Print me the first 100 powers of two, each on a newline”

If you see how the chatbot generates output, you’ll see that the agent processes the full input, which can be millions of tokens, loaded into a Python REPL environment as a variable; the agent does not read this text directly. Instead, it treats the input as an environment it can operate on.

First, the model performs exploration and inspection. It prints small slices of the context, checks structure, looks for headers, patterns, or repeated phrases, and uses tools like string slicing and regular expressions to understand how the data is organised. This step replaces passive reading with active scanning.

Next, the model applies programmatic filtering and indexing. Using Python methods such as split(), find(), re.findall(), loops, and conditionals, it narrows the massive input down to only the parts that matter for the task. Noise is discarded early, which prevents context overload.

Once relevant sections are identified, the model performs task decomposition. It breaks the main problem into smaller, well-defined subtasks. Each subtask fits comfortably within a normal model context window. Humans do not predefine this decomposition — the model decides how to split the problem based on what it discovers during exploration.

Then comes the key step: recursive self-calls. For each subtask, the model calls itself (or a smaller helper model) to process that chunk. These calls form a tree of reasoning, not a single chain. Each call returns a partial result, which is stored in variables inside the REPL environment.

After sub-results are collected, the model performs aggregation and synthesis. It uses Python logic to combine summaries, compare results, compute pairwise relationships, or assemble structured outputs like lists, tables, or long documents.

The model then applies verification and self-checking. It may re-run parts of the analysis, cross-check results with another recursive call, or validate logic using code. This creates multi-pass reasoning similar to human double-checking.

Finally, the model constructs the final output. Instead of being limited by token output size, it builds the answer piece by piece in variables and then returns the assembled result. This allows extremely long, structured outputs that traditional LLMs cannot produce.

What makes RLM special?

Press enter or click to view the image in full size

Recursive Language Models (RLMs) are special because they change an AI from a passive reader into an active problem-solver. Instead of trying to understand a huge input all at once, an RLM treats the input like a workspace it can explore, search, and break apart using code.

It decides what to read, how to slice the information, and when to call itself again to solve smaller pieces. By using programmatic access, recursion, and self-checking, it avoids getting confused by long or complex inputs and stays stable even as tasks grow harder.

This lets RLM handle massive contexts, high-complexity reasoning, and long structured outputs in a way traditional language models simply can’t.

How exactly does RLM work?

Press enter or click to view image in full size

Traditional LLMs work simply: you feed in a long string of tokens, and it gives you an answer in a single forward inference.

But when the context length exceeds hundreds of thousands or millions, this approach is like asking someone to read “War and Peace” in one go before answering a question — it’s bound to break down.

RLM’s approach is completely different.

It loads the entire long context into a Python REPL environment as a variable, such as context. The model no longer directly “eats” these tokens; instead, it accesses them by writing code, much like a programmer.

This means that for the first time, the model has a “tool.” It can:

To view a specific segment: print(context[:500])

Search keyword: re.findall(“festival”, context)

Split by chapter: part1, part2 = context.split(“Chapter 2”)

Constructing a subtask: sub_answer = llm_query(f”Please summarize {part1}”)

It can even recursively call itself: result = rlm_query(sub_prompt)

This is like giving the model “hands” and “eyes”. It is no longer a passive language generator, but an intelligent agent that can actively explore, actively deconstruct, and actively plan.

The examples in the study are very vivid. The model will first print the first 100 lines to check the structure before deciding how to slice them; it will use keywords to filter out potentially related paragraphs; it will break down the task into multiple sub-problems and then recursively call itself to solve them.

This isn’t prompt engineering; it’s program engineering.

What’s the limitation of RLM?

The main limitation of RLM is that its power comes with overhead and complexity. When the input is short and the task is simple, using the base model directly is often faster and more efficient, since RLM adds extra steps like environment interaction and recursive calls.

In its current form, RLM relies on synchronous, blocking sub-model calls, which increases end-to-end latency and can slow down responses. The paper also notes that system prompts are fixed and not tailored to different task types, leaving performance gains on the table.

Finally, letting the model write and execute code inside a REPL introduces real engineering challenges, especially around security isolation, safety, and predictable behavior.

In short, RLM is powerful for hard, large-scale problems, but it is heavier, slower, and more complex than standard models for simple tasks.

My impression :

RLM represents a shift from “how do we compress context?” to “how do we teach models to actively manage context like a skilled developer?”

Instead of fighting context limits with bigger windows or lossy summaries, RLMs embrace the constraint and learn to work within it — delegating, filtering, and focusing programmatically. It’s scaffolding that scales with learning, not just engineering.

I would highly appreciate it if you

❣ Join my Patreon: https://www.patreon.com/GaoDalie_AI