Teaching Robots to Learn Like LLMs: An In-Depth Look at RICL

Imagine you are using a Large Language Model (LLM) like GPT-4, and you want it to write a poem in a very specific, made-up dialect. You wouldn’t need to retrain the entire neural network to do this. Instead, you would simply provide a few examples of this dialect in the prompt—the “context”—and the model would adapt instantly. This capability is known as In-Context Learning (ICL).

Now, imagine you have a general-purpose robot. You want it to perform a new task, like picking up a specific tool it has never seen before and placing it in a tray. With current state-of-the-art robotic models, you cannot simply “show” the robot a few examples and expect it to work. Typically, you would need to collect a dataset and fine-tune the model’s parameters using gradient descent—a slow and computationally expensive process.

Why does this gap exist? Why can language models adapt on the fly, while robotic “Vision-Language-Action” (VLA) models remain rigid?

This is the core problem addressed by the paper “RICL: Adding In-Context Adaptability to Pre-Trained Vision-Language-Action Models.” The researchers propose a novel method to inject in-context learning capabilities into pre-trained robot policies post hoc. The result is a system that can learn new manipulation tasks from just 10-20 demonstrations without a single gradient update.

In this post, we will break down exactly how RICL works, the architecture behind it, and why this might be the “missing link” for truly generalist robots.

The Context: VLAs and The Missing Capability

To understand RICL (pronounced “rickle”), we first need to understand the baseline it improves upon: the Vision-Language-Action (VLA) model.

What is a VLA?

A VLA is the robotic equivalent of a multimodal LLM. It takes images (vision) and instructions (language) as input and outputs robot commands (action). The specific base model used in this paper is \(\pi_0\)-FAST. This model is trained via Imitation Learning—it watches millions of frames of robots performing tasks and tries to mimic the actions associated with those visual states.

The Problem: Imitation vs. Next-Token Prediction

LLMs develop In-Context Learning (ICL) properties naturally because they are trained on massive amounts of web text to predict the “next token.” The structure of web documents often includes examples followed by answers, which implicitly teaches the model to look at the context window to solve the current problem.

VLAs, however, are trained to map current pixels to current actions. They typically don’t see a “history” of other tasks in their context window during training. As a result, standard VLAs do not emerge with the ability to look at a set of demonstrations and figure out a new task. They are rigid. If they haven’t seen an object during training, they often fail to interact with it meaningfully.

The Solution: RAG for Robots

The authors propose solving this by borrowing another concept from NLP: Retrieval-Augmented Generation (RAG). In RAG, a system retrieves relevant information from a database and feeds it into the model’s context to help it answer a query.

RICL applies this to robotics. Instead of retrieving text, the robot retrieves visual demonstrations (images and actions) of the task it is trying to perform. The goal is to create a “RICL-VLA” that can look at these retrieved examples and immediately understand how to manipulate a new object.

As shown in Figure 7 above, the pipeline involves taking a standard VLA (\(\pi_0\)-FAST), subjecting it to a “Priming” phase (RICL training), and creating a model that can adapt to new tasks simply by retrieving relevant demonstrations.

The Core Method: Retraining for In-Context Learning (RICL)

The heart of the paper is the methodology used to convert a rigid VLA into an adaptive one. This is not training from scratch; it is a post-training recipe.

1. The Architecture

The RICL architecture builds upon the \(\pi_0\)-FAST model. The system is designed to process a sequence that includes:

1. The Query: The current live observation (images from top, side, and wrist cameras, plus proprioceptive state) and the language instruction.
2. The Context (Retrieved Data): A set of similar situations from a demonstration buffer.

Figure 2 illustrates this flow. On the left, we see the “Retrieval Buffer,” which contains recorded demonstrations. When the robot is acting (the “Query” at time \(t\)), the system uses the top-down camera image to search this buffer for similar images.

The retrieval is performed using DINOv2, a powerful visual encoder. The system embeds the current view and finds the “Nearest Neighbors” in the embedding space. These neighbors—specifically their images, states, and the actions taken in those moments—are fed into the VLA’s context window alongside the current query.

2. The “Priming” Phase

You might ask: “If the base VLA doesn’t know how to use context, simply feeding it retrieved images won’t help, right?”

Exactly. This is why the authors introduce a Priming phase. They take the pre-trained \(\pi_0\)-FAST model and fine-tune it specifically to pay attention to the context.

• Frozen Vision: The image encoder (SigLIP) is frozen.
• Active LLM: The language model component is fine-tuned.
• Data Structure: The training data is organized into sequences of {Retrieved Neighbors -> Query}.

The model is trained on a set of generic pick-and-place tasks. By forcing the model to predict the action for the “Query” while having access to the “Retrieved Neighbors,” the model learns a meta-skill: how to copy and interpolate from the examples provided in its context.

3. The Action Interpolation Layer

This is a critical technical innovation in the paper. The researchers found that simply letting the VLA predict actions wasn’t enough. They added an explicit mechanism to blend the VLA’s “brain” with the retrieved “memory.”

The final action output is a combination of:

1. The action predicted by the neural network (\(\pi_\theta\)).
2. The action actually taken in the retrieved nearest neighbor (\(a'\)).

They combine these using a distance-weighted interpolation:

Let’s break down this equation:

• \(\pi_{\text{RICL-VLA}}\) is the final output.
• \(a'\) is the action from the closest retrieved example.
• \(\sigma(\pi_\theta(...))\) is the model’s own prediction.
• \(d\) is the distance (similarity) between the current image and the retrieved image.
• \(\lambda\) is a temperature parameter.

What does this mean? If the robot’s current view is identical to a retrieved memory (\(d \approx 0\)), the term \(e^{-\lambda d}\) approaches 1. The system heavily relies on \(a'\)—it essentially copies the action from the demonstration. If the current view is very different (\(d\) is large), \(e^{-\lambda d}\) approaches 0. The system relies more on the VLA’s generalization capabilities.

This essentially gives the robot a “safety rail.” If it sees a situation it recognizes from the demonstrations, it mimics the successful action directly. If it’s in a slightly new situation, it interpolates.

Experimental Setup

To prove this works, the authors tested RICL on a physical robot setup: the Franka DROID.

Figure 3 shows the setup, including the multiple camera angles (Top, Wrist, Right) necessary for the VLA to function.

The evaluation protocol was rigorous. They defined a set of “Evaluation Tasks” that involved:

• Unseen Objects: Objects like a Pokeball, a bagel, or a squeegee that were not in the VLA’s original training data.
• Novel Motions: Actions that require unique movements, like dragging a squeegee or opening a specific shelf door.
• New Scenes: Moving the robot to a kitchen sink environment with different lighting and background.

For each new task, they provided the robot with just 20 demonstrations. These demonstrations were put into the retrieval buffer. The robot then had to perform the task.

Results: The Power of In-Context Learning

The results show a stark difference between the standard VLA and the RICL-adapted VLA.

Quantitative Success

The quantitative gap is massive. The authors compared the standard \(\pi_0\)-FAST-DROID against their RICL version.

Looking at Figure 4, the dark blue bars represent full task success.

• Base VLA (\(\pi_0\)): Struggles significantly. On many tasks (like “move idli plate”), it has a 0% success rate. It essentially wanders aimlessly because it doesn’t ground the language instruction to the unseen object.
• RICL-VLA: Achieves significant success rates immediately, just by using RAG and ICL.

The chart also shows an ablation (bottom right) for the “idli plate” task. As you increase the number of demonstrations in the buffer from 5 to 20, the success rate climbs. This confirms the model is actively using the data provided to it.

Qualitative Analysis: Seeing is Believing

The visual comparisons provided in the paper are perhaps the most compelling evidence.

Case 1: The Pokeball (Unseen Object)

In this task, the robot must “pick up the poke ball.” The base model has never seen a poke ball.

In Figure 1(a) (above), the base model (Left, Red Border) gets confused. It ignores the pokeball and picks up the yellow duck (a “distractor” object it likely saw during training). This is a classic “Language Grounding” failure. The RICL model (Right, Green Border) successfully identifies the unseen pokeball because it has 20 examples of the pokeball being picked up in its context buffer. It retrieves those examples, sees what the “pokeball” looks like, and executes the grasp.

Case 2: The Squeegee (Novel Motion)

Here, the robot must “move the squeegee to the right and drag it.” This requires a specific combination of lifting and pulling.

In the comparison above, the base model (Left) again fails, picking up the duck. The RICL model (Right) not only identifies the squeegee but mimics the dragging motion found in the retrieved demonstrations.

Case 3: The Idli Plate (Unfamiliar Grasp)

The “idli plate” has a unique shape with depressions. A standard grasp won’t work; the fingers need to slide into the depressions.

The RICL model successfully executes this nuanced grasp. Interestingly, the authors note that in some cases, the RICL model even elicited “latent actions”—performing successful movements that weren’t strictly in the retrieval data, suggesting a synergy between the base model’s knowledge and the retrieved context.

Robustness and Reactivity

One might worry that “retrieving” actions makes the robot a mindless replay machine. What if the object moves?

Figure 5 demonstrates that the system remains reactive. In this sequence, a human moves the red ball while the robot is reaching for it. Because the system performs retrieval and inference at every timestep (running at roughly 10Hz), it constantly updates its plan. If the ball moves, the query image changes, the retrieved neighbors might shift slightly, and the VLA’s visual encoder tracks the new position. The robot successfully adjusts and grabs the ball.

Implications: Fine-Tune Like You Pre-Train

The paper presents one final, powerful finding.

While the “training-free” ICL (just using the 20 demos in the buffer) works well, the best performance comes from taking those 20 demonstrations and performing a quick fine-tuning step on the RICL model.

The authors call this “Fine-tuning like you pre-train.” Because the model was “primed” to use context, fine-tuning it on the specific target task (while still using the retrieval mechanism) boosts performance significantly—doubling the success rate in aggregate compared to just ICL, and far outperforming a standard fine-tune of the base VLA.

This suggests that RICL isn’t just a way to avoid training; it’s a superior initialization for training on small datasets.

Conclusion

The RICL paper represents a significant step toward “ChatGPT-like” moments in robotics. It moves us away from the paradigm where a robot is a static system that must be retrained by engineers for every new object it encounters.

By teaching the VLA how to use context (the Priming phase) and giving it a mechanism to access memory (RAG + Action Interpolation), the researchers created a system that can adapt to the user.

Key Takeaways:

1. VLAs are currently rigid: They struggle with new tasks without parameter updates.
2. RICL injects adaptability: By training the model to look at retrieved neighbors, it unlocks In-Context Learning.
3. Hybrid Architecture: Mixing the LLM’s prediction with a retrieved action (via the interpolation layer) provides both generalization and precision.
4. No-Code Improvement: An end-user can improve the robot simply by providing 10-20 video demonstrations, without needing to touch the code or training loop.

As we look to the future, techniques like RICL suggest that the next generation of robots won’t just be pre-programmed tools, but adaptive learners that can pick up new skills simply by watching us do them a few times.