The dream of general-purpose robotics often conjures images of humanoid machines fluidly pouring water, handing over tools, or tidying up a cluttered room. While we have made massive strides in robotic control, achieving this level of “dexterity”—precise, coordinated movement, often using two hands—remains a formidable challenge.

Robots need to process complex inputs (vision, language, proprioception) and output high-dimensional actions instantly. Recent years have seen the rise of Diffusion Policies, which treat robot action generation like image generation: starting with noise and refining it into a trajectory. While effective, diffusion models can be slow, requiring many steps of iterative refinement (denoising) to produce a usable action.

In this post, we will explore ManiFlow, a new approach presented by researchers from the University of Washington, UC San Diego, and Nvidia. ManiFlow combines the benefits of Flow Matching with Consistency Training to create a policy that is not only more accurate than standard diffusion models but also significantly faster—generating high-quality actions in as few as 1 or 2 steps.

Figure 1: We introduce ManiFlow, a flow matching model excelling in complex manipulation tasks, including bimanual dexterous manipulation.

As shown above, ManiFlow is capable of controlling diverse robot morphologies, from single arms to dual-arm setups and even full humanoids, performing tasks like pouring liquids and handing over objects.

The Problem with Current Policies

To understand why ManiFlow is necessary, we first need to look at the limitations of current state-of-the-art methods like Diffusion Policies.

Diffusion models generate actions by iteratively removing noise. This process, while powerful, models a complex, curved trajectory from a noise distribution to the data distribution. Because the path is curved, the solver needs to take many small steps (inference steps) to navigate it accurately. In a real-time robotic setting, taking 10, 20, or 50 inference steps introduces latency. A robot pausing to “think” for too long can result in jerky motions or failed catches.

Flow Matching is an alternative generative framework that aims to learn a “velocity field” that pushes noise toward data. Ideally, the most efficient path from noise to data is a straight line. If we can force the model to learn these straight paths, we can jump from noise to action in a single step. However, existing flow matching policies often struggle to capture the full complexity of dexterous, multi-fingered interactions or generalize well to new environments.

The ManiFlow Solution

ManiFlow improves upon previous methods through two main pillars:

  1. Consistency Flow Training: A new training objective that enforces “straightness” in the generation trajectory.
  2. DiT-X Architecture: A specialized transformer architecture designed to handle multimodal inputs (vision, language, robot state) more effectively.

Let’s break these down.

1. Consistency Flow Training

The core innovation of ManiFlow is how it trains the model to predict actions. The researchers combine standard Flow Matching with Consistency Training.

Standard Flow Matching

In standard flow matching, the model \(\theta\) tries to predict a velocity \(v_t\) that moves a noisy sample \(x_t\) toward the clean data \(x_1\). The loss function looks like this:

Flow Matching Loss Equation

This ensures the model learns the direction of the data. However, it doesn’t guarantee that the path taken is the most efficient one.

Adding Consistency

Consistency training takes this a step further. It posits that points along the same trajectory should all point toward the same final destination. If you are halfway through the generation process, your predicted destination should be consistent with your prediction at the beginning.

ManiFlow incorporates a “step size” \(\Delta t\) into the model. During training, the system samples a current time \(t\) and a future time \(t + \Delta t\). It then enforces that the velocity predicted at the current step is consistent with the velocity required to reach the target estimated at the future step. This effectively “straightens” the flow path.

Figure 3: ManiFlow Consistency Training. Given a flow path that smoothly transforms action to noise, we sample multiple intermediate points via linear interpolation.

As illustrated in Figure 3 (above), the training process involves sampling intermediate points (\(x_t\), \(x_{t1}\)) and ensuring self-consistency along the trajectory. This effectively flattens the curve required to transform noise into action.

The consistency loss function is defined as:

Consistency Training Loss Equation

By optimizing these two objectives jointly, ManiFlow learns trajectories that are incredibly straight, allowing the robot to generate high-quality actions in extremely few steps (often just one step) during inference.

The Importance of Time Sampling

Another subtle but critical contribution of this paper is the analysis of Time Sampling Strategies. When training a generative model, you have to sample a timestep \(t\) between 0 (noise) and 1 (data).

Most models sample \(t\) uniformly. However, the researchers found that for robotic control, not all timesteps are created equal. The “high-noise” regime (when \(t\) is small) is where the model learns the coarse, global structure of the movement.

Figure 14: Comparison of timestep sampling strategies for flow matching models.

As shown in the distributions above, the researchers experimented with various strategies. They found that a Beta distribution (which concentrates samples near \(t=0\), the early noise levels) consistently outperformed uniform or logit-normal sampling. This suggests that teaching the robot to resolve high-level structure from pure noise is the most critical part of the learning process.

2. The DiT-X Architecture

A robust policy isn’t just about the mathematical objective; it’s also about how the neural network processes information. Robots deal with multimodal data:

  • High-dimensional inputs: Images, point clouds, language instructions.
  • Low-dimensional inputs: Joint angles, gripper status, timesteps.

Standard architectures often struggle to balance these. The researchers introduce DiT-X, a modified Diffusion Transformer.

Figure 2: Policy Architecture of ManiFlow.

The DiT-X architecture (shown in Figure 2) ingests visual tokens, language tokens, and noisy actions. The critical innovation lies in the DiT-X Block.

In standard Transformers (DiT) or even Multimodal Diffusion Transformers (MDT), the conditioning (how the robot’s state or the timestep influences the processing) is often limited.

Figure 4: DiT-X Block comparison.

As detailed in Figure 4, the DiT-X block introduces Adaptive Cross-Attention. It uses AdaLN-Zero (Adaptive Layer Norm with zero initialization) to condition not just the self-attention layers, but also the input and output of the cross-attention layers.

This means the model can dynamically scale and shift the visual and language features based on the current timestep. For example, at the beginning of a motion (high noise), the model might need to pay attention to broad visual features. Near the end (fine manipulation), it needs to focus on precise geometry. AdaLN-Zero allows the network to selectively modulate these features step-by-step.

Experimental Results

The researchers subjected ManiFlow to a rigorous battery of tests in both simulation and the real world.

Simulation Benchmarks

In simulation, ManiFlow was tested on benchmarks like Adroit (dexterous hand manipulation), DexArt, and RoboTwin (bimanual tasks).

Table 1: Main Simulation Results. Success rates on 12 dexterous tasks in 3 benchmarks.

The results in Table 1 are striking. ManiFlow outperforms both 2D and 3D Diffusion Policies and standard Flow Matching policies across the board. In point cloud-based tasks (3D), it achieves an average success rate of 66.5%, compared to 57.4% for 3D Diffusion Policy.

It also excels in Language-Conditioned Multi-Task Learning. When trained on 48 different MetaWorld tasks simultaneously, conditioned on text instructions, ManiFlow demonstrated a 31.4% relative improvement over baselines.

Figure 6: Comparison on language-conditioned multi-task learning on 48 MetaWorld tasks.

Robustness and Generalization

One of the hardest parts of robotics is generalization. A robot trained on a clean table often fails if you add a coffee mug (distractor) or change the lighting.

The researchers tested ManiFlow on the RoboTwin 2.0 benchmark, which is designed to break policies using domain randomization (novel objects, harsh lighting, cluttered scenes).

Figure 8: Visualization of Domain Randomized Evaluation.

As seen in Figure 8, the environment varies wildly. Despite this, ManiFlow showed superior learning efficiency.

Figure 7: Efficiency & Generalization. Comparison against pi0.

In Figure 7, ManiFlow is compared against \(\pi_0\), a large-scale pre-trained model. Remarkably, ManiFlow (trained from scratch with only 50 demonstrations per task) outperforms the pre-trained model in success rates on specific bimanual tasks. It also scales better: as you add more demonstrations (up to 500), ManiFlow achieves nearly 100% success, whereas baselines plateau.

Real-World Performance

Simulation is useful, but the real world is the ultimate test. The team deployed ManiFlow on three distinct setups:

  1. Humanoid: Unitree H1 with anthropomorphic hands.
  2. Bimanual: Two xArm 7 robots.
  3. Single-Arm: Franka Emika Panda.

Figure 9: Real-Robot Results.

In real-world tests (Figure 9 and Table 2 below), ManiFlow achieved an average success rate of 69.6%, almost doubling the performance of the 3D Diffusion Policy (DP3), which sat at roughly 37%.

Table 2: Detailed Comparison of DP3 and ManiFlow on 8 real robot tasks across 3 robot platforms

The “Handover” task is particularly telling. This requires one hand to pass a bottle to the other—a coordination challenge that requires precise timing and spatial reasoning. ManiFlow succeeded in 22/30 trials, while the baseline only managed 14/30.

Inference Speed

Finally, we return to the core promise of ManiFlow: speed. Because the Consistency Flow Training “straightens” the trajectory, the robot doesn’t need 10 or 20 steps to decide on a move.

Table 4: Few-step Inference.

Table 4 reveals that ManiFlow achieves 63.7% success with just 1 inference step, and 64.5% with 2 steps. In contrast, the Diffusion Policy baseline requires 10 steps to achieve a significantly lower success rate (42.7%). This speed allows for more reactive and safer robot operation.

Conclusion

ManiFlow represents a significant step forward in robotic learning. By combining the mathematical elegance of Flow Matching and Consistency Training with the robust DiT-X architecture, the researchers have created a policy that is:

  1. Fast: Capable of 1-step inference.
  2. Precise: Capable of controlling dexterous, multi-fingered hands.
  3. General: Effective across varying robot morphologies and robust to environmental noise.

This work suggests that the future of robot control may lie not just in larger models, but in smarter training objectives that simplify the complex geometric landscapes our robots must navigate. Whether it’s a humanoid folding laundry or a dual-arm robot assembling a kit, “straightening the flow” seems to be the path forward.