Feature Calibration
Multi-layer aggregation, normalization, and a representation-aware noise schedule prepare frozen DINO features for diffusion-based dynamics modeling.
ReWAM
Rethinking Representations
for World-Action Modeling
Let the policy shape what the representation encodes,
and the world model learn how it evolves.
1 Huazhong University of Science & Technology2 D-Robotics3 Horizon Robotics4 Fudan University5 Xi’an Jiaotong University
93.6%
RoboTwin 2.0 success
Both clean and random settings12.29
RoboDojo average score
With ~600 hours of embodied pre-training91.7%
Real-robot success
55 successful trials out of 600
Generative video pre-training
Built on frozen DINOv3 featuresTHE CENTRAL QUESTION
What makes a good
world-action representation?
World-action models need a shared space that supports both predicting future observations and choosing robot actions. Visual reconstruction quality alone does not tell us how useful that space is for control.
ReWAM starts from frozen DINO features, calibrates them for dynamics learning, and organizes them into temporal world states. The action objective shapes these states; the world objective learns their evolution. This division of roles supports strong robot policies without generative video pre-training.
Read the full studyTHE METHOD
Perception, organized for action.
Three components turn perceptual features into a space for prediction and control.
Temporal Representation Bottleneck
A trainable bottleneck combines paired frames across camera views into compact, 128-dimensional world-state tokens.
Action-Grounded Representation Shaping
Only action-loss gradients update the bottleneck. World-loss gradients stop at the representations, allowing the world model to learn their dynamics.
SIMULATION BENCHMARKS
From representations to results.
RoboTwin 2.0
50 tasks · 100 evaluation episodes per task and setting
93.6%Clean
93.6%Random
No embodied pre-training or generative video pre-training. Main results use five training epochs.
RoboDojo
Five capability categories · 35 training tasks
| Method / embodied pre-training | Score ↑ | Success ↑ |
|---|---|---|
| Fast-WAM / none | 3.48 | 2.03% |
| ReWAM / none | 9.42 | 5.82% |
| Fast-WAM / ~600 h | 10.95 | 6.80% |
| ReWAM / ~600 h | 12.29 | 8.28% |
The two ~600 h models use the same embodied pre-training data.
Why does representation design matter?
RoboTwin 2.0 success rates under a matched one-epoch budget. These experiments use a different budget from the main results above.
| Representation | Clean ↑ | Random ↑ |
|---|---|---|
| Raw DINO features | 71.00% | 71.54% |
| Feature Calibration | 83.90% | 83.03% |
| Fixed random temporal projection | 86.76% | 85.94% |
| ReWAM with AGRS | 90.70% | 89.12% |
ON PHYSICAL HARDWARE
Three tasks. A bimanual robot.
Evaluated on the Piper platform over 20 trials per task, following embodied pre-training and task-specific post-training.
01 / COLLECT OBJECTS
Pick, place, repeat.
Collect tabletop objects into a basket.
02 / FOLD CLOTHES
Handle the deformable.
Manipulate and fold a garment.
03 / UNPACK LUNCHBOX
Go beyond a single step.
Open a lunchbox and retrieve food.
BUILD ON THIS WORK
Citation
@article{jiang2026rewam,
title={Rethinking Representations for World-Action Modeling},
author={Haoyi Jiang and Liu Liu and Xinjiang Wang and Zhihao Sun and Zequn Chen and Sen Wang and Xinjie Wang and Xia Chen and Jingfeng Yao and Weiheng Zhao and Shanglin Yuan and Zhizhong Su and Wei Sui and Wenyu Liu and Xinggang Wang},
journal={arXiv preprint arXiv:2609.38163},
year={2026},
url={https://arxiv.org/abs/2609.38163}
}