Learning Humanoid Loco-Manipulation with Responsibility-Induced Specialized Experts for Vision-Language-Action Models
Video
Abstract
In this paper, we present Responsibility-Induced Specialized Experts (RISE), a responsibility-driven multi-expert Vision-Language-Action (VLA) framework that learns long-horizon humanoid loco-manipulation from segmented locomotion and manipulation-oriented demonstrations. Humanoid loco-manipulation remains challenging due to (i) heterogeneous whole-body behaviors with substantially different action distributions and (ii) the difficulty of collecting reliable long-horizon demonstrations that include behavior transitions. RISE addresses these challenges by inducing behavior-specialized experts via responsibility-driven learning, where each expert’s responsibility reflects how well it reconstructs the target action. This mechanism encourages experts to specialize in different behavioral patterns and enables the policy to model heterogeneous action distributions. The resulting expert weighting dynamics allow RISE to coordinate experts over time and infer behavior transitions during execution, enabling long-horizon loco-manipulation despite being trained only on segmented data. We evaluate RISE in simulation and on a real humanoid robot, demonstrating robust loco-manipulation with frequent locomotion–manipulation transitions and improved performance over baselines trained on segmented demonstrations.
Method Overview
RISE uses a shared vision-language backbone with multiple action experts and a meta-controller. Given the visual observation, language instruction, and robot state, the VLA module predicts an upper-body action together with a high-level locomotion command. A pre-trained locomotion policy then produces the corresponding lower-body action for whole-body humanoid control.
RISE
During training, RISE assigns each expert a responsibility based on its flow-matching reconstruction error. Experts that better explain a demonstrated action receive larger updates, encouraging specialization into different behavior patterns such as locomotion- and manipulation-dominant motions. The meta-controller learns to predict these expert weights from the shared representation.
Because the learned gating weights change with the dominant behavior, they also provide a useful signal for detecting subtask transitions. This lets RISE compose segmented demonstrations into long-horizon loco-manipulation sequences at inference time.
Simulation Experiments
We evaluate RISE on compositional humanoid loco-manipulation tasks in simulation, combining segmented locomotion and manipulation behaviors into multi-step move, pick, and place sequences.
Real-World Experiments
We further evaluate RISE on a real Unitree G1 humanoid, executing long-horizon loco-manipulation sequences that require repeated transitions between whole-body locomotion and dexterous manipulation.
BibTeX
@inproceedings{kee2026rise,
title = {Learning Humanoid Loco-Manipulation with Responsibility-Induced Specialized Experts for Vision-Language-Action Models},
author = {Kee, Hogun and Oh, Wooseok and Kim, Jooyoung and Jeong, Jaeyeon and Jung, Hyewoo and Son, Hyeondal and Lee, Hosung and Kang, Minjae and Oh, Songhwai},
booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
month = {September},
year = {2026}
}