Robots navigating messy indoor spaces need more than a 2D map. This paper introduces TANGO, a system that lets a humanoid robot follow natural-language instructions while adapting its whole body—arms, torso, and gait—to squeeze through cluttered 3D scenes without collisions.

TANGO takes two inputs: a spoken or written instruction in plain language and egocentric RGB (red-green-blue) camera images from the robot’s point of view. Instead of planning only a 2D path, the model directly predicts 29 degrees-of-freedom (DoF) joint-space actions. In other words, it outputs low-level whole-body joint commands so the robot can move its legs, torso, and arms in a coordinated way.

The team trained TANGO entirely in simulation. They built a pipeline that first creates collision-free global paths and then turns those paths into whole-body motions using kinematic motion generation. Those motions are edited to account for nearby obstacles and then refined with reinforcement learning (RL) based tracking so the actions are dynamically feasible. This simulated pipeline provided the action examples the model needed to learn language-conditioned whole-body policies.

In simulation experiments, TANGO achieved state-of-the-art results for vision-language navigation. The authors report that it outperforms strong modular baselines, especially in scenes that require negotiating obstacles with coordinated arm and torso movements. They also tried the model on a real robot: a Unitree G1 humanoid. Without any real-world training, TANGO was deployed zero-shot and the team observed robust language-guided traversal in cluttered real-world scenes.

This work matters because common navigation systems treat movement as a flat, floor-level problem. Humanoid traversal in tight or cluttered spaces needs continuous geometry-aware adaptation of the whole body. By learning to predict joint-level actions from vision and language, TANGO aims to bridge that gap and enable more natural, collision-free motion in complex 3D environments.