Learning to Evolve: Multi-modal Interactive Fields for Robust Humanoid Navigation in Dynamic Environments

Achieving safe manipulation-oriented navigation for humanoid robots is fundamentally challenged by locomotion-induced perceptual distortion and changes within the environment. We introduce the Multi-modal Interaction Field (MIF), a hierarchical framework that transforms the robot from a passive map-user into an active knowledge-evolver. MIF constructs three synergistic fields: a denoised Appearance Field using confidence-gated 3D Gaussian Splatting, a hierarchical Spatial Field for semantic reasoning, and a Geometry Field that uses a Flow Matching-based generative model to reconstruct high-fidelity meshes for Interaction Pose Safety verification. A closed-loop interaction and adaptation mechanism distinguishes sensor noise from genuine environmental changes and triggers local evolution to rectify obsolete memory. Real-world experiments on a Unitree G1 humanoid show that MIF improves success in dynamic relocation scenarios from 12% to 94% compared with static baselines, while reducing semantic memory footprint by 91.4%.

Conference paper

Robotics: Science and Systems