We propose a sensor-guided adaptive contrastive learning framework for visual representation learning in PointGoal navigation. During training, privileged LiDAR sensing guides the contrastive objective through a geometry-aware similarity metric and adaptive temperature scaling, encouraging visual embeddings to capture navigation-relevant structure rather than scene-specific appearance. The resulting encoder is pretrained independently, frozen, and used as the perceptual backbone for reinforcement learning, decoupling representation learning from policy opti- mization. We further introduce a cross-stage domain mismatch between representation pretraining and policy learning to sup- press environment-specific shortcuts and promote reliance on task- relevant features. Extensive experiments in high-fidelity simulation demonstrate that our approach significantly improves policy-level scene transfer across diverse indoor and outdoor environments. At deployment, the agent relies only on monocular RGB observations together with standard task-related inputs such as goal position and proprioceptive signals, without access to LiDAR or other privileged sensors. Our method outperforms large pretrained vision models, navigation-specific representation learning approaches, domain randomization methods, and standard contrastive baselines under severe appearance and semantic shifts. We also release a mul- timodal dataset to support future research on privileged-guided visual representation learning for navigation.
Robust Scene Transfer for PointGoal Navigation Via Privileged-Sensor Guided Contrastive Learning
Amirhossein Zhalehmehrabi;Tiziano Tezze;Alberto Castellini;Alessandro Farinelli
2026-01-01
Abstract
We propose a sensor-guided adaptive contrastive learning framework for visual representation learning in PointGoal navigation. During training, privileged LiDAR sensing guides the contrastive objective through a geometry-aware similarity metric and adaptive temperature scaling, encouraging visual embeddings to capture navigation-relevant structure rather than scene-specific appearance. The resulting encoder is pretrained independently, frozen, and used as the perceptual backbone for reinforcement learning, decoupling representation learning from policy opti- mization. We further introduce a cross-stage domain mismatch between representation pretraining and policy learning to sup- press environment-specific shortcuts and promote reliance on task- relevant features. Extensive experiments in high-fidelity simulation demonstrate that our approach significantly improves policy-level scene transfer across diverse indoor and outdoor environments. At deployment, the agent relies only on monocular RGB observations together with standard task-related inputs such as goal position and proprioceptive signals, without access to LiDAR or other privileged sensors. Our method outperforms large pretrained vision models, navigation-specific representation learning approaches, domain randomization methods, and standard contrastive baselines under severe appearance and semantic shifts. We also release a mul- timodal dataset to support future research on privileged-guided visual representation learning for navigation.| File | Dimensione | Formato | |
|---|---|---|---|
|
2026_RAL_RobustSceneTrasfer_Amir.pdf
accesso aperto
Descrizione: Paper
Tipologia:
Versione dell'editore
Licenza:
Dominio pubblico
Dimensione
4.73 MB
Formato
Adobe PDF
|
4.73 MB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.



