We propose a sensor-guided adaptive contrastive learning framework for visual representation learning in PointGoal navigation. During training, privileged LiDAR sensing guides the contrastive objective through a geometry-aware similarity metric and adaptive temperature scaling, encouraging visual embeddings to capture navigation-relevant structure rather than scene-specific appearance. The resulting encoder is pretrained independently, frozen, and used as the perceptual backbone for reinforcement learning, decoupling representation learning from policy opti- mization. We further introduce a cross-stage domain mismatch between representation pretraining and policy learning to sup- press environment-specific shortcuts and promote reliance on task- relevant features. Extensive experiments in high-fidelity simulation demonstrate that our approach significantly improves policy-level scene transfer across diverse indoor and outdoor environments. At deployment, the agent relies only on monocular RGB observations together with standard task-related inputs such as goal position and proprioceptive signals, without access to LiDAR or other privileged sensors. Our method outperforms large pretrained vision models, navigation-specific representation learning approaches, domain randomization methods, and standard contrastive baselines under severe appearance and semantic shifts. We also release a mul- timodal dataset to support future research on privileged-guided visual representation learning for navigation.

Robust Scene Transfer for PointGoal Navigation Via Privileged-Sensor Guided Contrastive Learning

Amirhossein Zhalehmehrabi;Tiziano Tezze;Alberto Castellini;Alessandro Farinelli
2026-01-01

Abstract

We propose a sensor-guided adaptive contrastive learning framework for visual representation learning in PointGoal navigation. During training, privileged LiDAR sensing guides the contrastive objective through a geometry-aware similarity metric and adaptive temperature scaling, encouraging visual embeddings to capture navigation-relevant structure rather than scene-specific appearance. The resulting encoder is pretrained independently, frozen, and used as the perceptual backbone for reinforcement learning, decoupling representation learning from policy opti- mization. We further introduce a cross-stage domain mismatch between representation pretraining and policy learning to sup- press environment-specific shortcuts and promote reliance on task- relevant features. Extensive experiments in high-fidelity simulation demonstrate that our approach significantly improves policy-level scene transfer across diverse indoor and outdoor environments. At deployment, the agent relies only on monocular RGB observations together with standard task-related inputs such as goal position and proprioceptive signals, without access to LiDAR or other privileged sensors. Our method outperforms large pretrained vision models, navigation-specific representation learning approaches, domain randomization methods, and standard contrastive baselines under severe appearance and semantic shifts. We also release a mul- timodal dataset to support future research on privileged-guided visual representation learning for navigation.
2026
Reinforcement learning, representation learning, vision-based navigation.
File in questo prodotto:
File Dimensione Formato  
2026_RAL_RobustSceneTrasfer_Amir.pdf

accesso aperto

Descrizione: Paper
Tipologia: Versione dell'editore
Licenza: Dominio pubblico
Dimensione 4.73 MB
Formato Adobe PDF
4.73 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11562/1209387
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact