We propose Question-Asking Navigation (QAsk-Nav), the first reproducible bench- mark for Collaborative Instance Object Navigation (CoIN) that enables an explicit, sep- arate assessment of embodied navigation and collaborative question-asking. CoIN re- quires an embodied agent to locate a target object specified through free-form natu- ral language (e.g., “Find the messy red bed”) using only egocentric visual observa- tions and interactive natural-language dialogue with a human (e.g., “Does the red bed have brown pillows?”). The dialogue serves as an active mean to resolve ambiguities of visually-similar object instances in cluttered environments. Existing CoIN bench- marks focus primarily on navigation success and do not offer an independent evalua- tion of collaborative question asking. To address this limitation, QAsk-Nav provides (i) a lightweight question-asking protocol scored independently of navigation, (ii) an en- hanced navigation protocol with realistic, diverse, high-quality target descriptions, and (iii) an open-source dataset, that includes around 26,000 quality-checked reasoning and question-asking traces for training, evaluation and analysis of interactive capabilities of CoIN models. Using the proposed QAsk-Nav benchmark, we develop Light-CoNav, a lightweight unified model for collaborative navigation that is 3× smaller and 70× faster than existing modular methods, while outperforming state-of-the-art CoIN approaches in generalization to unseen objects and environments. Code, benchmark, and dataset are available at https://benchmarking-interaction.github.io/.
Benchmarking Interaction, Beyond Policy: a Reproducible Benchmark for Collaborative Instance Object Navigation
Edoardo Zorzi;Francesco Taioli;Yiming Wang;Marco Cristani;Alessandro Farinelli;Alberto Castellini;Loris Bazzani
2026-01-01
Abstract
We propose Question-Asking Navigation (QAsk-Nav), the first reproducible bench- mark for Collaborative Instance Object Navigation (CoIN) that enables an explicit, sep- arate assessment of embodied navigation and collaborative question-asking. CoIN re- quires an embodied agent to locate a target object specified through free-form natu- ral language (e.g., “Find the messy red bed”) using only egocentric visual observa- tions and interactive natural-language dialogue with a human (e.g., “Does the red bed have brown pillows?”). The dialogue serves as an active mean to resolve ambiguities of visually-similar object instances in cluttered environments. Existing CoIN bench- marks focus primarily on navigation success and do not offer an independent evalua- tion of collaborative question asking. To address this limitation, QAsk-Nav provides (i) a lightweight question-asking protocol scored independently of navigation, (ii) an en- hanced navigation protocol with realistic, diverse, high-quality target descriptions, and (iii) an open-source dataset, that includes around 26,000 quality-checked reasoning and question-asking traces for training, evaluation and analysis of interactive capabilities of CoIN models. Using the proposed QAsk-Nav benchmark, we develop Light-CoNav, a lightweight unified model for collaborative navigation that is 3× smaller and 70× faster than existing modular methods, while outperforming state-of-the-art CoIN approaches in generalization to unseen objects and environments. Code, benchmark, and dataset are available at https://benchmarking-interaction.github.io/.| File | Dimensione | Formato | |
|---|---|---|---|
|
2026_BMVC2026_Benchmarking_Zorzi.pdf
accesso aperto
Descrizione: Paper
Tipologia:
Documento in Pre-print
Licenza:
Dominio pubblico
Dimensione
18.65 MB
Formato
Adobe PDF
|
18.65 MB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.



