Upper-Body Pose-Based Gaze Estimation for Privacy-Preserving 3D Gaze Target Detection

Gaze Target Detection (GTD), i.e., determining where a per-son is looking within a scene from an external viewpoint, is a challenging task, particularly in 3D space. Existing approaches heavily rely on ana-lyzing the person’s appearance, primarily focusing on their face to pre-dict the gaze target. This paper presents a novel approach to tackle this problem by utilizing the person’s upper-body pose and available depth maps to extract a 3D gaze direction and employing a multi-stage or an end-to-end pipeline to predict the gazed target. When predicted accu-rately, the human body pose can provide valuable information about the head pose, which is a good approximation of the gaze direction, as well as the position of the arms and hands, which are linked to the activ-ity the person is performing and the objects they are likely focusing on. Consequently, in addition to performing gaze estimation in 3D, we are also able to perform GTD simultaneously. We demonstrate state-of-the-art results on the most comprehensive publicly accessible 3D gaze target detection dataset without requiring images of the person’s face, thus pro-moting privacy preservation in various application contexts. The code is available at https://github.com/intelligolabs/privacy-gtd-3D.