Human pose estimation (HPE) based on deep neural networks (DNN) aims to predict the poses of human body in videos without needing markers. One of the main limitations in its applicability is consistently identifying and tracking the keypoints of an individual in multi-person scenarios. Despite various solutions based on image analysis being attempted, challenges such as model accuracy, occlusions, or individuals exiting the camera’s field of view often result in the loss of the association between humans and their keypoints across video frames. In this article, we propose a human identification and tracking methodology in multi-person environments based on data fusion between HPE software and wearable IMU sensors. We demonstrate how to align the data generated by these two sensor categories (camera-based HPE and IMUs) and assess the alignment between each skeleton of keypoints and IMU pair using a scoring system. Additionally, we illustrate how to combine different metrics, such as orientation, acceleration, and velocity, to address alignment problems caused by inaccuracies in sensor data.

Combining 3D Human Pose Estimation and IMU Sensors for Human Identification and Tracking in Multi-Person Environments

Mirco De Marchi;Cristian Turetta;Graziano Pravadelli;Nicola Bombieri
2024-01-01

Abstract

Human pose estimation (HPE) based on deep neural networks (DNN) aims to predict the poses of human body in videos without needing markers. One of the main limitations in its applicability is consistently identifying and tracking the keypoints of an individual in multi-person scenarios. Despite various solutions based on image analysis being attempted, challenges such as model accuracy, occlusions, or individuals exiting the camera’s field of view often result in the loss of the association between humans and their keypoints across video frames. In this article, we propose a human identification and tracking methodology in multi-person environments based on data fusion between HPE software and wearable IMU sensors. We demonstrate how to align the data generated by these two sensor categories (camera-based HPE and IMUs) and assess the alignment between each skeleton of keypoints and IMU pair using a scoring system. Additionally, we illustrate how to combine different metrics, such as orientation, acceleration, and velocity, to address alignment problems caused by inaccuracies in sensor data.
2024
Human identification and tracking
Data fusion
Human Pose Estimation
Body Area Network
Wearables
IMU
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11562/1127086
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact