Evaluating the Group Detection Performance: The GRODE Metrics

The detection of groups of individuals is attracting the attention of many researchers in diverse fields, from automated surveillance to human-computer interaction, with a growing number of approaches published every year. Unexpectedly, the evaluation metrics for this problem are not consolidated, with some measures inherited from different fields, other designed specifically for a particular approach, thus lacking in generalization and making the comparisons between different approaches difficult. Moreover, most of the existent metrics are scarcely expressive, addressing groups as atomic entities, ignoring that they may have different cardinalities, and that group detection approaches may fail in capturing the exact number of individuals that compose it. This paper fills this gap presenting the GROup DEtection (GRODE) metrics, which formally define precision and recall on the groups, including the group cardinality as a variable. This gives the possibility to investigate aspects never considered so far, such as the tendency of a method of over- or under-segmenting, or of better dealing with specific group cardinalities. The GRODE metrics have been evaluated first on controlled scenarios, where the differences with alternative metrics are evident, as well as on public datasets, providing a fresh-new panorama of the state-of-the-art.