Viewing the Viewers: A Novel Challenge for Automated Crowd Analysis

We focus on the automated analysis of spectator crowd, thatis, people watching sport contests alive (in stadiums, amphitheaters etc.),or, more generally, people "watching the activities of an event [. . . ] interested in watching something specic that they came to see". Thisscenario diers substantially from the typical crowd analysis setting (e.g.pedestrians): here the dynamics of humans is more constrained, due tothe architectural environments in which they are situated; people are expected to stay in a xed location most of the time, limiting their activitiesto applaud, support/heckle the players or discuss with the neighbors. Inthis paper, we start facing this challenge by following a social signalprocessing approach, which grounds computer vision techniques in social theories. More specically, leveraging on social theories describingexpressive bodily conduct, we will show how, by using computer visiontechniques, it is possible to distinguish fan groups belonging to dierentteams by automatically detecting their liveliness in dierent momentsof the match, even when they are merged in the stands. Moreover, wewill show how, only by automatically detecting crowd's motions on thestands, it is possible to single out the most salient events of the match,like goals, fouls or shots on goal.