The role of images in the last ten years has changed radically due to the advent of social networks: from media objects mainly used to communicate visual information, images have become personal, associated with the people that create or interact with them (for example, giving a “like”). Therefore, in the same way that a post reveals something of its author, so now the images associated to a person may embed some of her individual characteristics, such as her personality traits. In this paper, we explore this new level of image understanding with the ultimate goal of relating a set of image preferences to personality traits by using a deep learning framework. In particular, our problem focuses on inferring both self-assessed (how the personality traits of a person can be guessed from her preferred image) and attributed traits (what impressions in terms of personality traits these images trigger in unacquainted people), learning a sort of wisdom of the crowds. Our characterization of each image is locked within the layers of a CNN, allowing us to discover more entangled attributes (aesthetic patterns and semantic information) and to better generalize the patterns that identify a trait. The experimental results show that the proposed method outperforms state-of-the-art results and captures what visually characterizes a certain trait: using a deconvolution strategy we found a clear distinction of features, patterns and content between low and high values in a given trait.

Social profiling through image understanding: Personality inference using convolutional neural networks

Segalin, Cristina
;
Cheng, Dong Seon;Cristani, Marco
2017-01-01

Abstract

The role of images in the last ten years has changed radically due to the advent of social networks: from media objects mainly used to communicate visual information, images have become personal, associated with the people that create or interact with them (for example, giving a “like”). Therefore, in the same way that a post reveals something of its author, so now the images associated to a person may embed some of her individual characteristics, such as her personality traits. In this paper, we explore this new level of image understanding with the ultimate goal of relating a set of image preferences to personality traits by using a deep learning framework. In particular, our problem focuses on inferring both self-assessed (how the personality traits of a person can be guessed from her preferred image) and attributed traits (what impressions in terms of personality traits these images trigger in unacquainted people), learning a sort of wisdom of the crowds. Our characterization of each image is locked within the layers of a CNN, allowing us to discover more entangled attributes (aesthetic patterns and semantic information) and to better generalize the patterns that identify a trait. The experimental results show that the proposed method outperforms state-of-the-art results and captures what visually characterizes a certain trait: using a deconvolution strategy we found a clear distinction of features, patterns and content between low and high values in a given trait.
Image understanding, Social signal processing, Convolutional neural networks, Computational aesthetics, Personality computing
File in questo prodotto:
File Dimensione Formato  
Segalin2017CVIU.pdf

solo utenti autorizzati

Tipologia: Versione dell'editore
Licenza: Accesso ristretto
Dimensione 25.22 MB
Formato Adobe PDF
25.22 MB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11562/970307
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 40
  • ???jsp.display-item.citation.isi??? 30
social impact