Free energy score spaces: using generative information in discriminative classifiers

A score function induced by a generative model of the data can provide a feature vector of a fixed dimension for eachdata sample. Data samples themselves may be of differing lengths (e.g., speech segments, or other sequential data), but as ascore function is based on the properties of the data generation process, it produces a fixed-length vector in a highly informativespace, typically referred to as “score space”. Discriminative classifiers have been shown to achieve higher performances inappropriately chosen score spaces with respect to what is achievable by either the corresponding generative likelihood-basedclassifiers, or the discriminative classifiers using standard feature extractors. In this paper, we present a novel score space thatexploits the free energy associated with a generative model. The resulting free energy score space (FESS) takes into accountthe latent structure of the data at various levels, and can be shown to lead to classification performance that at least matchesthe performance of the free energy classifier based on the same generative model, and the same factorization of the posterior.We also show that in several typical computer vision and computational biology applications the classifiers optimized in FESSoutperform the corresponding pure generative approaches, as well as a number of previous approaches combining discriminatingand generative models.