Generative models for classification: classification, feature and features organization

La classificazione è uno dei problemi più studiati in machine learning e consiste nell'assegnare un etichetta o classed ad oggetti in input sulla base di informazione quantitativa. Esistono due principali tipi di classificatori, i classificatori generativi, basati su di un modello generativo, e i classificatori discriminative, basati sul concetto di confine separatore. In questa tesi viene mostrato cosa un modello generativo può fare per risolvere il problema della classificazione. In particolar un modello generativo può essere usato come classificatore, per organizzare le descrizioni quantitative degli oggetti o per fornire descrizioni che vengono poi classificate con metodi discriminativi.

Classification is one of the most studied problems in machine learning and involves the placement of data into groups based on quantitative information. Group membership is determined by whether or not a datum contains a specific ``feature'', a decision made is based on a training set of previously labeled items. In order to build robust classifiers, one has to capture various aspect of the data at the same time, this can be accomplished using generative models. Generative models are statistical models that can explain the input data as tangible effects generated from a combination of hidden variables, encoding the causes of data generation, coupled with conditional independencies. These models should be fairly simple, but capable of adapting to the data; the machine learning community has defined this models as flexible when they are minimally structured probability models with a large number of parameters that can adapt so as to explain the input data. Although a generative model can be used for classification using the Bayes rule and marginalization, another family of classifiers, the discriminative classifiers, in normal regimes, achieve better performances since they enable us to construct decision boundaries, incorporating during the learning the concept of discrimination. On the other hand generative models can deal with missing/hidden information, variable length descriptions and they are more robust to overtraining. The prevailing wisdom among the machine learning community is that an ideal classifier should combine these two complementary approaches. In this dissertation I will show what a generative model can do for classification. I will face the genotype classification task, showing with a computational Biology example how a generative model outperforms discriminative classifiers; the idea here is that genotype data are too complex and not separable in classes without calculating the hidden causes that have generated them (the haplotypes). Then I will present a novel family of generative models that can be used to spatially organize features in a set of image, for a more efficient use with a generic discriminative or generative classifier. Finally, I will show original ways to provide features from generative model: by deriving kernel functions for use in discriminative methods, by using by-product of generative models as features or by extracting similarities between samples under a generative model.