A Modeling-by-matching approach for clothed human animation

The realization of animated 3D characters has always been expensive in terms of resources and time. Animators spend months in the creation of realistic 3D shapes to be animated. Obtaining an automatic method for building a high-quality shape with the desired characteristics for the animation is a challenging problem. In the last decade, the availability of 3D scanning devices, though, is highly increased, and the cost is progressively decreased. Acquiring a 3D model from the real world has become within everyone’s reach. The most significant disadvantage of these methods is the noise and the irregularity of the 3d meshes obtained from the acquisition, so the interest by researchers in algorithms to handle and clean this type of data has grown highly. In this thesis, I present a complete pipeline to combine realistic geometries acquired from the real world and the desired properties to create an animation-ready 3D model using a modeling-bymatching technique. The idea is to provide a robust matching to transfer all the information we need for a proper animation from an artist-defined 3D model to a 3D scan of a clothed human. First, we present a new 3D dataset for clothed human segmentation and garment classification. This step of the pipeline divides the separate layers of the input 3D scans and classifies the typology of the outfit of the model, facilitating the following passages of the pipeline. Then, we explore a new method for skeleton transfer from one shape to another by using a novel approach that lets us transfer the position of the joints composing the skeleton building a spectral regressor. In addition, by using the transferred skeleton, we solve an optimization problem to find the 3D rotations of the pose of the input model, which is fundamental for the transfer of an animation sequence. Finally, we will show a robust method based to obtain a correspondence between shapes that led us to performa tessellation transfer on both humans and garments. All these three passages are based on the functional maps framework, the process of finding matches on functions defined on the surfaces. This matching approach is efficient and flexible and lets us be independent by the discretization, the details on the surfaces, and the pose of the shapes involved. With such fundamental steps, we obtain a fully automatic pipeline to provide an animated 3D setup starting from a single 3D scan, providing new approaches to modeling and animation. All the data and the code we produced are freely available for research purposes.