Lights on cultural heritage: analysis and visualization of surfaces for cultural heritage based on multi-light imaging and artificial intelligence

Reflectance Transformation Imaging (RTI) is a well-established imaging technique, which uses multi- light data to create relighting models, and is used to improve the visualization of Cultural Heritage (CH) objects. This thesis presents the work carried out during my PhD project, focused on improving and evaluating RTI techniques as a tool to assist experts to analyze cultural finds. In chapter 1 we introduce the problem, briefly describing which were the challenges and how they’ve been tackled. In chapter 2 we provide a background description of RTI, other techniques, and what has been done until the beginning of the project. In chapter 3 we discuss a neural based method to create RTI models, namely NeuralRTI, the modifications we’ve done to the network structure and experiments we performed to assess the improvements achieved. In chapter 4 we demonstrate how we integrated the relighting capability of NeuralRTI into an online viewer for visualizing RTI images, showing challenges we encountered and strategies we adopted to reach real-time rendering. In chapter 5 we discuss the usefulness of RTI when applied to real world use cases, presenting collaborations with researchers and conservators who evaluated the use of RTI to analyze different types of objects. In chapter 6 we make an overview of further problems regarding multi-light data, for example the detection and removal of self-casted shadows in the stack of acquired images. Finally, in chapter 7 we conclude describing the outcomes of this projects and which are the future directions, and we also briefly describe ideas on how to extend the use of RTI beyond its usual application, discussing data annotation, the creation of relightable images of translucent materials, and the combination of relighting and multispectral imaging.