Digital Techniques for Heritage Presentation and Preservation

This chapter introduces heritage as a concept that acts as a bridge between the past and the present and one that brings pride from the past to the present. Heritages are classified based on various criteria, such as tangible versus intangible, natural versus cultural, portable versus immovable, and domain specific, among others. The management of heritages is a multidisciplinary and multisectoral undertaking. Stakeholders ranging from historians to archeologists, governments, scientists, and international organizations are playing an active role in heritage management and preservation. Of these groups, the contribution of scientists is most visible, especially in the development of scientific approaches, computer-based techniques, and information and communications technology (ICT) tools and platforms employed in the management, processing, sharing, conservation, and preservation of heritages, particularly digital heritages (DH).

Language-based text classification has attracted interest over the years, especially in the fields of business, tourism, hospitality industry, international relations circles, sports, and social media. This chapter surveys various techniques for language-based text classification that have been developed over the years for application in different areas. Classification is performed on multilingual and monolingual text documents. Monolingual documents are classified based on the subject of the contents, whereas multilingual documents are classified based on language. The main techniques used are statistical methods (like regression models, KNN, decision trees, and Bayesian methods) and machine learning methods like neural networks, support vector machines, and deep learning classifiers. Classification performance among these techniques is comparatively evaluated.

In this chapter, we propose a technique for the classification of yoga poses/asanas by learning the 3D landmark points in human poses obtained from a single image. We apply an encoder architecture followed by a regression layer to estimate pose parameters like shape, gesture, and camera position, which are later mapped to 3D landmark points by the SMPL (Skinned Multi-Person Linear) model. The 3D landmark points of each image are the features used for the classification of poses. We experiment with different classification models, including k-nearest neighbors (kNN), support vector machine (SVM), and some popular deep neural networks such as AlexNet, VGGNet, and ResNet. Since this is the first attempt to classify yoga asanas, no dataset is available in the literature. We propose an annotated dataset containing images of yoga poses and validate the proposed method on the newly introduced dataset.

Spotting is finding the location of a particular object without explicitly knowing the entire content in a collection of objects. In this chapter, we consider two types of objects. We consider the word in a document image as an object. Another object is an artifact that is present in terracotta panel images. The proposed object spotting method is based on Wave Kernel Signature (WKS) under the foundation of quantum mechanics. The query image and the document/panel image are smoothened first, and then the Scale Invariant Feature Transform detector is used to obtain the keypoints in both the query image and the document/panel image. Each keypoint is described in terms of WKS. The WKS descriptors represent the average probability of measuring a quantum mechanical particle at a specific location based on quantum energy. In the case of word spotting, a two-step searching technique is introduced to find the region of interest in the document image under test. On the other hand, a single-step searching technique is used to spot figures present in the panel image corresponding to a particular query image. The proposed method is tested on three historical Bangla handwritten datasets and one historical English handwritten dataset, as well as a terracotta panel image dataset. The performance of the proposed method is evaluated using standard metrics.

Image restoration is a crucial computer vision task. This chapter describes two novel methods for the restoration of old, degraded handwritten documents using a deep neural network. In addition, a small-scale dataset consisting of images of 26 heritage letters is introduced. The ground truth data to train the desired network is generated semi-automatically, involving a pragmatic combination of colour transformation, Gaussian mixture model-based segmentation and correction by using mathematical morphological operators. In the first approach, a deep neural network has been used for text extraction from the document image. The background is then reconstructed based on a Gaussian mixture model. Note that a Gaussian mixture model requires setting parameters manually; to alleviate this problem, we propose a second approach where both the background reconstruction and the foreground extraction (i.e. extracting text with its original colour) are achieved using a deep neural network. Experiments demonstrate that the proposed systems perform well on handwritten document images with severe degradation, even when trained with a small dataset. Hence, the proposed methods are ideally suited for digital heritage preservation repositories where the number of samples is low. It is worth mentioning that these methods can be extended easily for printed degraded documents as well.

In this chapter, we propose a framework for the categorization and selection of crowdsourced images towards the 3D reconstruction of heritage sites. The categorization of crowdsourced heritage site images faces challenges due to high dimensionality and less inter-class variance. The categorization of low variant data using clustering techniques demands robust feature space representation. We propose IVAE (Inception-based Variational Autoencoder) to extract deep features from images towards latent space representation and clustering. The 3D reconstruction of heritage sites using categorized crowdsourced images is challenging due to the presence of redundant images affecting the computational complexity. Robust image selection plays a significant role, as it has a greater impact on the final 3D model. However, the image selection based on a single parameter is not sufficient for large-scale 3D reconstruction. We measure the similarity between the images using multiple parameters and generate a combined confidence score towards discarding images with redundant information. We demonstrate the results of the proposed framework using crowdsourced heritage data and show considerable improvement in the quality of reconstruction.

In this chapter, we propose a deep learning-based pipeline for filtering of internet-sourced images towards 3D reconstruction of heritage sites. The 3D reconstruction of heritage sites facilitates creation of virtual walk-through, digital museum and augmented reality. Using internet-sourced images for 3D reconstruction of heritage sites is challenging, as these images may contain blur, text, occlusion, shadow and many other noises. We propose to include pruning and selection of images in the pipeline to select a suitable set of images for 3D reconstruction. We propose a method for pruning of images using learning-based classification models to eliminate the contribution of unwanted images in 3D reconstruction. We also propose a method to select a suitable set of images using a combination of mean-shift and hierarchical clustering algorithms. We demonstrate the proposed pipeline by generating various 3D models of cultural heritage sites.

With increasing tourism and democratization of data, image landmark recognition has been one of the most sought-after classification challenges in the field of vision and perception. After so many years of generic classification of buildings and monuments from images, people are now focusing on fine-grained problems—recognizing the category of each building or monument and indexing information to it. In this chapter, we propose an ensemble network for the classification of Indian landmark images. To this end, the proposed method gives a robust classification by ensembling the predictions from the Graph-Based Visual Saliency (GBVS) network along with supervised feature-based classification algorithms such as K-nearest neighbor (kNN) and Random Forest. The final architecture is an adaptive learning of all the mentioned networks. The proposed network produces a decent score to eliminate false category cases. Evaluation of the proposed model was done on a new dataset, which involves challenges such as landmark clutter, variable scaling, and partial occlusion.

Indian classical dance is an over 5000-year-old multimodal language for expressing emotions. Preservation of dance through multimedia technology is a challenging task. In this chapter, we develop a system to generate a parseable representation of a dance performance. The system will help preserve intangible heritage, annotate performances for better tutoring, and synthesize dance performances. We first attempt to capture the concepts of the basic steps of an Indian classical dance form, named Bharatanatyam Adavus, in an ontological model. Next, we build an event-based, low-level model that relates the ontology of Adavus to the ontology of multimodal data streams (RGB-D of Kinect in this case) for a computationally realizable framework. Finally, the ontology is used for transcription into Labanotation. We also present a transcription tool for encoding the performances of Bharatanatyam Adavus to Labanotation and test it on our recorded data set. Our primary aim is to document the complex movements of dance in terms of Labanotation using the ontology.

This chapter addresses the evolution and interconnection of temple-building traditions across South and Southeast Asia. The remains of early temple architecture are mapped through the comparative analysis of temple geometry through 3D reconstruction. The chapter presents the 3D reconstruction pipeline for combining image-based analysis methods with flexible generative modelling techniques. These 3D schematic reconstructions of individual temples capture the architectural form of the temple as well as the knowledge of temple production and their architectural lineage. Drawing upon canonical descriptions and previous scholarship on temple geometry, the chapter presents schematic reconstructions of four individual temples. A comparative analysis of the similarities and differences between the temples reveals the role of canonical constructive mechanisms underlying these temples. The computational reconstruction of temple geometry is described in the chapter. First, canonical geometry identified from early Indian temple texts is formalized into 3D geometric constructions called scaffolds. Photo-based structure-from-motion (SfM) techniques are used to develop digital point datasets of temple remains. Dissections represent horizontal and vertical profiles that capture attribute features of a temple from field measurements and surveys. Geometric scaffolds and dissections are then combined to propose conjectural reconstructions. The chapter outlines the results and contributions of the work in developing the geometric modelling of early temples. It concludes with an overview of how such digital reconstructions can assist in the conservation of digital cultural heritage in South and Southeast Asia. More broadly, the methods posit a broader understanding of how individual buildings of a particular historical and philosophical lineage may be compositionally connected through computational means to provide a symbolic view of variance in architectural production over time.

The identification of plant species by looking at their leaves, flowers, and seeds is a crucial component in the conservation of endangered plants. Traditional identification methods are manual and time consuming and require domain knowledge to operate. Owing to an increased interest in the automated plant identification system, we propose one that utilizes modern convolutional neural network architectures. This approach helps in the recognition of leaf images and can be integrated into mobile platforms like smartphones. Such a system can also be employed in aiding plant-related education, promoting ecotourism, and creating a digital heritage for plant species, among many others. Our proposed solution achieves a state-of-the-art performance for plant classification in the wild. An exhaustive set of experiments are performed to classify 112 species of plants from the challenging Indic-Leaf dataset. The best-performing model gives Top 1 precision of 90.08 and Top 5 precision of 96.90. We discuss and elaborate on our crowdsourcing web application that is used to collect and regulate data. We explain how the automated plant identification system can be integrated into a smartphone by detailing the flow of our mobile application.