Visual Computing for Cultural Heritage

Archaeologists reconstruct past human activity from material culture remains. Recording, representing and reconstructing artifacts or contexts is a long, morose, and often expensive process. Computers have radically changed traditional methodologies and are creating opportunities to develop more eloquent images or graphic files that convey compressed information and engage the public in a more participative way. Archeological reconstructions are thinking tools that allow us to reason better and faster about our past and present, and computer graphics can replace the traditional long texts and orthographic images with a rich learning environment that transforms the learning experience into an active and critical mental process. This chapter analyses the current methodologies and evaluates the cost-benefits of the best off-the-shelf software packages and their potential to improve the recording, representing, reconstructing, and sharing archaeological contexts and artifacts.

Computer graphics tools and techniques enable researchers to investigate cultural heritage and archaeological sites. They can facilitate documentation of real-world sites for further investigation, and enable archaeologists and historians to accurately study a past environment through simulations. This chapter explores how light plays a major role in examining computer-based representations of heritage. We discuss how light is both documented and modelled today using computer graphics techniques and tools. We also identify why both physical and historical accuracy in modelling light is becoming increasingly important to study the past, and how emerging technologies such as High Dynamic Range (HDR) imaging and physically-based rendering is necessary to accurately represent heritage.

High dynamic range (HDR) technology enables the capture, storage, transmission and display of real-world lighting at a high precision as opposed to traditional low dynamic range (LDR) imaging. One of HDR’s main features is its ability to reproduce very bright and very dark areas simultaneously. Dynamic range describes the span between these extrema in the brightness scale. HDR research investigates the generation, capturing, processing, transmission, storage and reproduction of HDR content. Cultural heritage represents our legacy that must be passed on to future generations. As it is increasingly threatened with deterioration, destruction and disappearance, its documentation, conservation and presentation is of high importance. Given the real-world dynamic range and the limitations of conventional capture and display technology, HDR imaging represents an invaluable tool for accurate documentation, virtual reconstruction and visualisation of cultural heritage. HDR is used by academics, museums, and media to visualise the appearance of sites in various periods in time. Physically-based 3D virtual reconstructions are used for studying existing or ruined cultural heritage environments. This in turn enables archaeologists to interpret the past and deduce new historical knowledge. In this chapter we present the HDR pipeline, along with its use for cultural heritage preservation, recreation and presentation.

Accurate 3D reconstruction and realistic visualization of cultural heritage allow experts to fine-tune their theories on the lost links in the history of civilization. Although the 3D reconstruction is a significant challenge, precisely because of the state of degradation over the years, it constitutes a crucial task for experts to study and interact with long disappeared settlements and structures. Furthermore, the public, in general, will be provided with the conditions to explore them in virtual environments, thus fostering cultural, social, and scientific participation. Highly accurate reconstruction is, nevertheless, a very complex task, where all stages of image synthesis must be carefully executed from highly detailed 3D models to obtain a faithful depiction of the object of interest. Meanwhile, the textual descriptions and geospatial data collected by archaeologists on-site may be used to overcome the absence of visual information. Still, this data will not suffice, in which case procedural modeling turns out to be essential to avoid a great deal of time and labor-consuming modeling processes. Procedural modeling tools automatically generate three-dimensional models through computational processes that extend the base information according to a specific algorithm. In order to avoid reprograming the procedural modeling systems, we use mathematical methods that operate on parametrical symbolic descriptions that, flexibly, can model different types of objects. The most used mathematical methods are fractal geometry and formal grammars, particularly L-systems and shape grammars. In this chapter, we will approach the current advances in the area of procedural modeling and how these tools can be used to generate 3D models of cultural heritage. We also explore the relevant dimension of time, extending the modeling tasks to 4D. These applications do not focus on very specific landmarks, like cathedrals or palaces, which require manual effort or image-based techniques to capture the model with a high level of visual fidelity. Instead, we focus on modeling cities and their evolutions or the surroundings of these landmarks, that allow for an increased automation of the modeling process.

Keyword spotting is an alternative methodology for document indexing based on spotting words directly on images without the use of a character recognition system. In this paper, an overview of recent techniques and available databases for keyword spotting is presented focusing on the specific characteristics of old Greek machine-printed and handwritten documents. These documents are import treasures of cultural heritage and a valuable source of information for scholars. Indexing of such content is a very challenging task considering the additional problem of having many character classes as well as a variety of different diacritic mark combinations that may appear above or below Greek characters. All steps of a keyword spotting system are highlighted, namely, preprocessing for image binarization, enhancement and segmentation, feature representation, matching and word retrieval, in order to report the efficiency of current keyword spotting approaches when applied to old Greek documents.

Performing arts and in particular dance is one of the most important domains of Intangible Cultural Heritage. However, preserving, documenting, analyzing and visually understanding choreographic patterns is a challenging task due to technical difficulties it involves. A choreography is a time-varying 3D process (4D) including dynamic co-interactions among different actors (dancers), emotional and style attributes, as well as supplementary ICH elements such as the music tempo, the rhythm, traditional costumes etc. Recent technological advancements have unleashed tremendous possibilities in capturing, documenting and storing Intangible Cultural Heritage content, which can now be generated at a greater volume and quality than ever before. The massive amounts of RGB-D and 3D skeleton data produced by video and motion capture devices. The huge number of different types of existing dances and variations dictate the need for organizing, archiving and analyzing dance-related cultural content in a tractable fashion and with lower computational and storage resource requirements. Motion capturing devices are programmable to extract humans’ skeleton data in terms of 3D points each corresponding to a human joint. This information can be combined with computer graphics software toolkits for modelling, classification and summarization purposes. In this chapter, we present recent trends in choreographic representation in terms of modelling, summarization and choreographic pose recognition. We survey recent approaches employed for the extraction of representative primitives of choreographic sequences, the recognition of choreographic pose and dance movements, as well as for the analysis and semantic representation of choreographic patterns.

The advent of powerful image processing and machine learning tools has generated opportunities for more automated and efficient paleography. In this work, methods were developed to automate aspects of the transcription process of ancient papyri. First, a technique is proposed that employs color thresholding and contouring for the automatic extraction of symbols from papyri photographs. Second, two symbol classifiers are considered: a support vector machine that intakes Histogram of Oriented Gradients features, and a convolutional neural network. Finally, a novel system is described based on the sliding window approach that limits the number of windows to be considered and uses a combination of a support vector machine and a convolutional neural network to benefit from both accurate and fast detections. The performance of these methods was evaluated on a set of papyri photographs from the Petrie Museum of Egyptian Archaeology.

Underwater Cultural Heritage (CH) sites are widely spread; from ruins in coastlines up to shipwrecks in deep. The documentation and preservation of this heritage is an obligation of the mankind, dictated also by the international treaties like the Convention on the Protection of the Underwater Cultural Heritage which fosters the use of “non-destructive techniques and survey methods in preference over the recovery of objects”. However, submerged CH lacks in protection and monitoring in regards to the land CH and nowadays recording and documenting, for digital preservation as well as dissemination through VR to wide public, is of most importance. At the same time, it is most difficult to document it, due to inherent restrictions posed by the environment. In order to create high detailed textured 3D models, optical sensors and photogrammetric techniques seems to be the best solution. This chapter discusses critical aspects of all phases of image based underwater 3D reconstruction process, from data acquisition and data preparation using colour restoration and colour enhancement algorithms to Structure from Motion (SfM) and Multi-View Stereo (MVS) techniques to produce an accurate, precise and complete 3D model for a number of applications.

Virtual Reality (abbreviated VR), although far from being a new concept in computer science, is increasingly considered nowadays as the digital media technology that can most directly linked with Archaeology and Archaeological Reconstruction (with the term “reconstruction” in this context being officially agreed on meaning the “re-building of a monument to its state at a time of its history chosen for that particular representation”). The potential along with the many degrees of freedom offered by this branch of technology in the sector recently led experts to even start talking about the dawn of the hyper-tourist era. The ever increasing amount of research in the area, as well as the number of actual archaeological sites that have been reconstructed in a VR environment to the present, appear to support both directly and indirectly such a strong statement. It is not an exaggeration to say that archaeological research is now dependent on VR more than ever before. Also, many of these applications include a pedagogical aspect in their design that makes them ideal educational platforms for students in archaeology and professionals in the area alike.

Underwater sites are a harsh environment for augmented reality applications. Divers must battle poor visibility conditions, difficult navigation, and hard manipulation with devices under water. This chapter focuses on the problem of localizing a device under water using markers. It discusses various filters that enhance and improve underwater images and their impact on marker-based tracking. Then, it presents different combinations of ten image-improving algorithms and four marker-detecting algorithms and tests their performance in real situations. All solutions are designed to run real-time on mobile devices to provide a solid basis for augmented reality. The usability of this solution is evaluated on locations in the Mediterranean Sea. Results show that image improving algorithms with carefully chosen parameters can reduce the problems with underwater visibility and enhance the detection of markers. The best results are obtained with marker detecting algorithms specifically designed for marine environments.

In this work, a new and innovative way of spatial computing that appeared recently in the bibliography called True Augmented Reality (AR), is employed in cultural heritage preservation. This innovation could be adapted by the Virtual Museums of the future to enhance the quality of experience. It emphasises, the fact that a visitor will not be able to tell, at a first glance, if the artefact that he/she is looking at is real or not and it is expected to draw the visitors’ interest. True AR is not limited to artefacts but extends even to buildings or life-sized character simulations of statues. It provides the best visual quality possible so that the users will not be able to tell the real objects from the augmented ones. Such applications can be beneficial for future museums, as with True AR, 3D models of various exhibits, monuments, statues, characters and buildings can be reconstructed and presented to the visitors in a realistic and innovative way. We also propose our Virtual Reality Sample application, a True AR playground featuring basic components and tools for generating interactive Virtual Museum applications, alongside a 3D reconstructed character (the priest of Asinou church) facilitating the storyteller of the augmented experience.

Heritage Crafts (HCs) involve craft artefacts, materials, and tools and encompass craftsmanship as a form of Intangible Cultural Heritage. Intangible HC dimensions include dexterity, know-how, and skilled use of tools, as well as identity and traditions of the communities in which craftsmanship is, or was, practiced. HCs are part of history and have impact upon the economy of the areas in which they flourish. Despite their cultural significance, efforts towards HC digital representation and presentation are scattered. In this chapter, as a first step towards the Representation and Presentation of Heritage Crafts as cultural heritage (CH), pertinent requirements and needed technological components are investigated. Representation is expected to capture the wide spectrum of knowledge that a HC covers, from objects and their making, to hand gestures and tool uses that define craft motor skills, to the societal value, economic impact, and historical significance of HCs. Presentation is expected to address the need of exploiting Representation to conserve cultural resources, contribute to their accurate interpretation, provide essential and authentic experiences, as well as stimulate revenues of cultural resources through thematic tourism. This chapter lays the foundations and envisions the further development and formalisation of a suitable approach to HC representation and presentation in the form of a generic protocol that together with the appropriate technological tools can be applied to any HC instance.

Serious games and gamification have been proposed as approaches to solve problems in various areas by utilizing game technologies, game design components and even fully fledged games. However, when games are applied in a context outside the gaming sphere where users are not used to game interfaces and game culture, this may cause problems. In the case of cultural heritage applications this may create confusion or even put people off if they don’t understand what to do to take part in the experience. This chapter contributes a synthesized retrospective overview of three successive research and development projects conducted at the University of Skövde since 2007 and will present theoretical frameworks, conceptual studies, and production models for cultural heritage experiences for diverse audiences. In particular, we present a detailed case of a cultural heritage site which has been enhanced by game design concepts and visualizations to provide a richer experience for visitors. The chapter will also show the importance of user experience testing as an integral part of the production cycle in order to ensure a pleasant and understandable visit for visitors with different backgrounds and experiences of video games.

Since museums, heritage sites and archives are important for the preservation of our cultural heritage; recently there has been an attempt to promote better absorption of cultural knowledge by involving the learners in the process.

Digital games provide a recognised means of engagement and education when addressing challenges in educating and immersing individuals in their own heritages, and those of other cultures. Similarly, gamification techniques, commonly expressed as the addition of game elements to an existing process, have been successfully applied to augment existing resources and programmes. The many examples of gamification or serious games focusing on cultural heritage also highlight the potential benefits of using these principles for the purposes of supporting preservation and learning. In this chapter, we present I-Ulysses, a virtual-reality game designed to engage based around the notable work Ulysses by Irish author James Joyce. The rationale for the selection of Ulysses as a basis for the game’s content and design was two-fold; firstly because of its cultural impact within Ireland, and secondly as its content appeared well-suited to exploration as a virtual reality experience. Facets of gamification are explored in I-Ulysses through key mechanics, including a focus towards virtual worlds and crowd intelligence based on real-world data, to highlight how these principles can be employed for cultural heritage preservation and knowledge transfer. Through feedback obtained from focus groups interacting with I-Ulysses, it can be seen that the gamified mechanics presented through the lens of virtual reality provide an informative and educational guide to Ulysses that would engage and appeal to a wide audience.

Digital storytelling in cultural heritage has been recognized as an effective technique for communicating heritage interpretation to the public. This chapter reports on the iterative design, development, and evaluation of a mobile digital storytelling experience for the Athens University History Museum. The experience combines a visually rich, illustrated and animated story of a fictional character, an aspiring young student of the University in 1840, with informational content about the museum’s exhibits. The chapter discusses the insights gained from the composition and integration of the visual form, the design of the story and overall experience, and the evaluation of its impact on visitors, focusing both on the technology and the story perspective. It includes insights for the effective design of digital storytelling experiences, identifying also challenges and needs for future work in the field.

In modern era digital technologies are intensively used for presentation of tangible and intangible cultural heritage. High fidelity 3D reconstructions of cultural monuments and sites, virtual presentations of dances, crafts and traditions offer the users time travel to the past. However, all these digital assets need to incorporate the information on the context of presented cultural heritage objects, as well as related events and characters. This is implemented through digital storytelling. Interactive digital stories become parts of museum exhibitions, virtual and augmented reality applications and, of course, serious games. This chapter will present an overview of techniques and implementations of digital storytelling and its incorporation into virtual cultural heritage presentations. It will discuss challenges of adjusting classical storytelling methods and tools to virtual environments and VR videos, telling stories which users will find educational and entertaining at the same time, stories fitting into VR and AR navigation and complementing the 3D geometry with emotional and emphatic content. Novel interactive digital storytelling methods presented in this chapter will revolutionize the visual language and offer new media expressionism for the future.

This chapter explores the integration of affective storytelling in virtual museum (VM) experience. The use of Information and Communication Technologies (ICT) affects the way people create, communicate and learn, opens novel opportunities and provides new means for museums to create narratives, express their points and provide quality experiences. Museums’ integration of new media, empowers people to construct their own understandings within an open-ended framework. ICT permeate cultural life, not only by introducing new forms of creative expression and meanings for art, but also by enriching, transforming and enhancing the museum experience. ICT in museums can empower curators to disseminate their ideas and facilitate understandings of the complexities regarding museum exhibits and thus develop aesthetic perception, sensitivities and creativity. In this chapter, the function and cultural significance of storytelling as such is addressed, before venturing into discussing digital storytelling vis-à-vis cultural heritage organizations’ practices. Furthermore, the potential of digital storytelling as fulcrum for rethinking museums as affective spaces in-dialogue with their audiences is delineated.

Digitized historical photographs are invaluable sources and key items for scholars in Cultural Heritage (CH) research. Properties of photographic items, such as position and orientation of the camera, can be automatically estimated using Structure from Motion (SfM) algorithms to enable spatial queries on image repositories. Interactive spatial and temporal browsing of photographs of architecture and corresponding 3D models allows historians to gain knowledge about the development of a city, as well as about the changing interest of photographers in depicting particular buildings over time. In this chapter, we present a classification of phenomena modeling the statistical distribution of historical photographic depictions of architecture. This classification serves the design of specialized visualization methods that show statistical aggregation of photographs in spatial contexts, thus supporting research workflows of art and architectural historians.

The London Charter for the Computer-Based Visualisation of Cultural Heritage (2009) lays out best-practice guidelines for producing reconstructions of ruined buildings but does not mandate specific tools, workflows, or data formats, acknowledging that technology will change over time. Their implementation is left up to the individual researcher. The approach described here is designed to produce a virtual 3D model for public consumption within the scope of a small, individual research project. It allows the user to query metadata and understand degrees of accuracy without sacrificing a photorealistic, immersive experience. Recognising that accuracy is dependent on the level of detail at which the reconstruction is to be made and viewed, it is presented as a matrix rather than a linear scale. This allows elements of the reconstruction to be sorted into 12 discrete categories of accuracy. The goal is a scientifically validated virtual reconstruction that can be used to teach a non-professional audience about the metadata that goes into such work.

The preservation and presentation of cultural heritage (CH) encompasses many domains and disciplines and ranges from tangible CH, traditionally taking the form of museum exhibits and historical sites that are open to the public to intangible CH, focussing on human and societal aspects of CH, as opposed to physical artefacts. The use of computer graphics (CG) and related techniques such as interactive virtual environments since the 1990s has had a profound impact on the presentation of and public engagement with CH, allowing virtual reconstruction of archaeological/historical sites as well as the virtual (re-)construction of culturally and historically relevant artefacts. These are frequently implemented using bespoke or proprietary systems, often explicitly created with a CH application in mind, which may require specialist expertise or significant investment. There exist, however, alternative approaches that can simplify and improve the uptake of CG for CH. In this chapter we discuss how off-the-shelf CG systems such as developer and artists’ tools for the entertainment industries, which are comparatively inexpensive, usually provide open developer licenses, and sometimes are even available free of charge, or affordable consumer-level hardware, can be used for the preservation and presentation of tangible and intangible CH, the application of which we illustrate with a set of case studies.