Augmented Reality Art

The explosion of mobile technology and its growing accessibility to the global public has made augmented reality (AR) a powerful tool for activism. This is the story of the first generation of activists that began working with augmented reality to further their causes. These activists pioneered the development of mobile AR in search of what made it unique from other mediums and what traits could be used to further activist’s agendas. Many of these works are the first explorations of their type with this new technology and act as a road map for future activists working with AR. What dangers do those working with this technology face? Does AR have the ability to empower the masses? Can it create real social change and can it unite society by turning virtual experiences into physical ones? The activists in this chapter set out to find these answers.

This chapter describes augmented reality interventions led by the author in 2011 with the artist group Manifest.AR at the Venice Biennale and in collaboration with the design office PATTU at the Istanbul Biennale. The interventions used the emerging technology of mobile augmented reality to geolocate virtual artworks—visible for viewers in the displays of their smartphones as overlays on the live camera view of their surroundings—inside the normally curatorially closed spaces of the exhibitions via GPS coordinates. Our interventions used the site-specific character of the technology to create works of art that stand in dialogue with the sites and will retain their relevance long after the biennials are over. The site figures as the canvas for the artworks and forms an integral visual and contextual component of each artwork. Unlike physical art interventions, the artworks cannot be removed or blocked by the curators or other authorities and will remain at those locations as long as the artist desires. The artworks exploit the site specificity as an integral part of the artwork while simultaneously questioning the value of location to canonize works of art, and the power of the curator as gatekeeper to control access to the spaces that consecrate works of art as part of the high art canon.

Existing augmented reality (AR) taxonomies act as generic classifiers of AR systems and do not provide any insights into technology adoption for a given use context. For this reason, this chapter proposes an alternative activity-based taxonomy method that is designed for technology adopters and aims to highlight how well are opportunities created by advances in technology really utilized in a specific context. The proposed method is evaluated on the case study of AR technology adoption in the context of art and cultural heritage. In this process, we build an AR taxonomy for art and cultural heritage which was used to classify 86 AR applications in this domain. The results of classification provided meaningful technology adoption insights, such as: (i) general lack of support for personalization and communication activities of visiting a museum where applications fail to exploit AR potential such as providing support for: context aware bookmarking, artistic expression of the visitor (e.g. enabling visitors to curate augmentations for the exhibited artefacts), sharing the visit experience of “I was here”; (ii) low quality of support for analytical activity where applications fail to show interesting information such as information that is there but cannot be seen by the naked eye; (iii) low quality of support for sensual activity where provided augmentations fall short of extending the art form or the artistic experience.

In this chapter, Gregory Ulmer theorizes augmented reality and ubiquitous computing in general, while John Craig Freeman presents examples of his work in place-based augmented reality public art and describes the work within the framework of electracy (the digital apparatus). Apparatus theory correlates technological innovations with the corresponding inventions in institutional practices, including individual and collective identity behaviors. Ulmer and Freeman, working with an electrate consultancy—the EmerAgency—test an augmented deliberative design rhetoric intended to overcome individual alienation from collective agency. It is an electrate equivalent of the ancient theoria, a community practice in which a team of trusted citizens travelled to sites of events to sort out fact from rumor. Results of this theory tourism were reported in the public square and certified as truth. Theoria, augmented by literacy, became journalism—the fourth estate of a democratic society. The konsult practice described in this essay updates theoria for a fifth estate with a new function supporting collective well-being, in the global experience of a potentially ubiquitous public square.

This chapter proposes that augmented reality art and open data offer the potential for a redefinition of urban interventionist art practices. Data has emerged as a significant force in contemporary networked culture from the commercial commodification of online presence as practised by internet giants Facebook and Google to the 2013 revelations of the unprecedented scale of the US Government’s data collection regime carried out by the NSA (Gellman and Piotras, U.S., British intelligence mining data from nine U.S. Internet companies in broad secret program, http://​www.​washingtonpost.​com/​investigations/​us-intelligence-mining-data--us-internet-companies-in-broad-secret-program/​2013/​06/​06/​3a0c0da8-cebf-11e2-8845-d970ccb04497_​story.​html, 2013). Big data and its effective deployment are seen as essential to the efficient running of any enterprise, from city governance to commercial enterprise and, of course, government intelligence services. In parallel to developments in big data, open data sources have proliferated opening access to myriad data sources previously only available to government and corporations. We have seen advances in techniques of data scraping and manipulation which have democratised the ability to parse, analyse and effectively manipulate data, and developments which have powerful implications for artists and activists. This chapter examines the possibilities for redefining the activist art practice of urban intervention with data and augmented reality to introduce new hybrid techniques for critical spatial practice (Rendell, Critical spatial practice. http://​www.​janerendell.​co.​uk/​wp-content/​uploads/​2009/​06/​critical-spatial-practice.​pdf. Accessed 18 September 2017, 2008). The combination of AR and Open Data (in the broadest post-wikileaks sense) is seen to provide a powerful tool-set for the artist/activist to augment specific sites with a critical, context-specific data layer. Such situated interventions offer powerful new methods for the political activation of sites which enhance and strengthen traditional nonvirtual approaches and should be thought of as complementary to, rather than replacing, physical intervention. I offer as a case study this author’s NAMAland project, a mobile artwork which used open data and augmented reality to visualise and critique aspects of the Irish financial collapse.

Since its emergence as an art medium, Augmented Reality has developed as a number of evidential sites. As an extension of virtual media, it merges real-time pattern recognition with media, finally realizing the fantasies of William Gibson through goggles or handheld devices. This creates a welding of a form of perceptual vision and virtual reality, or optically registered simulation overlaid upon actual spatial environments. And even though AR-based works can be traced back into the late 1990s, much of this work required at least an intermediate understanding of coding and tethered imaging equipment from webcams to goggles. It is not until the advent of marker-based AR possessing lower entries to usage, as well as geolocational AR-based media through handheld devices and tablets that Augmented Reality as an art medium would begin to propagate. While one can make arguments that much AR-based art is a convergence between handheld device art and Virtual Reality, there are gestures that are specific to Augmented Reality that allows for its specificity as a genre. In this chapter, we will look at some historical examples of AR and critical issues of the AR-based gesture, such as compounding of the gaze, problematizing the retinal, and the representational issues of informatics overlays. This also generates four gestural vectors analogous to those defined in The Translation of Virtual Art (Lichty in The Oxford Handbook of Virtuality. Oxford University Press, Oxford, 2014), which we will examine through case studies. Through these case studies, historical and recent to the time of this publication, we may determine the issues of the gestures and aesthetics of AR.

Cinematic Apparatus theory of the 1970s set the stage for cinematic deconstruction in avant-garde film art. The material and production elements repressed in the normal ideological apparatus became the arena for new expression. As cinema through its acceleration of mechanization and sequence became the essential medium of its era, Augmented Reality accelerates the electric image and realizes a movement begun in video art and installation. This essay excavates and diagrams the AR apparatus to search out the repressed in viewers’ perception and point a way forward toward an avant-garde AR(t).

Augmented reality art, as a new media subset, distinguishes itself through its peculiar mechanics of exhibition and performative re-contextualization. It allows the artist to translocate the borders and constraints of the experience from physical to virtual, expressing the piece onto spaces independent of physical or locative constraint, yet still tethered to the real world. This practice of anchoring virtual assets to the physical world allows artists to make use of virtual properties such as mutability and replication, while engaging with issues of embodiment, performance, and presence. The art installation occurs not in the gallery, but on the hard drive of mobile devices. In this way AR artworks align themselves more perhaps with movements like net.art, where one must look to the loading screen as the gateway to the gallery, a space which—while mutable and infinitely configurable—is still proscriptive. AR may allow the artist to set many more of the work’s boundaries than in more traditional media, but even that freedom is still subject to the affordances of the software composing the work. Yet the ability to customize those boundaries, to draw one’s own curatorial borders and parameters, is in itself a freedom drawing from augmented reality’s strengths, inviting a model of the world as not one in which art happens, but one which is conditionally defined and experienced as an integrative work of art.

Google Glass brought a new level of excitement to Augmented Reality for the mainstream. It did not, however, bring the type of features that most AR enthusiasts have been looking forward to for decades. AR has traditionally been defined by the capability to integrate real-time 3D computer graphics into a person’s field of view in such a manner as to be convincing that they are as real as the physical objects surrounding them. Interestingly though, current AR technology has taken a turn away from this attempt at a sensorial suspension of disbelief in favor of a new social form of immersion. In this new model, space is collapsed not between the real and the virtual—but instead between people in distance and time. What was once a phone call is now a real-time mobile media stream of video, audio, text, hyperlinks, hashtags and sensor data. We can also quite simply capture our surroundings and post them as photographs, tweets, video and audio to cloud-based social media platforms whereby these are viewed by our social networks and then threaded, parsed and responded to as well as archived into spatial and temporal timelines. In context of this new mobile form of Augmented Reality that is based on social interactivity, artists are now beginning to explore the cultural potential this new medium can offer. This chapter will explore several components of this new artistic medium and some markers from art history and gaming culture that help to explain the history of how we have arrived at this new social AR medium. Specifically, we will look at socially immersive artworks and collaborative locative media as outcomes of this new medium based on social immersion rather than sensorial immersion.

Augmented reality technologies open up new techniques for art creation and simultaneously provide the conditions for new conceptual domains in which art can be created and experienced. This is particularly true in the case of digital dance performance, as well as the digital augmentation of images to create performative gallery experiences. This chapter undertakes an examination of the use of augmented reality in recent examples of digital performance and installation investigation at the Deakin Motion.Lab. In particular, we discuss the concept of ‘digital dualism’ as a means of mapping some of the conceptual shifts augmented reality makes possible for dance and performance technology. Digital dualism sees the disjuncture between ‘real’ and ‘virtual’ in digital performance, as in life, as an artefact of an earlier technological/cultural moment in which the digital had not yet become embedded within and a conduit for everyday life. We argue that digital performance within an augmented reality framework provides a demonstration of the inability of digital dualism to stand up even in relation to what might be considered the most unlikely candidate for digital distribution—the embodied experience of the human body. These works open up a dialogue around the ways in which augmented reality technologies enable a conceptual shift in digital performance and installation.

This chapter focuses on a use of Augmented Reality that is more closely related to traditional painting and sculpture than to interactive game-like AR installations. Based on an analysis of the author’s experimental paintings and sculptures, presented in his solo exhibition Hidden Realities and the outdoor installation The Enterprise Jigsaw, it deals with a particular type of Augmented Reality paintings that integrate gallery-quality art prints of digital paintings with augmentation by 2D and 3D objects. This type of painting can provide one easy and reliable solution to the acute problem of the saleability of Augmented Reality Art. Alongside theoretical considerations, the first-ever Augmented Reality painting for sale on Amazon is presented—the author’s artwork The Half Kiss. Similar possibilities for AR sculptures are also analysed.

This chapter looks at how the concept of Augmented Reality graffiti enables us to experience an expanded view of the urban environment. It examines how the intersection between graffiti, street art and AR provides us with a complex socially and technologically encoded interface, which has the potential to combine the first-hand experience of public space with digital media, and creative practices, in a hybrid composition. The chapter begins by looking at the tradition of graffiti and street art; this is followed by a discussion around the philosophical implications for digitally augmented graffiti. A number of key techniques and technologies are then explored through the use of two practice-based case studies.

In our investigations, here and elsewhere, art serves a neurological role, shaped by evolution: art, not as a thing that is, but as a function that occurs, which we will call Behavioral Art (BA). An important aspect of BA is “borrowing intelligence” from a humanly organized source, such as a painting, applied to a computer process. The resulting artifact of this auto-creative process might easily be mistaken for an objet de (computer) art. But we must look further into the larger dynamic system, one that includes the audience as well. As we will discuss, the machine itself is incapable of any type of organization. A human (often the programmer) must supply the organizational paradigm to the input, and a human must recognize one in the output. However, by integrating resources from the environment via machine, a process we can now call Augmented Reality. We might imbue whatever quality triggered an interpretation of “potentially meaningful” in audience members regarding the off-screen image, to our computed output. In this chapter, we address how and why humans tend to employ this subtle particular form of nonverbal expression.

This research looks at the way the aesthetics of object-oriented ontology performs in association with augmented reality art made on the borders of Internet connection. The focus of the research is on the notion of “wilderness onticology” by Levi Bryant, and the ideas of “hyperobjectivity” by Timothy Morton, while examining artworks by George Ahgupuk, Alvin Lucier, Mark Skwarek, Nathan Shafer, v1b3, and John Craig Freeman. Most of the conclusions of the research point to the praxis of the art historical anti-tradition as a tool for negotiating ontologies of the wilderness, or the unknown, as well as the virtual objects which exist there, for creating socially useful forms of art. Other topics include the usage of the Earth art binary of site/non-site, media ecology, and the flaneur.

The methodology and technique used to design and develop an Emotional Compass, a device for orientation in urban environments which uses geo-located content harvested from major social networks to create novel forms of urban navigation. The user-generated content harvested from social networks is processed in real time to capture emotional information as well as geo-location data and different types of additional meta-data. This information is then rendered on mobile screens under the form of a Compass interface, which can be used to understand the direction and locations in which specific emotions (or their combinations) have been expressed on social networks. This gives rise to achieve novel ways for experiencing the city, including peculiar forms of way-finding techniques which rely on emotions rather than street names and buildings. The resulting experience constitutes an interesting mix of Augmented Reality and Rhabdomancy.

This chapter proposes a new method for evoking the complexity of the Past from the archaeological record, based on a transdisciplinary approach linking archaeological science, art and IT technology. Inspired from the fractal theory, this method employs different levels of augmentations from general context to detail and uses a combination of Augmented Reality techniques and visual media, with a high artistic quality, to create a mixed-reality user experience. The chapter presents an experimental Augmented Reality application on mobile devices and discusses the efficacy of the method for an educational strategy to help communities recover and transmit their immaterial heritage to future generations. The research was based in Vădastra village, southern Romania, in an archaeological complex of a prehistoric settlement.

The urge to create art that escapes the frame and ventures into the world is a long-standing goal of artists which predates modern technologies. In honor of this I examine “Apocalypse,” an essay on the possibilities of street art that William S. Burroughs wrote as a collaboration with Keith Haring (pop and graffiti artist, social activist 1958–1990). While written in 1988, this essay can serve as a guide for current and future artists who work in Augmented Reality interventions into public spaces by situating their work within a 2000+-year-old cycle of revolution and counterrevolution in art, culture, and spirituality. My contention is that looking backward to pre-modern mythology in this way provides larger frames of reference that are even more useful to contemporary Augmented Reality artists than they were to the graffiti artists of the 1980s that this essay was originally discussing, as the technological and artistic affordances of mobile devices have expanded the possibilities of street art to begin to match Burroughs’ vision.

The most common way to consume art is through observation and acknowledgement of its existence. From the viewpoint of preserving art and cultural heritage, such passive consumption seems adequate. From the viewpoint of preserving art and cultural heritage, such passive consumption seems adequate. Yet, passive consumptions hinders users’ potential for exploration, creation and expression by not allowing to build upon existing artworks. Novel technologies could change this and augmented reality (AR) is one of the most promising by offering the possibility of mixing physical artworks with digitally augmented users’ creations and/or curation of personalised exhibitions. However, novel technologies could change this and augmented reality (AR) is one of the most promising by offering the possibility of mixing physical artworks with digitally augmented users’ creations and/or curation of personalised exhibitions. In a similar way that web enabled users to become active participants in, for example, commenting, sharing views, rating and deciding on the course of television shows in real time, AR could act as a medium to leave digital augmentation of artworks in real physical spaces. In this chapter, several AR ideas and solutions are presented with a common theme: each allows users to engage with art or cultural heritage in different ways. The chapter finishes with a presentation of user engagement continuum based on how AR solutions support engagement with artwork consumption and creation, and concludes with implications such AR solution would present.

The AR user experience is currently shifting from passive viewing to active engagement. Augmented reality wearables are going to be as common as smartphones are today. In a 24/7 augmented reality world, we risk becoming empowered and controlled by efficiency-focused algorithms. How to be your own robot is going to be the main challenge. In the near future, there is not just an opportunity for art to manifest itself in new ways, but there is also an urgency for that to happen. This chapter describes this shift with references to some of author’s own projects to illustrate the various ways in which AR art can manifest itself.

Experimental art deployed in the AR medium is contributing to a distinctly twenty-first-century reconfiguration of traditional perceptions of art, audience participation, and technological experience. This chapter examines an influential selection of experimental mobile augmented reality Art [ARt] in order to explore the progressive conceptual and ethical threads that are emerging from this relatively new but powerful cultural form. Using the concept of the ‘software assemblage’, the author traces the movement of AR beyond its native root system in the industrial, entertainment, and the engineering worlds, and toward the rhizome of radical practice that has come to define mobile ARt. A number of artists, critical engineers, theorists, historians, and participants to AR experiences, have in recent years been contributing to the emergent field of mobile ARt, and significant advances have been made. Clearly, this book is one of them. In the context of the second edition, the author posits the software assemblage concept as an alternative and relational modality through which to converse with ARt.

This chapter traces the evolution of the Riot Grrls App, a proposition applying the inherent possibilities of image-based augmented technology to an historical exhibition of paintings by the Riot Grrrls, a 1990s feminist punk movement, at the Museum of Contemporary Art, Chicago. The intention was to exploit the structural necessities of augmented reality, by conceptually and visually layering related references in real time, to both poetic and pedagogical ends. To do this, a School of the Art Institute professor and an art historian with expertise in user-experience worked as a team to lead a School of the Art Institute class of young students to create augmented art works using the historic paintings as both augmented triggers but also artistic material. They created inventive formal solutions that engaged museum goers intellectually and esthetically and were intentionally open-ended.