Developing a Ludonarrative Engine for a Historical Site Using Locative AR and Music: Psychogeography for Battle of the Boyne

Psychogeography is a term invented by Guy Debord in 1955 and used to describe how different places make us feel and behave. The idea saw a renaissance in the 1990s when artists, such as Iain Sinclair and Patrick Keiller, began creating works focused on the experience of exploring locations by walking, particular in urban and suburban environments. In 2021, Haunted Planet Studios combined psychogeography with location-based Augmented Reality in a novel way in the app Jack B. Yeats: Psychogeography AR,¹ in which the player could view the art of the famous Irish symbolist painter from the location where he physically painted it. For example, the artwork “The Liffey Swim” could be seen as an AR image through the screen of a smartphone layered on top of the modern buildings and the river Liffey from Bachelor’s Walk in Dublin [1].

Psychogeography was the premier inspiration for the design presented in this paper, in the specific context of a cultural heritage site: How does the place make us feel? How do we react to the events of the history from the perspective of time? What did people feel back at the time of the Battle? What happens now? We interpreted location as a monument of history itself and took the approach of putting music and audio front and centre to develop a mental vision of layers of time, instead of the typical visual design and graphics. Historical narrative, music narrative and spatial audio navigation are closely intertwined with game mechanics, which we have captured in a unique “ludonarrative engine” designed for experiences that emphasize the significance of the landscape as a silent witness of time. Nature characters are important components of both storytelling and game mechanics. Instead of modelling the battle as a classic game of opposing teams, we focused on aesthetics and exploration, to give additional information and experience to what the local museum already provides – history. We reviewed the research already done on historical sites, such as tour guides’ role perception [2] and Augmented Reality for historical interpretation [3, 4] and arrived at the music-led ludonarrative engine, where the narrative and historical fiction can help us comprehend heritage and historical events [5, 6]. Our character dialogues are written from first-person point of view, speaking directly to the player in the genre of documentary fiction [7, 8].

Being anchored in psychogeography, emotion is of course important for our narrative game design. Katherine Isbister notes: “Layering techniques for evoking social emotions onto this foundation gives games their unique power to create empathy and connection” [9]. Music narrative [10] is a powerful source for emotional engagement, also our nature characters are companions because of their presence in all time layers, are highly lovable, empathic and easy to connect [11, p. 300]. We wanted to add societal values to our game to have an impact on music education in Ireland, with dissemination through primary and secondary schools on the Music Generation program.² Through the exploratory nature of our AR game [12], we wanted to draw people’s attention to the landscape and its psychogeography, to think metaphysically across times and events and respect the nature in this fast climate-changing world.

The experience is audio-first to ensure a feeling of presence in the physical site while not compromising immersion into the story content by visual navigation. (The smartphone remains in the pocket of the player.) The following considerations have been made:

To connect the site’s past(s) and present, we created three time layers: Neolithic/Celtic Mythology (3000 BCE), Battle of the Boyne (1690) and Modern Ireland (2025), each containing period-specific points of interest (POIs), ambient soundscapes, music, and historical content. Time travel is facilitated through special POIs with portal functionality, designated as either Forward or Backward portals, scattered across the site. The nature characters Fox and Raven accommodate the Time Travel game mechanics.

The system uses a centralized TimeLayerManager that coordinates transitions between historical periods through an event-driven architecture. Each time layer contains its own collection of POIs, ambient audio events, and contextual content, ensuring complete historical authenticity for each period. The audio implementation employs a single FMOD ambient event with a discrete TimeLayer parameter that seamlessly switches between period-specific soundscapes without interruption (Fig. 2).

Full size image

The game provides a spectator mode designed to offer the experience for less mobile family members. For each game session, a unique code is created that can be shared with other players and when entered on another device, it grants access to the same game session, allowing the spectator to follow along with the player’s progress in real-time. The spectator sees where the player is and has access to the ‘treasure chest,’ which holds music, artefacts and (historical) information about the characters the player has already encountered. Game state and progress are shared through Firebase, ensuring that all connected devices are synchronized. The choice of Firebase was motivated by its real-time synchronization capabilities, simplified implementation and easy integration with Unity.

King William III was assigned the theme of “Yorkshire Feast Song” (1689) composed for coronation of William Orange and Mary by English composer Henry Purcell (1659–1695), who was commissioned to produce ceremonial music for coronation of royalties and social events. Identifying instrumentation of the character in Logic Pro: Vienna orchestra mix. (See Fig. 3.) Raymond Gillespie writes:

The Dutch wars of the 1670s were a defining moment in the life of William of Orange, for they gave him a profound hatred of King Louis XIV and of France. (…) William found himself in a league with a wide range of anti-French powers, including the papacy, which viewed Louis XIV’s ambitions as dangerous for European stability. Such alliances explain why many Catholic European powers, not least the papacy, welcomed the defeat of Louis’s ally, King James, at the Battle of the Boyne. [14, pp. 291–292]

King James II is represented in the game by march of John Playford (1623–1686), who was specialising on military marches of England of that time, and the piece is specifically named: “James the Second & Prince Ruperts March” arr. Hans Jorgen Messerschmidt (IMSLP library). Identifying instrumentation in Logic Pro: trumpets. (See Fig. 4.) Gillespie writes: “By 1689 James’s relationship with his Protestant subjects was disintegrating rapidly. (…) The Jacobite parliament with its Act of Attainder and restructuring of the land settlement speeded up the disengagement between James and what Protestant support he had left” [14, p. 291].

Full size image

The Sun King Louis XIV has a theme of Menuet (3/4) and Rigaudon (in C-major (1687) by Jean-Baptiste Lully (1632–1687), the French Court composer and dancer, who wrote many dances for Versailles entertainment and is considered a ‘father of French opera.’ ID instrumentation in Logic Pro: Harpsichord.

Full size image

The nature characters (the Fox and Raven) carry emotional charge that capture the symbolics of the character, but they also play a role in in the gameplay as portals through time. When the player meets the Fox, it brings them one time-layer forward whereas the Raven brings the player one time-layer back. Transition between time-layers have a signal specific sounds.

Ambient sounds of the Battle employs clunks of metal swords, cannons, horses, soldiers screams etc., according to the historical sources: “It was within this context that William arrived at Carrickfergus, County Antrim, on 14 June 1690 with 15,000 men, a train of artillery, a printer and a war-chest of £200, 000” [14, p. 292].

Other game characters, such as Celtic Child, Farmer, Village Woman, Barber Surgeon, Grenadiers, French Soldier, Jacobite Patrick Sarsfield, tell their stories to give different perspectives on the events of the past. In his memoirs, King James II recorded what happened:

‘Just as they were beginning to move, Sarsfield and Maxwell, who had been to view the ground betwixt them that ran along the small valley that divided the two armies.’ Sarsfield’s reconnaissance was correct. Schomberg’s men had already marched into the bog from the other side. It proved to be so swampy that they were forced to pull back. [15, p. 108]

The modern time layer of Irish heritage is represented by music of contemporary composers both from Ireland and Northern Ireland, such as Dublin-based contemporary composer (DCU) John Buckley, “The Golden Apples of the Sun, the Silver Apples of the Moon” on a poem by W. B. Yeats “The Song of Wandering Aengus,” as well as Northern Ireland composer Ian Wilson, “A Haunted Heart,” Elaine Agnew, “The child within,” and others.

To encourage engagement into the story and the time travel mechanic, we avoided formal commentary and adopted dramatised character dialogues written with emotional charge from a first-person point of view. For example, the Battle Oak (now):

“I was already tall on the day the battle swarmed around me. I saw the men fight, and I saw the men fall: Irish men, English men, Dutch men, French men. They fought and they fell. And the crows fed for days. I hold it all. My roots drank their blood, so they are all a part of me now. I cannot escape. I must remember. I remember the day of the battle: a boy tied a little white ribbon to one of my branches. He touched my trunk and prayed. I felt the fear in his touch, I can still feel it now. He never came back. I remember many days before the battle, and many after. I remember the harshness of every winter. I remember the famines. I remember trees that stood, trees that fell. And now, I am the last of the old ones standing.”

The application primarily implements two distinct audio modes that create a comprehensive player experience: Wander Mode and Interact Mode. (See Fig. 5 (choice of character X or Y, and Fig. 6 (audio zones).

To create a convincing spatial audio experience, a sophisticated integration of sensor data is required. Our system combines GPS positioning, device orientation tracking, and audio spatialization to create a cohesive auditory. The spatial audio implementation relies on advanced sensor fusion techniques combining multiple data sources.