Technologies for Interactive Digital Storytelling and Entertainment

Interactive Narrative is an approach to interactive entertainment that enables the player to make decisions that directly affect the direction and/or outcome of the narrative experience being delivered by the computer system. Interactive narrative requires two seemingly conflicting requirements: coherent narrative and user agency. We present an interactive narrative system that uses a combination of narrative control and autonomous believable character agents to augment a story world simulation in which the user has a high degree of agency with narrative plot control. A drama manager called the Automated Story Director gives plot-based guidance to believable agents. The believable agents are endowed with the autonomy necessary to carry out directives in the most believable fashion possible. Agents also handle interaction with the user. When the user performs actions that change the world in such a way that the Automated Story Director can no longer drive the intended narrative forward, it is able to adapt the plot to incorporate the user’s changes and still achieve dramatic goals.

In this paper, we present an extensible framework for interactive multimodal contents, with emphasis on augmented reality applications. The proposed framework, EDSUN, enables concurrent and variable narrative structures as well as content reusability and dynamic yet natural experience generation. EDSUN’s main components include a canonical specification of 5-state lexical syntax and grammar, stochastic state transitions, and extensions for hierarchical grammars to represent complex behavioral and multimodal interactions. The benefits of EDSUN in enabling classical contents to support the affordances of AR environments and in complementing recent published works are also discussed.

Context-aware systems are conceived for diminishing the cognitive load of users that perform tasks such as retrieving information or accessing services. A wide range of applications is available, with emphasis on tourism, cultural heritage and e-commerce. This work explores the possibility of using an agents based context-aware architecture for controlling the evolution of a story on the basis of different types of context, including the user profile, the location, the user history and time. In order to prove the suitability of such architecture to the domain of storytelling, a real novel by a famous Italian writer was rewritten in the form of an interactive 3D world where users play the roles of the different characters of the story.

Hypervideo is one of several possible ways for interactive non-linear media. In its characteristics it is closely related to the purposes of digital narratives. The techniques of hypervideo could be used for the presentation of digitally told stories and vice versa. Many projects in both fields of work show the impressive possibilities each. But there seems to be a lack of using narrative intelligence in hypervideo. This paper shows how both fields of work could benefit from each other. Therefore two projects are introduced and their combination is discussed. The hypervideo environment HyPE includes an authoring tool, a stand-alone player for hypervideos and offers an API for the use in higher-ranking applications. Secondly the storytelling system called Jeherazade is introduced, which bases on the idea to enhance the classical theory of Aristotle to the new form of digital narrative. It is primarily developed for the use in presentations in distributed exhibitions but it is planned to be used for e-leraning as well.

In this paper we present our continuing efforts to generate narrative using a character-centric approach. In particular we discuss the advantages of explicitly representing the emergent event sequence in order to be able to exert influence on it and generate stories that ‘retell’ the emergent narrative. Based on a narrative distinction between fabula, plot and presentation, we make a first step by presenting a model based on story comprehension that can capture the fabula, and show how it can be used for the automatic creation of stories.

Mindstage

is a real-time multi-user 3D virtual environment used to explore the relationships between a linear story and the virtual world in which it unfolds. The prototype uses as its narrative spine an illustrated lecture on film design by Christopher Hobbs. It provides a stage for interaction featuring a customized 3D environment based on this material, with the necessary actors and objects in it. The main design issues were mapping the linear talk onto the virtual space, and the implementation of various interactive features within it. We argue that a careful use of spatial design supports a degree of non-linear story-telling without compromising the core linear content.

Most research in Interactive Storytelling (IS) has sought inspiration in narrative theories issued from contemporary narratology to either identify fundamental concepts or derive formalisms for their implementation. In the former case, the theoretical approach gives raise to empirical solutions, while the latter develops Interactive Storytelling as some form of “computational narratology”, modelled on computational linguistics. In this paper, we review the most frequently cited theories from the perspective of IS research. We discuss in particular the extent to which they can actually inspire IS technologies and highlight key issues for the effective use of narratology in IS.

After offering reasons why film can offer insights into computer entertainment and reviewing the concept of ‘schemas’, the results of a qualitative investigation of viewer’s talk about a short film, Ice Cream Dream, are reported, related to genre, character relationships, use of props, schematic sequences, locations and sound. This material is extended and deepened by drawing on an ongoing cognitive analysis of another short film, The Wrong Trousers.

This paper is a review of a story generation system called Minstrel. It uses complex but hand-crafted Lisp knowledge structures to generate short computer-generated stories within the King Arthur domain. The knowledge representation model of Minstrel is reimplemented using a W3C standard language to analyze the pros and cons of technology updates over this kind of classic AI projects.

A generic structure for media-oriented modular storytelling is presented. The concept is based on a simple container and content model. We also show how various paths along a continuous storyline and branching threads can be organized. An initial implementation is compared with a second, more general and flexible structuring approach.

Mythology and its general relevance for popular culture is is a framework of growing importance for understanding the way narrative games function as cultural artefacts within society. Myths are better compatible than conventional stories with the key characteristics of games: interactivity, (world)simulation and gameplay. Furthermore, games as a technologically advanced medium, open up new mythological perspectives on contemporary society and technology, a perspective where the hacker is proposed as the new hero of this day and age.

Today the concept of art is studied in the term of aesthetic object. On the one hand, it is noticed that the aesthetic object has changed its form from material into interpretive and semantic entity. On the other hand, the aesthetic experience is shifting its site to the center of aesthetic object. Contemporary computer-based media art works are composed of physical bodies such as computer CPU, monitor, projector, camera and speaker including technologies of computer vision, computer graphics, virtual reality, artificial intelligence, augmented reality, etc. In particular, interactive art not only embodies characteristics of already changed aesthetic object, but also reveals contexts of returning the problem of aesthetic object into the aesthetic perception for the reconsideration. Therefore interactive art is the triggering and motivating point to discuss new aesthetic object in terms of art and science.

Storytelling models are usually constrained to the applications they are implemented in because of the particular characteristics of the data used to define story events and the way those events are linked. In order to develop a more generic model to create storytelling applications, we need to focus the solution not on the data itself, but on the manner this data, in the form of events, is organized and conveyed to the user. In this paper, we present SRST (Storytelling RST), our proposal for a generic storytelling ontology model based on the organization of events using the relations proposed by the Rhetorical Structure Theory (RST) and how narrative principles are applied to these RST relations to generate coherent stories.

Creating an interactive drama requires authors to produce large quantities of story content. A programmer or knowledge expert typically creates this content because they have experience with the story environment. By using an authoring tool someone with less experience with the story environment can organize and create story content. The Scribe Authoring Tool is an authoring tool that will be used to create interactive dramas. The tool will follow certain requirements to make it relevant to any story environment and will be usable enough for someone not familiar with the environment to author story content for interactive dramas.

Recent progress in the area of sensor technology has enabled the development of context-aware systems that are able to dynamically adapt their behaviours to the current situation and the individual user. In this paper, we present a framework for a new generation of context-sensitive stories that dynamically adapt to changing environmental conditions and user states. The framework combines approaches to interactive storytelling with work on context toolkits that foster the rapid prototyping of context-aware applications.

The U-CREATE project aims at efficient content creation for new technologies, in particular interactive setups, Mixed Reality experiences and location-based services. A graphical authoring tool is developed to allow one to create elaborated contents in a fast and easy way. The present paper introduces the tool and elaborates on its main components, which allow the user to author an interactive story and also to manage the connected interaction devices.

This contribution presents and discusses

Scenejo

as an experimental platform for Interactive Digital Storytelling, focusing on the authoring process as initial viewpoint for its development. Special emphasis is on the construction of conversational threads for virtual actors using pattern matching, employing transition graph representations as the main interface for authoring. In the conclusion, the opportunities and challenges of graph structures are discussed.

For 3D story composition, interactive controls such as scene composition, character actions, and camera placement are important processes. In this paper, a Mixed Reality based interactive tool for 3D story composition is introduced. Using the proposed tool, non-experienced users composed 3D scenes through interfaces in his/her real environments. Preliminary studies showed that proposed 3D story composition tool was convenient and useful by interactively updating scenes through user’s control over the stage items and camera viewpoints.

We report on an effort to ease authoring of XML-based knowledge sources for interactive Virtual Reality (VR) environments by generating them semi-automatically from a common ontology. The generation process is done using XSL Transformations. The automatic generation of knowledge sources ensures consistency and completeness of the modeled entities, by avoiding typical errors that result from hand-crafting knowledge sources for big domains.

Over the last several years, conferences in the field of interactive entertainment have showcased numerous presentations concerning concepts and technology for Interactive Digital Storytelling: runtime systems, such as story engines, intelligent / autonomous agents, drama managers and conversational systems. They focus on solutions to the problem of combining dramatic storytelling with user interactivity. In contrast to the many technical contributions concerning runtime performance, few discussions have been initiated about the question of how a new breed of “interactive storytellers” would create concrete original artefacts with the proposed systems. In our opinion, the authoring process is likely to be a serious bottleneck for generating innovative products in the future.

Interactive Narrative is an approach to interactive entertainment that enables the player to make decisions that directly affect the direction and/or outcome of the narrative experience being delivered by the computer system. One common interactive narrative technique is to use a drama manager to achieve a specific narrative experience. To achieve character believability, autonomous character agents can be used in conjunction with drama management. In this paper, we describe the problem of

failing believably

in which character believability and drama management come into conflict and character agents must intelligently produce behaviors that explain away schizophrenic behavior. We describe technologies for implementing semi-autonomous believable agents that can fail believably.

In order for interactive agents to be believable, they will need to respond to any likely situation in a manner that is consistent with their personality, as well as their position within social hierarchies. Thus believable agents will need to have a clearly defined personality, social role, and other traits that will govern their actions in a virtual world. The goal of this paper is to present a template that can be used to define such traits of a character in order to maintain consistency. The template will be dominated by a model that defines aspects of personality typically used to define persons across cultures, aiding both intuitive creation by authors, and acceptance by users. It will also be able to take advantage of character stereotypes to ease the authoring process. In addition to this, a social hierarchy framework is given.

Human emotions are known to play an important role in the users’ engagement, namely by activating their attention, perception and memory skills, which in turn will help to understand the story – and hopefully perceive, or rather “feel” it as an entertaining experience. Despite the more and more realistic and immersive use of 3D computer graphics, multi-channel sound and sophisticated input devices – mainly forced by game applications – the emotional participation of users still seems a weak point in most interactive games and narrative systems. This paper describes methods and concepts on how to bring emotional experiencing and emotional expression into interactive storytelling systems. In particular, the Emotional Wizard is introduced, as an emerging module for authoring emotional expression and experiencing. Within the INSCAPE framework, this module is meant to improve elicited emotions as elements of style, which are used deliberately by an author within an integrated storytelling environment.

We describe extensions to VirtualHuman, a multimodal dialogue system. It uses an interactive game scenario to demonstrate real-time multi-party dialogue between two human users and three virtual characters. The focus is to make the interaction more natural, robust and flexible. Here, we address issues of speech recognition in noisy environments, resolution of spatial references, and enhancements in the character interactions.

Believability is a basic requirement for non-player characters of videogames. Players enjoy characters with personalities that reflect human behavior, specially if those personalities combine well with players’ temperaments. This paper explains a model for customizing automatically non-player characters (NPC) according to the players temperament, which is obtained before the game session. The model uses Case-Based Reasoning and Ontologies to adapt the behavior of a NPC, which is the companion of a player character in the described example.

Personalizing the playing experience is a key factor in making players of computer games feel involved in the virtual world; however, current Massively Multiplayer Online Role Playing Games only to a limited degree allocate development or running resources towards facilitating a personalized experience. In Pen and Paper Role Playing Games, the player-controlled characters form a key component in facilitating the formation of a personalized experience. In these games, characters are often more than the association of stats and skills popular in online games, and several approaches towards utilizing the character-based information directly to personalize the game playing have been developed, e.g. personality systems. Some of these systems can be integrated into online games, providing a realistic and financially feasible method for improving the ability of these games to personalize the experience to the individual players.

This paper proposes and outlines a novel method for designing multiplayer computer role-playing games that revolve around procedural interactive narratives, which are further developed by the participating players. The proposed Plot Cluster system is evaluated practically during the design and construction of the experimental game

Castle of Oulu 1651

. The functionality and applicability of the approach is further validated with the empirical field experiment with 260 test subjects. Plot clusters enable the game designers to create controlled but re-playable emergent storylines that intertwine the participating players using multi-tier network structures.

The Pen-and-Paper role-playing game is a successful example of collaborative interactive narrative. Meanwhile, computer-based role-playing games, while structurally similar, offer quite different narrative experiences. Here results are presented of an experimental study of role-playing gamers in Pen-and-Paper and computer-supported settings. Communication patterns are shown to vary significantly on measures such as the share of in-character statements and the share of dramatically motivated statements. These results are discussed in the light of differences between the two gaming forms and finally some design implications are discussed.

A study on how role-play through narrative elements of social communication can be implemented in a computer-based environment. Contemporary computer role-playing games do not support playing roles in a social aspect. Play is conducted in a functional and task-oriented way due to the design of the gameplay. An experimental role-playing game,

Castle of Oulu, 1651

was constructed to test aspects of social play and elements borrowed from more traditional forms of role-play. The aim is to determine some essential elements the players need to assume a role and play towards socially structured goals.

MMORPGs are an increasingly popular form of entertainment, yet are limited in their ability to tell stories when compared to other media. This paper analyses some of the underlying reasons for this inability, using techniques form narrative analysis. One of the basic problems identified is that the design of MMORPGs inhibits the use of techniques used in other media to create engaging stories by manipulating the presentation of time. The other issue identified is the problems MMORPGs experience in presenting stories with meaningful consequence. A means to a possible solution to these problems, in separating the personal player view point from that of the overall world view, is discussed.

‘Gulliver’s World’ is a Multi User Mixed Reality Environment that is part of the Ars Electronica Center Exhibition in Linz/Austria. In a public laboratory situation, the installation functions simultaneously as interactive edutainment platform as well as flexible infrastructure for the expansion of mixed reality environments via innovations in information and communication technology and media art. The installation is characterized by a nonlinear exhibition concept at the nexus of theater, digital film production and game development. At seven workstations, people of all age groups range along the reality-virtuality continuum while collaboratively creating 3D-worlds. As research project, ‘Gulliver’s World’ features multilevel infrastructure with exemplary content: The latest insights and models to emerge from HCI research, concepts of mixed reality and virtual environments and their supporting technology are brought together and developed further. They are assembled on a multimedial platform that enables scientific development to dovetail with an educational application that is demonstrated in this paper.

In this paper we present an approach that examines if storytelling techniques can be used to relate navigation instructions and if such a presentation improves retention of instructions. We describe the concept of a system to present geographical information as a sequence of story elements; present an initial prototype and the design of an experiment to evaluate this approach.

The recent past has seen a proliferation in the amount of digital video content being created and consumed. This is perhaps being driven by the increase in audiovisual quality, as well as the ease with which production, reproduction and consumption is now possible. The widespread use of digital video, as opposed its analogue counterpart, has opened up a plethora of previously impossible applications. This paper builds upon previous work that analysed digital video, namely movies, in order to facilitate presentation in an easily navigable manner. A film browsing interface, termed the MovieBrowser, is described, which allows users to easily locate specific portions of movies, as well as to obtain an understanding of the filming being perused. A number of experiments which assess the system’s performance are also presented.

The Eastern edge of Nottingham (UK) city centre is undergoing substantial restructuring, which will have a major impact on the people living in the area. Future Garden is an interactive story, delivered on a handheld PDA device, which explores the past, present and possible futures of Sneinton Market, one part of the area to be re-developed. This paper introduces Future Garden including its novel navigation interface based on video and self-reporting and some early results of the still ongoing evaluation.

Today’s popular presentations of immersive digital narrative experiences commonly utilise wide screens or forms of panoramic projections to display visually realistic content. The shift of focus to other media forms that includes non-realistic and aesthetic expression using audio and small displays is a challenging step. This paper describes an interactive narrative content and design approach that explores small displays and wide perceptual audio projections to achieve an immersive experience. A mixed media content known as Autophobia was implemented as a proof of concept. Autophobia uniquely presents film, radio, and photomontage to achieve an alternate reality experience.

The authoring and the design of immersive, non-linear plots remains one of the main challenges in interactive digital storytelling. This paper introduces the concept of

interactive audiobooks

, which combines the potential of complex (non-)linear narratives (e.g. books and radio plays) with interactive elements from computer games. The design concentrates on a flexible degree of interaction, in a way that the listener’s experience ranges between a passive listening to an interactive audio-only computer game. In this paper we discuss the story-engine used in interactive audiobooks, as well as present an authoring framework along several design guidelines to create them. Finally, we demonstrate the capabilities of our system with an adaptation of a short story from Edgar Allen Poe.

“CitizenTalk”, an applied research project at FH Erfurt, investigates the potential and challenges of interactive communication using chatbots as an innovative tool for involving citizens in public planning processes. This comprises research into the state of the art in virtual planning communication, as well as the technical possibilities of chatbot communication. We present the opportunities and limitations associated with the adoption of chatbot concepts and show future prospects offered by storytelling mechanisms. An authoring tool is presented together with a new scheme to control the communication process better than current chatbot technologies and approaches allow.