Interactivity, Game Creation, Design, Learning, and Innovation

In 2015 the EAI conference series steering board declared that the international conference ArtsIT, which began in 2009, was to be extended for its 5th iteration to reflect contemporary trends of increased activities in interactive and game-based arts and technology and be renamed as The International Conference ArtsIT, Interactivity and Game Creation. Hosting would take place in the new auditorium campus buildings of Aalborg University Esbjerg, Denmark where it was also successfully held in 2011.

Augmented Reality (AR) has been recognized as one of the most promising technologies for the gaming industry. In this study, the designers intend to apply AR technology to developing an educational interactive game. This paper presents an AR featured educational game specifically designed for 4–6 years old pre-school children. The main objective of the game was to teach children knowledge about color mix, mathematics and 2D-3D geometrical shapes. This game allows user to interact with both in-screen and physical objects at same time, and different interaction forms like the touch screen (click) and AR game (rotate) for better interaction with real world and learning. This paper focuses in detail on the design and interactive behavior. Furthermore, beyond the needs of children, this game also serves for parents. Through the Token Economy method, parents can control kids’ playing time, and track and modify their everyday behavior.

This paper presents the design, implementation and deployment of a new version of the popular farm game as deployed within an Ambient Intelligence (AmI) simulation space. Within this space, an augmented interactive table and a 3D avatar are employed to extend the purpose and objectives of the game, thus also expanding its applicability to the age group of preschool children from 3 to 6 years old. More importantly, through the environment, the game, which builds on knowledge stemming from the processes and theories used in Occupational Therapy and activity analysis, becomes capable of monitoring and following the progress of each young player, adapt accordingly and provide important information regarding the abilities and skills of the child and their development over time.

The School of Creative Arts & Technologies at Ulster University (Magee) has brought together the subject of computing with creative technologies, cinematic arts (film), drama, dance, music and design in terms of research and education. We propose here the development of a flagship computer software platform, SceneMaker, acting as a digital laboratory workbench for integrating and experimenting with the computer processing of new theories and methods in these multidisciplinary fields. We discuss the architecture of SceneMaker and relevant technologies for processing within its component modules. SceneMaker will enable the automated production of multimodal animated scenes from film and drama scripts or screenplays. SceneMaker will highlight affective or emotional content in digital storytelling with particular focus on character body posture, facial expressions, speech, non-speech audio, scene composition, timing, lighting, music and cinematography. Applications of SceneMaker include automated simulation of productions and education and training of actors, screenwriters and directors.

We present the design and evaluation of an interactive installation to be explored by movement and sound under Human-Activity Model. In the installation, movement qualities that are extracted from the motion tracking data excite a dynamical system (a synthetic flock of agents), which responds to the movement qualities and indirectly controls the visual and sonic feedback of the interface. In other words, the relationship between gesture and sound are mediated by synthetic swarms of light rays. A test session was conducted with eleven subjects, who were asked to investigate the installation and to fill out a questionnaire afterwards. In this paper, we report our preliminary work on the analysis of the tensions of interaction with the installation under the Human-Artifact Model. Our results indicate exploration and discovery as the main motives of the interaction. This is different than utilitarian HCI artifacts, where the instrumental aspects are typically in the foreground.

We compared a game-based experiment carried out in a lab study to crowdsourced set ups (both uninformed and informed). We investigated the device’s human resolution - the minimum size for dragging the finger onto a target on a touch screen. Participants in the lab consistently produced fewer errors than those from the crowd. For lab participants, errors significantly increased between targets of 4 mm and 2 mm in width. The uninformed crowd had too many errors to determine significant differences but the informed crowd yielded useful data and performance declined already for targets between 8 mm and 4 mm width. The smallest selectable target width for dragging for all three groups combined, was between 2 mm and 4 mm on mobile touch devices.

The research project discussed here, examines attempts to increase the motivation of elementary school students in basic science by the means of designing a science game. To realize this goal, the maze and mirror game was designed and a workshop was held based on the game which teaches the concepts of light and reflection during joyful group play. The game was initially designed for this research consisted of a maze pattern, mirrors and bases, a buzzer and a laser. The results of the game were evaluated by three types of questionnaires and showed improvement in all aspects. The questionnaires revealed that 91% of students liked the game very much and 81% favored having these kinds of games in their science classes. The main achievement of this game for students can be categorized into four main areas: Creating thinking and independent learning, playful learning atmosphere, a chance to learn in group activities, and a feeling of accomplishment.

In this paper we present an architecture for the study of telepresence, immersion and human-robot interaction. The architecture is built around a wearable interface that provides the human user with visual, audio and tactile feedback from a remote location. We have chosen to interface the system with the iCub humanoid robot, as it mimics many human sensory modalities, including vision (with gaze control) and tactile feedback, which offers a richly immersive experience for the human user. Our wearable interface allows human participants to observe and explore a remote location, while also being able to communicate verbally with others located in the remote environment. Our approach has been tested from a variety of distances, including university and business premises, and using wired, wireless and Internet based connections, using data compression to maintain the quality of the experience for the user. Initial testing has shown the wearable interface to be a robust system of immersive teleoperation, with a myriad of potential applications, particularly in social networking, gaming and entertainment.

With advances in information and communication technologies (ICT), dramatic changes have been produced in physiotherapy provision. However, low adoption of the developed technologies calls attention for better theoretical model and methods for ICT design, which may fulfill the needs of physiotherapists and their patients. In this work we discuss the framework of designing ICT for physiotherapy based on some results obtained from the perspectives of physiotherapists and patients on electronic health records for physiotherapy. We underscore the importance of considering the context - the conditions in social and physical environment as well as end-users internal conditions to be in place - for a specific physiotherapy process.

Mobile location-based games are experiences that entertain its players by requiring interactions mainly at points of interest (POIs). Navigation between POIs often involve the use of either a physical or digital map, not taking advantage of the opportunity available to engage users in activities between POIs. The present paper examines, how riddle solving as a navigational method affects enjoyability, flow, and spatial presence.

Bilingual ethnic Chinese parents are concerned about their preschoolers’ learning of their mother tongue. Many allow their children to learn Mandarin by accessing Mandarin language applications on mobile devices. However the effectiveness of solely using mobile devices as a learning tool for preschoolers is debatable. This paper presents a field investigation on how adult-facilitated craft-making promotes interaction between the adults and their children, generates greater interest the reading of Mandarin eBooks and retention of the stories. The data suggests pairing of activities may be useful to children of across language abilities. This also highlights a need for designers and educators to formulate a holistic design approach in the development of preschool mobile learning content.

This study presents that the computer game is being philosophized as an object of thoughts that generates a number of philosophical discourses. In order to define the concept of philosophizing computer game, my paper examines Racing Car Game with BCI (Brain-Computer Interaction) and reconsiders the rule of the game in the conflict between the game and the narrative. It proposes that the philosophizing computer game with BCI contributes to the decolonizing knowledge enabling new forms of collaborations between sciences, engineering, arts, and design.

This paper introduces the CollaTrEx framework for collaborative context-aware mobile exploration and training. It is particularly designed for the in-situ collaboration within groups of learners performing together diverse educational activities to explore their environment in a fun and intuitive way.Aside from employing both absolute and relative spatio-temporal context for determining the available activities, different buffering levels are an important conceptual feature supporting seamless collaboration in spite of temporary connection losses or when in remote areas.CollaTrEx comprises a prototypical front-end implementation for tablet devices, as well as a web-based back-end solution for the creation and management of activities which can be easily extended to accommodate both future technologies and novel activity types.

This study investigates how geometric field of view, motion blur and camera altitude can be utilized in 3D third-person racing games in order to increase the perceived velocity. Related studies have concluded that geometric field of view can be used to increase the perceived velocity and, based on subjective measurements, that motion blur has no effect on the perceived speed. This research objectively measures these effects along with the effect of different camera altitudes. The results show that increasing the geometric field of view significantly increases the perceived velocity. They also show that a strong setting of motion blur decreases the perceived velocity. Moreover, the results show that higher altitudes at high velocities increase the perceived speed.

This paper considers the pros and cons of stand-alone displays, analog (e.g. billboards, blackboards, whiteboards, large pieces of paper etc.) as well as digital (e.g. large shared screens, digital whiteboards or similar), as tools for time management processes in a creativity-driven learning environment. A qualitative study was conducted at the Medialogy education at Aalborg University Esbjerg to probe for attitudes towards time management tools in general and towards having these on a stand-alone display in the workspace of students working in groups on joint projects. Results show that students who use a stand-alone display primarily use an analog one, whereas students who use digital collaborative time management tools prefer to access these on their individual laptops. The students who express preference for analog displays emphasize the advantage of being able to switch to a different modality for in-group discussions, as well as the increased awareness of the time plan caused by such a display. While these advantages would also be advantages for a digital, interactive display, the disadvantages of analog displays listed by the students relate to the fragility of these displays compared to storing information digitally. The findings could indicate a possible market for stand-alone, interactive digital displays combining the ‘touch and feel’ character of an analog board with the convenience of digital data storage.

From the study of social-interaction enhanced gaming design, aimed at providing a public environment which supports tangible & social interactions among children, we designed Sandtime. Sandtime is a public installation designed to encourage such interaction. Using the Tangible Interaction Design approach, this gaming installation features collaborative play and social interactions under public context, where children can collaboratively interact with the virtual in-screen characters by manipulating physical objects. This design is based on the study of how interactive gaming facilities can help to ease anxiety and enhance social interactions among children. In this paper, we want to continue this line of research by exploring further the elements that can enhance such interaction experience. This paper focuses specifically on sensory play and how it can help to facilitate social interaction.

Smart IoT technologies set a milestone in supporting new enjoyment models for Art and Cultural Heritage, providing amazing technological experiences. However, users, while interacting almost purely by mediation of smart devices or augmented VR displays, practically keep themselves out from living the fullness of the surrounding cultural sites experience, establishing no direct dialogues or interactions with artworks. It sounds quite like to be in the living-room, looking at very appealing documentaries, equipped with exciting smart technologies. This paper focuses on the importance of “re-humanize” art recreation models, proposing a human-like interaction driven approach. Holographic projections, reproducing human or fantasy characters, play the human presence imitation game, when users are detected close to any artwork, interacting and dialoguing with them in natural language. An IoT infrastructure, an NLP platform and a Holographic Projection Engine implement a system for supporting holographic projections. Preliminary experiments were promising, thus motivating authors to further investigations.

Two workshop case studies are described that document the potential for applying novel approaches to the use of technology in multisensory environments. In contrast to current trends, the environments were regarded as a space within which to work rather than as a given set of technologies. Stimulating interactive story-worlds were enabled for groups of children with mixed Special Educational Needs where technology could empower the different groups to affect the environment as a whole. Arts-based leadership experience was regarded as key within the studies as were methods for moving beyond passive modes of interaction with sound and music. A novel approach to creating interactive ‘soundtracks’ is described that takes adaptive techniques from computer game audio and applies these within a physical space.

Nowadays, game developers are much more focused on providing players with short-term rewards for overcoming challenges than they have been previously. This has resulted in a lot of games having more scaffolding to teach the players what to do, so they don’t quit the games in frustration of not knowing what to do.This paper investigates the effects that scaffolding in games has on players’ experience of a game. To this end, a custom game was designed and implemented that contained a number of different scenarios with different types of scaffolding. This was used to conduct an experiment on 18 participants, measuring their experience with the scenarios they were tasked with completing. It turned out that participants overall found the scenarios with subtler scaffolding more interesting than the ones with text based scaffolding or no scaffolding at all. Additionally, they felt better about completing the scenarios that did not make use of scaffolding.

We present a procedural method and an implementation of multi-Kinect skeleton fusion on Unity environment. Our method calibrates two Kinects by combining the relative coordinates of a user’s torso onto a single coordinate system. The method is tested with a small number of participants in scenarios involving multiple users, results indicate that the method provides an improvement over a single camera, and it is accurate enough for games and entertainment applications. The video demonstration of the system is provided, and future directions to improve accuracy are outlined.

Online recommender systems are useful for media asset management where they select the best content from a set of media assets. We are developing a recommender system called 360-MAM-Select for educational video content. 360-MAM-Select utilises sentiment analysis, emotion modeling and gamification techniques applied to people’s comments on videos, for the recommendation of media assets. Here, we discuss the architecture of 360-MAM-Select, including its sentiment analysis module, 360-MAM-Affect and gamification module, 360-Gamify. 360-MAM-Affect is implemented with the YouTube API [9], GATE [5] for natural language processing, EmoSenticNet [8] for identifying emotion words and RapidMiner [20] to count the average frequency of emotion words identified. 360-MAM-Affect is tested by tagging comments on the YouTube channels, Brit Lab/Head Squeeze [3], YouTube EDU [28], Sam Pepper [22] and MyTop100Videos [18] with EmoSenticNet [8] in order to identify emotional sentiment. Our results show that Sad, Surprise and Joy are the most frequent emotions across all the YouTube channel comments. Future work includes further implementation and testing of 360-MAM-Select deploying the Unifying Framework [25] and Emotion-Imbued Choice (EIC) model [13] within 360-MAM-Affect for emotion modelling, by collecting emotion feedback and sentiment from users when they interact with media content. Future work also includes implementation of the gamification module, 360-Gamify, in order to check its suitability for improving user participation with the Octalysis gamification framework [4].

This research intends to develop music games as mobile device applications on android system for head start experience in music. The study of design content includes the perception, knowledge formation, musical knowledge and ability, and children’s play and learning motivation. 8 mobile device applications across two levels have been created and 4 of the first game level are tested by 21 preoperational children. In the latter part of this qualitative research, researchers collect data from participants’ observation, video recording, and tablet input data records. The credibility and validation study consisted of two steps: analyzing and comparing 3 dimensions of attitude, interaction, and problem solving of collected data.

This paper investigates whether the illusion of internal state in passive tangible widgets is stronger when using one touchscreen device or two devices. Passive tangible widgets are an increasingly popular way to interact with tablet games. Since the production of passive widgets is usually cheaper than the production of widgets with internal state, it is much more cost-efficient to induce the illusion of internal state in passive widgets than to use tangible widgets with an actual internal state. An experiment was conducted where the participants’ belief in the illusion was determined by means of an interview with questions regarding the functionality of the tangible widgets. The results show that using two devices is significantly better at inducing the illusion of internal state.

Humans interact with each other by utilizing the five basic senses as input modalities, whereas sounds, gestures, facial expressions etc. are utilized as output modalities. Multimodal interaction is also used between humans and their surrounding environment, although enhanced with further senses such as equilibrioception and the sense of balance. Computer interfaces that are considered as a different environment that human can interact with, lack of input and output amalgamation in order to provide a close to natural interaction. Multimodal human-computer interaction has sought to provide alternative means of communication with an application, which will be more natural than the traditional “windows, icons, menus, pointer” (WIMP) style. Despite the great amount of devices in existence, most applications make use of a very limited set of modalities, most notably speech and touch. This paper describes a multimodal framework enabling deployment of a vast variety of modalities, tailored appropriately for use in blended learning environment.

Playing music is about conveying emotions and the lighting at a concert can help do that. However, new and unknown bands that play at smaller venues and bands that don’t have the budget to hire a dedicated light technician have to miss out on lighting that will help them to convey the emotions of what they play. In this paper it is investigated whether it is possible or not to develop an intelligent system that through a multimodal input detects the intended emotions of the played music and in real-time adjusts the lighting accordingly. A concept for such an intelligent lighting system is developed and described. Through existing research on music and emotion, as well as on musicians’ body movements related to the emotion they want to convey, a row of cues is defined. This includes amount, speed, fluency and regularity for the visual and level, tempo, articulation and timbre for the auditory. Using a microphone and a Kinect camera to detect such cues, the system is able to detect the intended emotion of what is being played. Specific lighting designs are then developed to support the specific emotions and the system is able to change between and alter the lighting design based on the incoming cues. The results suggest that the intelligent emotion-based lighting system has an advantage over a just beat synced lighting and it is concluded that there is reason to explore this idea further.

Artist’s sketchbooks may provide important insights into the genesis of the finished works and may also contain artworks that are at least as interesting and sometimes even more fascinating and fresh. However, sketchbooks are delicate and problematic exhibits; displaying them in a showcase leaves at most two pages visible, and allowing visitors to handle the books does not make sense. This paper describes an interactive, virtual sketchbook technology intended for the display of books which, at the same time, is faithful to the original book and provides an enhanced spatial experience, a gigantic pocketbook which you may seem to enter spatially and bodily. The installation has been shown at The Italian Culture Institute in Copenhagen (2011), two public libraries (2012–13), two Danish art museums (2014), and the Book Fair in Bella Center, Copenhagen (2015).

In this paper, the authors draw on methods used in the field of interaction design, emphasizing a user-centred design approach including methods such as usability testing, design metaphors, interview with users, video observations, focus groups, and think aloud sessions. However, a challenge of these methods is that they are designed for adults and are not necessarily appropriate to investigations including children. The guiding questions for this systematic literature review are (1) the motivation for conducting usability tests with children, and (2) the kind of methodological, practical, and ethical considerations that should be considered when involving children in usability studies. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist and PRISMA flow diagram are applied in order to assure the quality of the process of this investigation. Nine articles are analyzed and then synthesized by applying the constant comparative method. The synthesis of the literature review is based on the identified thematic priorities, which are categorized as follows: (1) the motivation for involving children as test persons in design processes, (2) definitions of usability, (3) practical considerations, (4) methodological considerations, and (5) ethical considerations.

Many games are inspired by real life events. The presented adaptation framework is based on the design of a board game with a companion app that addresses the Syrian refugee crisis. The aim of the game is to allow players to simulate the experience of being a Syrian refugee traveling through Europe. We applied an agile development method and participatory design to achieve our ambition. In conclusion we found that turning real life events into board games can be advanced by the following game design adaptation framework, which balances four interrelated layers: (1) real life events (game fiction), (2) game system (formal game elements), (3) movement system (game mechanisms), and (4) meaning (player choice) which prioritise game over story.

This paper presents a scheme of creating an emotion index of cover song music video clips by recognizing and classifying facial expressions of the artist in the video. More specifically, it fuses effective and robust algorithms which are employed for expression recognition, along with the use of a neural network system using the features extracted by the SIFT algorithm. Also we support the need of this fusion of different expression recognition algorithms, because of the way that emotions are linked to facial expressions in music video clips.

The opportunities of applying virtual reality technologies to the presentation of cultural events (musical concerts, city excursions, theatrical performances, and others) in virtual reality devices are described in this work. Challenges that appear during shooting video 360° and their solutions are analyzed. The experience of shooting more than 50 different videos is generalized. The opportunities of further development of practice described within the frames of the International Technology Cluster “Infocommunication and Optical Technologies in Culture and Arts” are analyzed. The example of realization interactivity in video 360° connected to the virtual reconstruction is described.

This study reports on research into the impact of digital technological interventions for including kids with attention and developmental deficits into school class contexts. It describes, how the authors have approached the challenge of researching inclusion of kids with attention and developmental deficits for communication, collaboration and knowledge sharing. The analysis assesses the potential of interventions with digital technology for acting as stimulating enzymes for life and learning. On the basis of a thorough discussion of the findings, the authors assess the degree to which interventions with digital technologies, e.g. Virtual Learning Environments (VLEs), may promote inclusion through stimulating the participation in life and learning of kids with attention and developmental deficits.

With the current emphasis on innovation and research in higher education, this paper proposes design-based research as base for a teaching approach to enhance the learning environment of university college students. The paper depicts how students, professors, professional educationalists, and people with learning disabilities worked together to develop five new visual and digital methods for interviewing in special education. Thereby not only enhancing the students’ competences, knowledge and proficiency in innovation and research, but also proposing a new teaching paradigm for university colleges and providing new tools for communication in special education.

Schoolwork of learners with developmental and attention deficits is often characterised by low productivity, many errors due to carelessness or inattention and poor organisational ability. Focus learners have difficulties performing at the same level as their peers. This paper addresses the challenges and investigates the potential of technologies for creating and facilitating environments, where learners are well-supported with respect to overviewing, structuring and planning tasks, evaluating and adjusting participation and management of time.

Pyramid methods are useful for certain image processing techniques due to their linear time complexity. Implementing them using compute shaders provides a basis for rendering image effects with reduced impact on performance compared to conventional methods. Although pyramid methods are used in the game industry, they are not easily accessible to all developers because many game engines do not include built-in support. We present a framework for a popular game engine that allows users to take advantage of pyramid methods for developing image effects. In order to evaluate the performance and to demonstrate the framework, a few image effects were implemented. These effects were compared to built-in effects of the same game engine. The results showed that the built-in image effects performed slightly better. The performance of our framework could potentially be improved through optimisation, mainly on the GPU.

In this paper we investigate the notion of intangible cultural heritage as a driver for smart city learning applications. To this end, we shortly explore the notion of intangible heritage before presenting the tripartite digitization model that was originally developed for indigenous cultural heritage but can equally be applied to the smart city context. We then discuss parts of the model making use of a specific case study aiming at re-creating places in the city.

With the advances of technologies, the application of computing to aesthetics has rapidly increased. However, little effort has been put to apply aesthetics to computing. Aesthetic Computing is an attempt to fill that research gap. The present paper revisits and elaborates its concept, and highlights “embodiment” as a new format of representation of Aesthetic Computing. The present paper also describes how embodiment can provide more opportunities for accessibility and personalization of computing across different projects – art, STEAM education, and in-vehicle interfaces. Finally, more aesthetic components in Aesthetic Computing are discussed for future research.

In this paper we describe AcuTable, a new tangible user interface. AcuTable is a shapeable surface that employs capacitive touch sensors. The goal of AcuTable was to enable the exploration of the capabilities of such haptic interface and its applications. We describe its design and implementation details, together with its strengths, limitations, and possible future applications.