Co-located augmented play-spaces: past, present, and perspectives

In this survey we refer to play as a social, bodily activity that people (partially or primarily) engage in for fun and entertainment. Play in that sense has been researched for decades. Best known are the early works based on analysis of (human) cultures, language and practices by Roger Caillois, and by Johan Huizinga. Both authors explain that there are many different types of play including but not limited to goal-oriented outcome games, cultural performances, and games that simply stimulate the senses [9, 10]. Both authors view play as being omnipresent in our nature and culture. Both the developmental psychologist Lev Vygotsky and Jean Piaget referred to play as being an important element in the way children develop, although the two have different views/theories on (the stages in) children’s development [11, 12]. Iona and Peter Opie also did essential work in researching play in the second half of the 20th century, with the archiving, collecting, recording, and analysis of children’s play and tradition in the UK. We refer the interested reader to [13] in which the Opies’ work is compared to current day play in the UK. Recently Jaakos Stenros wrote a thesis on the spectrum of playfulness, play, and games, with an elaborate review of definitions and positions of these and other authors [14]. Based on this work, from our focus and point-of-view, we see play as ranging from structured play with non-changing rule-based games to open-ended play which is more frivolous, imaginative, and non-deterministic. Both ends of the spectrum have their benefits and downsides with regard to what effects play can have outside the activity itself, for example, stimulating creativity, improving cognitive development, learning social skills, or (better) enhancing physical skills.

Interactive play allows for enhanced play experiences by combining traditional play with advances in technology [1, 15]. We think that true interactivity is more than simply turning a product on or off and instead requires a dialogue of actions and reactions [16‐18]. Interactive play is more than electronic toys such as remote controlled objects (drones, cars, and balls), light sabers, and walkie-talkies. Although such electronic toys also combine technology with playful activities, we see these electronic toys as inherently different from interactive play systems. Looking at the field of interactive play, we see 4 elements that together separate interactive play from this type of electronic toys. First and foremost, all systems that we include in our definition explicitly require body movement for interaction, creating an embodied interaction that is different from the interaction required by computer games played with a joystick, mouse or touchscreen [19, 20]. The systems respond to this movement-based type of input. Second, the feedback is enhanced, more than just the physical impact of the movement. The timing of the feedback is ‘direct’ thus not after the entire interaction, and the feedback is offered in gradual forms, for example, lights/visuals in different colours, a variety of sounds, and movement/vibrations in various intensities [17, 18]. Third, there is some history of state: for example, the system remembers where a player was standing a few seconds ago in order to switch between the states or to keep a score [21]. Fourth and optional, depending on the type of device and the goals, systems can be made more interactive by sending and comparing the states of multiple devices/players (between devices) and this provides more opportunities for play with multiple players, for example, turning on the lights around another goal once a player has passed a defender and has scored¹ [23].

Besides promoting interactions and providing pleasing forms of feedback, interactive play systems can sense, detect, and observe behaviour of the user. This allows us to intervene during play and to adapt the game based on the players’ interaction and performance [2, 24‐26].

Children are used to playing with digital entertainment, which also leads to children spending more time with digital games [3].² There is an overall trend that has caused people on average to adopt a more sedentary lifestyle³ [27, 28]. Introducing technology to make movement based playful activities more appealing could help to (partially) counter this trend [29] as it seems to be a promising way to encourage children [24, 30, 31], teenagers [32], adults [33], and elderly people [34] to move more at least on a short-term basis [35]. A few warnings recognisable in the work of Marshall & Linehan for providing a transparent argumentation related to physical activity is to not overestimate (long-term) effects of exertion games, to recognise the importance of food intake when considering weight loss, as well as to recognise that discouraging certain health-related behaviours can go against what users actually want [36]. Marshall & Linehan also point out that researchers active in the HCI domain should be careful in interpreting the literature from other research fields. They advise against the use of the obesity epidemic as a rationale for promoting exertion games, and instead mention that exertion games (and trying to stimulate physical behaviour) can have other benefits.

A second type of stimulation of physically active behaviour is focusing on physical skill development. In Japan it has been shown that some types of physical ability have been declining in the last decades as well [27]. This skill development can be stimulated with simulation of sport elements, adding motivation with game elements, incorporating ways for improved reflection on performance, and quantifying player progression [30, 37‐40]. A goal of interactive play systems can also be to create a motivating activity in the rehabilitation process, where the systems help players to (re)gain skills that have declined through health problems [41, 42].

Digital entertainment compared to traditional play might lead to fewer social interactions—more children are interacting through and with their technology (e.g. mobile phones) at the same time being together but alone: ‘Alone together’ [43]. Turning technology from a problem into the solution, well-designed interactive play could instead also increase social interactions by stimulating player interactions directly by giving players different roles [26, 44] or by starting discussions about games, sharing interpretations of interactive elements, and stimulating negotiations regarding resources or rules [1, 15, 45, 46].

A subclass of stimulating social interactions consists of stimulating social interactions between people that are geographically separated. Often this is combined with exertion interfaces, ‘an interface that deliberately requires intense physical effort’ [47, p1], often based on sports. Combined with the distribution this becomes Sports over a Distance, a category of systems that attempt to break away from social isolation and sedentary behaviour that seems to be supported by traditional digital games [48]. Such systems include technological ways to provide augmented sports, such as joint jogging [49, 50], kicking/throwing a ball against a wall [51], (kick) boxing [52], and table tennis [48]. Some systems also provide ways for haptic feedback, such as a game of tug of war [53] or arm wrestling.⁴ This is primarily a different goal than the previously mentioned stimulation of actual sports movement, as it uses sports to get people to interact socially over a distance, instead of being focused on training certain abilities. Nonetheless, it is important to realize that many pervasive play-spaces often target several of these goals simultaneously.

Play is important for the development of children in the physical, social-emotional and cognitive domains [54, 55]. By interacting with other children, they train negotiation and social skills. Cognitive skills are often achieved by creating and adapting game rules, scenarios, and characters [54‐56]. It seems that introducing technology into traditional play could also aid in children’s development. Various design strategies for creating interactive play systems fit quite well with current psychological models about learning [8]. Some installations explicitly build on these models to create interactive playgrounds that explain mathematical notions such as bar charts [57] or algorithms [58, 59]. The installations can also be applied for explaining other educational topics such as geometry, physics, geography, music concepts, and language, or for understanding more moral topics such as environmental issues, cultural diversity, and social justice [8, 60]. Furthermore, they can be used to show the relation between educational elements, for instance showing that science is a network of knowledge [61]. A variety of interactive play systems also try to stimulate creativity. A well-known approach is open-ended play or emergent games, in which interactive elements provide an emergent space in which players are stimulated to create their own goals, games, and adapted rules; instead of strictly prescribing games and how they should be played by their rules [1, 5]. This is an approach that is related to open-ended interactive art works which are not completely defined by an Author/Artist but rely on the interpretation of the reader/visitor [62].

A fourth reason that is mentioned is hedonistic, a focus on applying interactive play in order to provide a (new) fun experience, perhaps improving well-being (indirectly) with positive effects for the general health of the players, or simply for commercial reasons [5].

Beautiful music, splendid landscapes, mesmerising scents, and pleasing fluffy materials are all well-known ways to provide such an experience. Food intake is another important way to provide such a joyful experience. Although currently uncommon, edible interactions can be used to augment interactive systems [63] and vice versa [64], and food intake has proved to be a interesting stimulus to investigate multimodal hedonic experiences from a neurological point of view [65].

There is a variety of interactive toys, objects that can be carried and which are enhanced with interactive elements. Due to the differences between them there is also a variety of terms to describe them. We will use the following set to categorize the interactive toys: tabletop tangible interactive toys [3], handheld playground props [1, 4], wearables [22], and semi-portable playground props [21, 31].

We now turn to the second main line of systems: interactive environments. This includes systems that embed the environment with sensors. It may be that sensors are put into fixed objects, a floor or a wall, or that an entire room can be equipped with sensors. The systems fitting these physical characteristics mainly seem to come in two types: fixed interactive objects and interactive screen environments.

GEO-location devices make use of GPS (sometimes Wifi, Bluetooth, or RFID enabled locating) to respond to being located somewhere. The games played with it, geo-location games, clearly provide a form of interactive play. However, We will not focus on them as they differ slightly from most previous systems as they trigger moving over larger distances, are (ideally/theoretically) not confined to a certain space, nor do they need co-located social interactions, and (most) do not need to be played by people at the same time. Therefore, the following set of systems can be seen as less complete than the previous types of systems. We provide a description of several types of systems that we have encountered in this domain, mainly using the ‘early’ and/or famous examples.

The recent hype around Pokémon Go and its success clearly shows that these games have a large attraction value. One reason for this rise, besides targetting a nostalgic fantasy world [75], is probabaly the now easily available location-specific infrastructure [85].³⁹ The games have great attraction value and are successful in getting children to move. However, only the future can show us whether such games are actually suitable enough (for young children). The issue of safety, especially, could become a concern if the games could persuade children to go to unsafe zones.

Vogiazou et al. created CitiTag, a game where a PDA device is used to play a location based version of the traditional game of tag [147]. Björk et al. created Pirates, a mobile game themed around a pirate world, that uses proximity sensors to link visiting physical locations to sailing to and visiting virtual islands [148]. Flintham et al. created Uncle Roy All Around You, a mix between a geo-location game and theater, revolving around the concept of trust [149]. Some players have to find ‘Uncle Roy’ by walking around on the streets of London with handheld computers. Benford et al. also created ‘Can You See Me Now’. This is also a tag-like game where performers/actors are walking around a city with a PDA in order to chase after online (navigating) players [150]. Furthermore, Benford et al. also created Savannah, an educational game for six children at a time about ‘the ecology of the African savannah’ [151]. Rogers et al. created Ambient Wood, a digital augmentation of a woodland, aimed as a learning experience for children carrying out a scientific inquiry [152]. Van Leeuwen et al. created Beagle, an app consisting of a ‘radar’ with which hospitalised children search for bluetooth tokens (Beagles) distributed throughout a hospital [153]. Piekarski and Thomas created ARQuake, which is one of the first examples of an augmented reality game in an outdoor setting [154]. They build on the Quake game in which players have to shoot monsters and can collect objects. Cheok et al. created human Pacman, which uses a similar setup with improved hardware, including a see-through HMD augmenting the physical world with computer graphics [155]. They also added physical interaction with Bluetooth-enabled objects, and even sensing touch of an object or player. A similar game PacManhattan, was created by NYU students but was less technologically enhanced. Players had no HMD and had to update their own whereabouts at each street corner.⁴⁰

So far, this section presented an overview according to the physical characteristics. We now also provide an overview of these systems (in cited papers) looking at which modalities were used for feedback, and in what combinations. This overview also includes a limited number of available commercial systems that were used in the research activities such as the Wii, Kinect, and Donkey Konga. We omit duplicates of systems. That is, often identical systems are used for different types of studies and in the table we include only the first paper encountered with that version of the system (n=18).

Furthermore, review style papers that do not need to clearly describe feedback modalities (e.g. [71]), often reporting on more than 6 devices, were also omitted from the overview (\(\hbox {n}\approx 13\), e.g. [3]). Many papers cited in this manuscript are used for underlying theory and do not include a clear description of systems (\(\hbox {n}\approx 62\)⁴¹). Five papers with additional systems related to new forms of CAPs (not included in Sect. 3) were explicitly added during the review process to exemplify modalities [63, 64], future directions [74], and addressing theory [156, 157]. The resulting overview is presented in Table 1, where the numeral indicators help to recognise what combinations were addressed.

For Table 1 we have used a pragmatic subdivision of modalities into 15 categories suitable for our purposes. We used the use cases as reported in the referenced papers and as interpreted by the first author of this survey. For the visual sense we made a subdivision into displays (including LED displays), projections, LEDs, (spot) lights, and movement of objects. Regarding the auditory sense we divided that into the categories: music, voice, and sounds. We noticed that it was hard to recognise whether only sounds were made or actual music was created. We omit further subdivision in localised sounds with a (virtual) point of origin (e.g. [99]), directed sounds (audible in small area), or type of sounds.

With regards to the haptic modality (or the somatosensory modality, a term with similar meaning), based on personal communication with two colleagues working on haptics, we included 4 subdimensions: tactile, kinestehtic/proprioceptive, thermoception, and nociception. Another subset triggering pruritoceptors related to ‘itches’ might be considered as an additional subset of nociception [158]. Although sufficing for the perspective of what is targeted, there can be some debate whether these are all to be seen as fitting the haptic container.

The olfactory sense, the gustatory sense, and the sense of balance (equilibrioception) as well as the subdimensions of thermoception and nociception were included during categorisation but later removed from the table. The few systems targeting these are reported on individual basis in the text.

Possibly some systems include modalities that were not reported clearly. Furthermore, all systems, besides these enhanced forms of feedback containing certain dynamics, can also contain more static and inherent visual, tactile, auditory, olfactory, and taste characteristics. For example, a responsive moving styrofoam ball (visual-movement) that makes certain sounds based on user-input [18] also has a certain taste, a certain colour, and might have a certain smell. Elements like these are not responsive to input, (probably) not targeted specifically by designers, and therefore not considered as a targeted modality.

A first technique for evaluation is simply listening to what people have to say during and after their play activity. It can be an important source for information during evaluations. Various techniques have been developed to stimulate people to verbalize what they experience(d). Often quotes are used as examples to describe how people experienced a method [91] or design [98].

Most questionnaires make use of Likert scales, consisting of statements that the participant agrees or disagrees with (often on a scale of 4,5, or 7). Several statements belong to one construct, and multiple constructs can be used to investigate a certain topic of interest, for example, the perceived presence of other players. Instead of Likert scales a type of semantic differential scale can also be used, where opposite verbal anchors are at the ends [168]. The questions (or agreement with statements) measuring one construct should be answered with approximately the same scores by one person showing that indeed one construct is measured. This can be expressed with the Cronbach’s alpha. The most well-known example of such a validated questionnaire is probably the big five inventory regarding personality traits [169]. Such personal traits can influence results and is often used to explore interesting links between personality and the experience or use of a system.

Another predictor can be the tendency to get immersed in an interaction, and it could be helpful to apply a version—revised by Berthouze et al.—of the Immersive Tendency Questionnaire ((G)ITQ) (based on Witmer and Singer’s work [170]) before the interaction starts [171].

Video analysis is a method/tool often used in evaluation with children [1]. Druin et al. do mention that (in the old days) recording children was sub-optimal, as (sound) quality was mediocre and recording could also influence the behaviour as they tended to ‘perform’ in front of the camera [193].⁴⁹ Other studies did not observe such a change in behaviour and did successfully use video analysis, perhaps due to incorporating different strategies of evaluation protocol, or perhaps due to habituation to the recording technology [132, 152]. In some cases first person view/head mounted cameras are applied [90], in others multiple cameras are needed to cover the span of the playing field [87], in order to follow several children at the same time [151], or to look from different angles [39, 84]. Making a trade-off between in-depth analysis and efficiency, to shorten the time consuming process of video analysis one could also make a pre-selection and or shorten/edit the video clips to be analysed [98].

One of the measurements related to fun is the time participants spend on an activity out of own volition. Commercial platforms from both Kompan and Yalp include web interfaces that can be used to see how often their equipment is used and which games are played. It seems this would also allow for long-term testing in real-life settings. Various systems can also make use of logs of the system regarding interactions speeding up the evaluation process, for example, time played [119, 145] or use [119, 153], the performed movements/actions[200], and positions of players [25, 26].

To aid in the design of interactive play systems many researchers share their insights in the form of guidelines, frameworks, taxonomies, or lenses.

Besides showing how to design through examples of the design cases it is often good practice to show whether the design suits the context of use. Senda mentioned a broad distinction of four physical categories of contexts where children could play: (1) streets, (2) parks, (3) schools and education facilities including museums and libraries, (4) public spaces [27]. Many of the papers and systems mentioned do indeed target one of these settings. The suitability in this physical context for the intended target group forms an important factor to show that a system is fit for its (designated) purpose. Some systems aim at an older target group in a context of art galleries [221], exhibitions, and trade fairs [37]. Games such airhockey over a distance are envisioned to be more appropriate for (employee) gathering areas (canteens, reception areas), arcades, airports, youth clubs, and children’s hospitals [128].

These examples show that a wide variety of contexts can be targeted and some authors show tests regarding the applicability in their envisioned environment, or they do their actual tests in the appropriate context to make their results applicable to such a realistic context. For instance, this was exemplified with the high throughput of people interacting with a water fountain, tested at the ‘Universal Forum of Cultures’ [108]. Morrison et al. did most of their investigations during art exhibitions [62]. Kajastila et al. placed their Augmented Climbing Wall in a commercial climbing centre [119]. Mast et al. [124] and van Boerdonk et al. [134] performed their user studies with respectively cooperative Tetris and TouchMeDare during a large music festival.⁵⁵ Lund et al. showed qualitative and prelimanry results with a pilot for home rehabilitation with their interactive Playware tiles [34]. Van Delden et al. placed their playground in an art-gallery for several months, where children enjoyed playing in it and came back to play with it again [146]. Hof et al. did their testing in an after-school care centre to deal with the influence of the environment and to provide known physical objects stimulating creativity [181]. To address the fairly specific context of disabled people, Larsen recorded numerous interaction sessions of this target group and their caregivers with his interactive play systems in the real-life setting of a care centre [17]. Van Delden et al. tested their personalised interactive gait rehabilitation games with therapists and rehabilitants during actual sessions at a rehabilitation centre [42].

Besides the focus on structuring the design process, and investigating the fit for purpose in context, we have also seen a large amount of research into interactive play that focused on investigating certain design elements. We cannot report all of these influences but they do give a good impression of this type of contribution and to this end we highlight some that had impact on our own work as examples.

Intervention based play research includes a focus on doing ‘experimental research’. There is a difference between (A) showing the possibility of a new technology, exploring the design space, or investigating specifics of an interaction, and (B) showing the effect of certain concepts, choices, or designs. While for the former it suffices to make one design, discuss some of its hypothetical possibilities while reporting successful user experiences but refraining from any conclusion on causality of design elements (e.g. [222]), for the latter more advanced experimental research should be done.

Claims regarding a certain guideline, fit for purpose, or design element are made more powerful when there is a comparative study between such a choice and an alternative. It is important to actually evaluate and compare multiple design options in order to show that a suggested design decision was indeed of influence.

We noticed that a comparative experimental design of evaluation is targeted by many in this field. To create interesting comparisons different versions of concepts can be developed to investigate their influence, for instance in a broader context of a research through design approach [179].⁵⁶ The comparative experimental design of evaluation is shown in use cases where physical play behaviour is deliberately changed with design elements [24, 26, 125], with HUGs regarding incorporating HR or not [90] and incorporating physical or virtual objects [87], and regarding the effect of embodied interaction on social presence, social interactions and bonding [19, 47, 124], on video quality [47], and on engagement [19] and excitement⁵⁷ [155].

In other words, in many lines of research related to CAPs, we see a focus on making a difference between intentions and effect. This focus makes a difference between design options allowing for certain behaviour to occur and ‘proofing’ it actually encourages, promotes, or elicits it. Luckily for us all, it is also easier to compare two relative experimental variables than it is to prove that a single one works [165]. Therefore, we emphasize the importance of using an appropriate experimental design in Intervention Based Play Research. By investigating interventions (e.g. design options, user characteristics, or a certain context) in a well-structured comparative experimental design with randomised control groups or ‘randomly’ assigned conditions, we try to exclude that the encouragement of the wanted behaviour is not merely the effect of context or the very nature of the players during evaluation (testing, instrumentation, etc.). This provides us with comparative results. Unfortunately this also often requires a controlled study set-up which results in a less ‘holistic picture of how children [or players in general] play’ [181].

We have also seen that many researchers show that their research fits their underlying argumentation for developing certain kinds of systems. For example, the effects that they measure in experiments relate to the impact that they set out to achieve. Their motivation and argumentation can focus on the research contributions, or focus more towards end-user related goals. In order to work towards achieving the underlying argumentation, it seems good to also actively promote certain kinds of behaviour. We can, and probably should, investigate what elements of a design elicit such positive effects. We can make use of the possibilities introduced with the introduction of interactive technology during play. For instance, the introduction of a controller that requires embodied interaction can have positive effects on goals such as increasing physical activity and social interaction [19]. These kinds of interventions, when chosen well and evaluated appropriately, fit into intervention based play research: an interaction-design oriented approach that should add both from the end-user perspective as well as from the research perspective (the knowledge base) while using scientific comparative experiments for evaluations.

Obviously, one should take care not to over generalize and to be ‘reading too much into the data’, especially when a single group of children is involved in combination with statistical methods [186]. Furthermore, there are important formative evaluations that often require more qualitative insights, that might benefit from being investigated in a more efficient manner. These formative evaluations are necessary to get to a good design, and sharing such findings can also be informative for others. Therefore, our suggestion for following an intervention based research approach focuses more on ‘end’ evaluations when interaction design needs to be less explorative.

We will now mention opportunities of interactive play that fit the intervention based play research approach, and could help to bring the researcher perspective and the end-user perspective closer together.

Poppe et al. mentioned that stimulating behaviour change during play and incorporating adaptive systems can be fruitful directions for interactive play research [2]. Smart solutions that balance based on the players’ effort seem to be promising for allowing people with different physical skills play together [121, 226].

Play can also be actively steered to temporarily increase or decrease activity [24, 68]. Steering refers to reaching goals by the deliberate introduction of interactions that change in-game physical play behaviour in desired directions [146]. This steering is closely related to what Altimira et al. called inducing behaviour and might allow us to improve the experience during longer lasting sessions, balance a game, and cause people to move more or interact more [68, 123]. Furthermore, some players might not (be expected to) be as socially involved in the game as the others. This could be sensed or set by a facilitator, and subsequently the game could give this player another role and/or have them lured into the play by others [26, 44]. In a learning context for children with hearing impairments, where making mutually understandable vocalisations between children is one of the goals, a game can stimulate the use of hands to prevent sign language from being used [138]. In these types of context there is an opportunity to influence children’s play in desired directions while they are playing. This steering of interactive play behaviour during the game is triggering a change of behaviour in wanted directions with playful elements. It also seems to be slightly different from most ‘traditional’ persuasion and nudging activities. It does not primarily aim to change long-term lifestyle behaviours outside the game, such as smoking, (un)healthy diets, medication intake, or daily level of physical activity. It is different from constraining behaviour [67], or manipulation and deception [227], even if the participant might not perceive this as such the first time: it does not deliberately hide options or enforce a way of interaction by making it the only means of input. Instead, and similar to using different ways to explain suggested use to people [134] or explicitly leaving it out for intended ambiguity [223], it tries to change the play interaction itself: influence the players’ activity, performance, or role, change the interactions between players, the locations players visit, or the type of interaction players perform [24, 26, 44]. Further investigation of such techniques might bring us closer to successfully addressing the goals we have mentioned as argumentation.

From Table 1 it is clear that the sound and visual types of feedback are predominately used. Especially uncommon are addressing the in general more exotic modalities for HCI: olfactory (smell), gustatory (taste), haptic-thermoception (temperature), haptic-nociception (pain, as well as itching), and equilibrioception (balance). We have already pointed out recent endeavours that showed intriguing systems when several modalities are targeted more in depth, leading to thrilling experiences [112], cross-modal interactions [64, 225], or making interactive spaces accessible for people with certain disabilities [70, 138].⁵⁸

Depending on what one defines as multimodal one might question whether some CAPs that seem to respond only to body movements should be considered to be multimodal. Following Oviatt’s description of ‘two or more combined user input modes’ [160, p1] would identify them as not being multimodal. Turk’s description would allow for identifying them as both: ‘a channel describes an interaction technique that utilizes a particular combination of user ability and device capability (such as the keyboard for inputting text.... Multimodal interaction, then, may refer to systems that use either multiple modalities or multiple channels’ [159, p191]. Whereas the description of Bekker et al. in [180, p329]: ‘we use the word modality to indicate a form of sensory output of the play object’, would identify most of the CAPs of this paper multimodal, see Table 1.

Research by Oviatt [160] of a more traditional HCI setting on how multimodal interaction (voice and pen input) changed communication, how providing the possibility for multimodal input did not mean users would interact using that input, how additional modalities could be complementary but also asynchronous, triggers the question how addressing more modalities would change interaction in CAPs. Besides changing the interaction possibilities from a human perspective, multimodal cues might also be used differently and for other types of sensing: for instance, currently we are looking into whether sound levels combined with movements of players can help to make a system adaptive to ‘engagement’ in play. Furthermore, from the indicated examples of sensory modalities (related to user abilities) by Turk, we see that many are (almost) not used, such as facial expression, lip movements (which might have changed interaction for children with a Cochlear Implant [138]), speech input (for simple instructions), other types of non-speech audio, and face-based identity (perhaps suitable to create personalised, adaptive, and adaptable CAPs).

At this point it is good to emphasize that the input modality for a system is not one-on-one related to the accompanying sensory modality of the human: using an example from CAPs microphones might be related to a human making sounds but can also be used to primarly measure movement or touch in a malleable interface [16]. We also noticed this difference between on the one hand, modalities for system input and output, and on the other hand, human action and perception, when we created the overview of ways of augmented feedback presented in Table 1. Early on in the process we made the decision to indicate the modalities that where augmented by the system and using a category based on triggering the player’s senses (where a single LED might be perceived different than a screen or than a projection) and the table is discussed as such. However, it is also interesting to see how an interaction can trigger stimulation of another human sense, for example triggering a feeling of vertigo with a non-augmented swing. We have seen both an example of this swing being moved by the system [112] and of it being actived by the user [113]. This raises the question, where we ourselves have not yet looked into, whether humans would perceive the augmented feedback differently in a play-space, and if this is not the case this also limits the accuracy or usefulness of Table 1.

Another way in which addressing modalities is important is in the way they are reported. During the creation of the overview presented in Table 1 we noticed that descriptions of visuals are in general more detailed compared to the other modalities. Sounds in particular were often described in very minimalistic ways. This might be inherently related to the traditionally paper-based medium of writing scientific papers. We suggest that for audio this could partially be resolved with a movement towards including and referring to videos and giving access to audio files, as is possible for this current journal for this reason, and is promoted for conferences such as CHI and SIGGRAPH. In turn this can also make it a more prominent and recognised part of scientific dissemination. Other modalities will have to do with descriptive analogies and metaphors, at least for the time being.

Many studies on pervasive play-spaces focus on first time use [1]. Due to the novelty of interactions such studies are often heavily influenced by first time use; in the longer run behaviour might change. This could also mean that the effect of design elements can change on a longer term, and children might become less inclined to play again after several sessions [181]. For instance, sounds might be of added value in the beginning but could annoy people (especially adults, neighbors or bystanders) if they are monotonic, uniform and are played over and over again with limited variations [113]. Bekker and Sturm already suggested in 2009 that showing the true promise of interactive play(-objects) also requires longitudinal studies [179]. At the same time, Hof et al. noticed that performing a user test several times (three times, once each week) with observations and questionnaires with the same groups of children is already very difficult to arrange [181].

The use of automatic measurements might aid in such play analyses in the longer term. The commercial systems of Kompan and Yalp provide logs of how long which game is played, they can also be updated from a distance. With the increasing number of playgrounds sold around the world (over a 100 for some systems), it could become interesting to start scientific research with these systems, and long-term tests using A/B testing, investigating certain game elements and design pattern, and then evaluate whether it affects the game play in order to inform future design.

The ability to update the systems over a distance also allows for changing the content in order to keep it up to date following contemporary trends, and regarding to some aspects (e.g. a quiz) keep it unpredictable. Both features, the automatic logging of game play and structurally changing interesting game elements similar to the ones mentioned earlier, allow for studies on a longer term leading to interesting insights, seemingly providing an interesting way to bring the research perspective closer to the end-user’s perspective.

One currently popular context for play-spaces are escape-the-room games. Additional technology in these rooms allows to create new dedicated devices and tools for thrilling puzzle activities, for instance, using video conferencing systems in two similar but slightly different rooms allows to distribute the play in interesting ways[73, 74]. We also see ample opportunities for integration of other existing technology including Virtual Reality and Augmented Reality systems. We use the work of Pan et al. and Shakeri et al. to speculate that using such technologies (with inherent altered, private, and public information) could create augmented real-life escape rooms that can trigger types of playful embodied interaction to stimulate verbal communication, sharing of information, and solving augmented puzzles [73, 74]. We also envision that players can be assigned specialist roles in the escape rooms giving unique abilities and thereby forcing certain social roles (c.f. [44]. Furthermore, such a context also allows for investigating most of the promising directions we have discussed. We envision experimental research with similar groups visiting similar but slightly different rooms, where aspects of interest can be altered. We see opportunities to steer behaviour to let players investigate social roles or balance between player skills by providing hints in a more adaptive manner. The systems can include more exotic modalities, where players might be required to eat, use cool/heated devices, incorporate released smells in order to solve puzzles, or to encounter in pleasantly painful or equilibrioception-rich (targeting sense of balance) experiences.