1 Introduction

Festivals are exciting events, visited by a large number of people. People go to festivals to watch musical performances, participate in novel activities and enjoy the relaxed atmosphere. Another incentive is the open and friendly community in festivals; visitors want to meet new people and share the fun together. Building upon this, we have designed an interactive canvas that encourages strangers to engage in an intimate full body meeting together, using music as a motivator.

In order to design this unique meeting experience, we investigated design processes (Sect. 2.1), and related projects on meeting and moving (Sect. 2.2) and on collaborative music making (Sect. 2.3). The starting point for our design was a target group analysis (Sect. 3.1) and idea generation (Sect. 3.2), which resulted in the final idea (Sect. 3.3). Section 4 focusses on the development of the final idea into a concept demonstrator, while Sect. 5 continues with the second design iteration, ending with a case study in Sect. 6. We will end this paper with a discussion and conclusions (Sects. 7, 8).

2 Related work

2.1 Interaction design approach

In the field of interaction design, the user experience paradigm is becoming as important as, if not more than, the usability paradigm. Interaction design is no longer limited to usability, including efficiency, effectiveness and satisfaction [37]; more and more projects focus on hedonics, aesthetics, richness, pleasure and fun, affect and emotion, aspects that are all part of the user experience-oriented design [29]. Where the user experience paradigm focusses on the design of a product, the concepts of embodied interaction [13] and co-experience [5] include the user experience resulting from both product and social interaction. Embodied interaction is about ‘the creation, manipulation and sharing of meaning through engaged interactions with artefacts’. The focus in this field lies not only on the relationship between action and meaning, but also on the social context of the interaction. Co-experience is about the user experience that is created during a blend of interaction with the product and social interaction, where the user experience cannot be the same or even possible without the product and other people interacting with it concurrently. The product we describe in this paper fits into these two concepts.

A field of interaction design that is closely related to embodied interaction is called tangible interaction [16, 41]. This new and emerging field tries to counter the recent trend of everything becoming digital, e.g. photos, videos, communication and even people. For many applications, this digitisation is a great trend, in particular for databases, but for other applications the physical world has advantages that are difficult to recreate digitally. Therefore, the tangible interaction community studies the advantages and disadvantages of tangibility and physicality in interaction, while in the process creating many demonstrators, technologies and designs, and also theories and frameworks. One recent framework [20] broadens the perspective of tangible interaction into different fields by looking at it from different perspectives. One of those fields is the expressive movement-centred view, which shows parallels with the choreography of interaction approach [24] that we adhere to in this paper.

Klemmer et al. [23] describe aspects of human embodied engagement taking along aspects of interaction design, user experience design, and embodied interaction. One aspect of their focus is on thinking through doing, which is essential when designing interactive products that need to embody a high complexity of interaction issues. This idea closely resembles the design research process. We used the design research process as presented in [42], as it has proven to be an effective way to elaborate the design of experiential interactive products. It builds on the fact that designers are capable of understanding the user experience and have the power to translate this into a product by thinking through doing. The idea is that the knowledge of the designer is developed via evaluation iterations with experiential prototypes.

Concurrently, the design process was based on the choreography of interaction approach. In this approach, artefacts and installations are created through and based on movement. The primary focus of this approach is on the creation of novel activities with a specific physical involvement, dynamic form and experience of meaning. Artefacts or installations are developed integral to the development of the activity; they are made to elicit the activity. During the design process, designers use their body as a tool to explore, discover and create activities and products. This approach is in parallel with an increasing focus on the role of movement in interaction and user experience-oriented design [21, 30].

2.2 Meeting and movement projects

The aim of our project was to let visitors get acquainted with each other by an intimate ‘dance’ through an interactive canvas. A project example that shows similarities with our work is Keep in Touch [31], an interface for communicating and supporting intimacy in long-distance relationships. Both partners in the relationship have a translucent fabric with sensors embedded and stretched across a doorframe to show a blurred image of their partners’ environment. Once they touch their partners’ blurred body, the image comes into focus revealing their partners’ features. This way, both partners can create an experience of intimacy.

Obviously, this intimacy that is caused by touching each other can often be a barrier in the case of two strangers touching each other. Krueger [25], who used visual representations of people on large projected screens, already stated that a stranger’s normal embarrassment about touching was in conflict with their desire to explore this new way of interacting. In our situation, this trade-off between intimacy and curiosity was even stronger, since exploring was not only rewarded by new experiences, but also by new musical feedback.

Another example of people ‘meeting’ each other in a new way is the Bump project by Association Creation [4]. Two catwalks are placed in the public space of different cities. Once someone steps on the catwalk, the corresponding board will rise in the other city. Just like our installation, it evokes a direct reactive play between people through touch impressions.

Murmuring Fields [39] is a mixed reality environment in which people can trigger spoken sounds of philosophers with their movements. The movements are measured by a body tracking system. People’s movements influence the sound and therewith also the soundscape of the other users. In contrast to our work, there is no direct social interaction intended here.

Another interactive sound making environment is The Sound Maker [2], which creates sound sequences based on a body-based metaphor where body movements are connected to output sounds. This project was carried out to explore if embodied metaphors helped children to learn. The results showed that embodied metaphors did help. In both these projects, audio can be manipulated through movement, like in ours, but no social interaction was intended.

2.3 Collaborative music making

In the development of our meeting, we aimed to encourage visitors to cooperatively make and shape bodily contact with each other. We investigated how to motivate people to do so. Therefore, in our design, music was not chosen as the goal of interaction, but as a means or material for social interaction. In other words, we could have chosen another means, other than sound or music, yet we did choose this means since it best supports our social interaction goal: meeting strangers through touching.

The relatively new and small field of collaborative music making offers interesting findings that fit with our intentions. Research on social impact showed that ‘people appeared to be more comfortable socialising and engaging with strangers as they gathered around a shared object’ [8]. This conclusion supports our aim: offering people a means to get acquainted with each other, and thereby increasing the social coherence of the festival.

Apart from the resemblance of this field with our project, there is one small, yet, essential difference. Most studies (e.g. [8, 10, 22, 36, 44]) focus on the cooperative creation of music, but not primarily on the creation of social interaction. In these studies, people can contribute individually to the musical composition, yet music is not used for a specific experience of social interaction. In our situation, music is the means for people to interact with each other. Whereas in the previous projects an ‘instrument’ could be played by one person alone, we wanted to create an ‘instrument’ that could only be played by two or more persons; music is only played through social interaction.

One project did relate to this idea, the Tooka project [15], focussing on the way in which intimacy between two players could be expressed through sound. The Tooka consists of a hollow tube in which pressure is transformed into sound. The pressure is modulated by two players, who place their mouth on each side of the tube. Two players have to collaborate in order to achieve the necessary amount of pressure and also interaction is required since the pressure has to be controlled together. Their results show that subjects felt linked to each other, but were not able to articulate the feeling.

A later publication of Blaine and Fels [7] concluded that for novice players, ‘the overall experience takes precedence over the generation of music itself’. The focus explicitly lies on the social aspect of the musical activity, which was also the aim of our study. Our activity would easily result in a nice musical composition, lowering the threshold for participation and allowing more attention for social interaction.

3 Ideation process

Our aim was to develop a new user experience to contact strangers. Usually, new meetings are hindered or biased because of things such as a ‘first contact’ threshold, visual appearance and prejudice. We wanted to stimulate meetings between strangers, without prejudice and thresholds.

To gain a general insight into the aspects that characterise a meeting, we first defined what a meeting exactly is. Combining the definitions from [33, 35], we defined our intended meeting as a process of two people getting acquainted with each other. Additionally, we observed and analysed existing meetings in a variety of places, followed by movement explorations to gain insight into characteristics of meetings. In these movement explorations, we, as the designers, investigated the act of meeting and its movement characteristics: we tried to get a grip on meetings through experiencing and acting out different types and variations.

The meeting aspects we distilled from our observations, analysis and movement exploration are the following pivotal interdependent elements: the level of familiarity between people, the change of distance between people, the amount, place and shape of bodily contact and the role of visual prejudice when making contact. Traditional meetings show that the bodily contacts people make correspond to their familiarity with each other, varying from distant waves and handshakes to hugs and intimate kisses. In our new meeting, we wanted to play with these traditional meetings and elements, and change them accordingly.

3.1 Target group analysis

Festivals have an open community and a relaxed atmosphere; visitors are open to meeting new people and experiencing new and unusual things. On the other hand, the atmosphere and social coherence determine partly a festival’s success. The design we wanted to make would fit the context of a festival well; therefore, we decided to focus on festival visitors, in particular visitors of the Dutch Lowlands Festival [26], as the target group for this new interactive product.

At the start of the ideation phase, an analysis of the target group was conducted to gain insight into the visitors’ individual characteristics, experiences and motivations, as well as the way the visitors form the Lowland community. For this analysis, a literature review on the subject of festivals and its ambience was conducted (e.g. [27]), in combination with semi-closed interviews with frequent festival goers and the Lowlands organisation. We found that the majority of the visitors literally do as they please at a festival: they act and visit performances without much preliminary planning. Their driving forces are enjoyment, curiosity, exploration and dare. Visitors were eager to try novel things as long as these were fun. The organisation and ambience at the Lowlands Festival exploits this explorative consensus, by providing all kinds of novel, unconventional and challenging activities. These findings were used as input to create personas [12] that describe the typical Lowlands visitors, and mood boards [14] that communicate the overall ambience in the Lowlands Festival.

3.2 Generation of meeting idea

The driving forces of the festival visitors were used in combination with traditional meeting characteristics to develop ideas based on choreography of interaction. By playing with and changing these characteristics, specific movements that create a new meeting were designed. In this interaction choreography, strangers are engaged in an intimate meeting, leading ultimately to full body contact, without the barrier of visual prejudice. People explore the possibilities of increasing and changing bodily contact. Touch, in this choreography, starts with small, groping movements, with hands and fingers only. Gradually, people gain trust and dare to decrease the distance between them, hence increasing the bodily contact together with the level of intimacy. Eventually, they might dare to touch with and get touched by more and different body parts, ultimately leading to intimate full body contact, with continuous shifts and variations (see Fig. 1).

Fig. 1
figure 1

The initial choreography of interaction: increase and dynamical change of bodily contact

Full size image

Having developed the fundamentals of the meeting choreography, we set the general requirements for an installation: we needed an installation that would encourage people to act and meet in the choreographed way, offering a play with the variables of our meeting action. Ranges of installation ideas were generated, like ‘human 3D twister’ and ‘feeling around in a flexible bag’ (see Fig. 2).

Fig. 2
figure 2

Various concepts to elicit the choreography of interaction

Full size image

3.3 Final idea

Eventually, the developed idea consisted of a flexible, opaque canvas through which people can make bodily contact. Several mock-ups were explored to find a way to elicit bodily contact through the canvas [9]. This led to the involvement of musical output: music could be manipulated by making bodily contact through the canvas. Initial quick evaluations led to the choice of music over visual output. Visual feedback showed that the focus moved away from the other person to the product itself, which was something we surely did not intend. In addition, visual feedback clouded the physical impressions made by another person, removing even more focus. Additionally, with our aim of achieving full body contact between strangers, visual feedback brings yet another constraint: in order to see visual feedback, one has to be able to see the canvas. This limits the possible movements and requires a distance between one’s head and the canvas. Gaver [17] already showed that an advantage of sound feedback is that it can be heard from different locations, thus removing the constraints that visual feedback brings; visuals require a focus on the visual, which is not the case with audio.

Just as music was the means to motivate full body contact, technology was the means to enable the music: touch needed to be measured and translated to music. Since we did not want people to wear any sensors on their body, and we wanted to detect touch from any body part such as the hip, the only viable approach was making the canvas touch-sensitive, thus incorporating sensors into the canvas itself.

The concept of the canvas and the involvement of music required a more specific choreography of interaction, focussing on which variations of bodily contact should be chosen to vary musical output. Direct man-to-man ‘bodily contact’ in our initial choreography of interaction was turned into bodily contact through the canvas. This gave users a controlled way to offer body parts to the other user that may be touched, allowing people to determine which parts of the body could be touched by the other user, and which parts could not. This led to the choreography as described in the following scenario (which is visualised in Fig. 3):

Fig. 3
figure 3

Scenario of the final, more specified choreography of interaction

Full size image

Imagine a flexible canvas, with one person on each side. The opaque canvas prevents seeing the person on the other side, yet when someone touches the canvas you see an impression. You can push it gently, stroke it, or hardly touch it at all. When mutual touch is created, a music sample starts to play. The person on the other side might react and shift his or her position. You can either choose to follow the movement on the other side or choose not to. If you follow it, other music samples can be found. If you break it, the sample will fade out again. After a moment, another impression appears, and yet another. To touch them all, you come closer to the canvas, and use multiple parts of your body. The more bodily contact, the more music samples can be found. This interaction feels challenging and intimate, yet you are anonymously separated from the other by the canvas.

4 Concept demonstrator

An installation concept was generated and a demonstrator was built to collect suggestions for improvements. To explain the design of the installation, we used three characteristics: (1) physical properties, (2) sensor system and (3) mapping touch and music variables. These characteristics should elicit the intended choreography of interaction, which we simplified into three meeting variables: (a) decrease of distance between people, (b) increase and dynamic change of the mutual touch and (c) the level of getting acquainted. In the next part, we will explain these characteristics of the installation in detail and the way they intend to provide the meeting variables. Following this, we present the evaluation of the demonstrator, to see to what extent it really affords the meeting variables. These outcomes served as a starting point for the next optimisation iteration.

4.1 Installation design

The demonstrator consisted of a wooden frame of 2 by 2 metres, with a soft, opaque, flexible canvas stretched across it. Inside the canvas, nine capacitive sensors were invisibly distributed in an irregular pattern, and a microcontroller detected if sensors were touched from one or both sides. The microcontroller signalled ‘active’ only when the sensor was touched from both sides (see Fig. 4).

Fig. 4
figure 4

An explanation of the detection by the microcontroller: ‘inactive’ signal when sensor was touched from one side; ‘active’ signal when sensor was touched from two sides

Full size image

Each sensor was linked to a pre-composed looping music sample. When a sensor was activated, the volume of the sample was amplified from 0 to 100%. The sample immediately became mute when one or both sides of the sensor were released. Every sensor triggered a different sample (e.g. bass, percussion, guitar); multiple activated sensors therefore amplified multiple samples, creating a variable musical composition (see Fig. 5). The samples played synchronically to secure rhythmical integration.

Fig. 5
figure 5

An explanation of the mapping in the concept demonstrator: multiple activated sensors amplified multiple music samples

Full size image

The physical properties of the canvas, the sensor system and the mapping were specifically chosen and designed to allow the meeting we developed beforehand. According to us, the soft fabric encouraged users to touch the canvas, creating an opening to discover the interactive possibilities. The canvas was taut to serve as a strong, yet flexible, plane to build surfaces of bodily contact on. The flexibility of the canvas showed impressions of bodily contact, showing people where they could touch the other person and thus make bodily contact through the screen. On one side of the screen, a static visual pattern was present for an attractive appearance (see Fig. 6). The hidden sensors throughout the canvas evoked a cooperative search for music samples: people had to shift their mutual touch in order to find and activate a hidden sensor. The sensors were invisible for the same reason as the feedback was not visible, namely to prevent people from focussing on the screen instead of on the other user. The distribution of the sensors would elicit an increase and variation of bodily contact, since multiple sensors could be activated and combined to play different combinations of samples (see Fig. 7). The hidden sensors and the possibility to feel and follow each other’s touch opened the way to creative collaboration.

Fig. 6
figure 6

The concept demonstrator prototype, with a static visual pattern on one side

Full size image
Fig. 7
figure 7

The coupling of bodily contact variables and music variables for the concept demonstrator and the reaction this triggered from the sensor system

Full size image

4.2 Evaluation of the demonstrator

Our primary goal with the designed meeting experience was to motivate strangers to get acquainted with each other in an enjoyable way and by means of bodily contact. To achieve this, choreography of interaction was developed, using the concepts of enjoyment, curiosity, exploration and dare. In order to elicit this choreography, an interactive canvas was created that gave musical output when touched at the same position, from two sides simultaneously. These three aims (eliciting meetings, the choreography and the interactive canvas) are highly interdependent, making a concrete evaluation complex. Therefore, we decided to test all three aspects separately, in order to get a better understanding of how people interpret and use our design, and to get a better grip on the complexity of evaluating such designs.

A small-scale evaluation was conducted involving ten people (who did not know each other), divided in pairs. The duration of the evaluation was 1 h per couple. The participant group consisted of self-reported festival visitors between the age of 17 and 22 years old, three women and seven men. The physical test room consisted of two separate spaces divided by the installation and curtains. Three video cameras were installed, two taping the participants from behind (captured in real time) and one taping both participants from the side (see Fig. 8).

Fig. 8
figure 8

The schematic structure of the user evaluation area

Full size image

Each participant was placed at a different side of the installation without seeing or hearing the other person. Talking was not allowed during the evaluation, in order to constrain their attention to bodily contact, and because this would better resemble the noisy Lowlands context. The participants were informed that they could make music together by making bodily contact with each other through the canvas. They should start at our signal and could stop whenever they wanted to. Immediately after they stopped, the participants filled in questionnaires and answered nine questions during a video-recorded semi-structured interview. The canvas itself was evaluated using observations, several Likert scales in a questionnaire and interview questions. Questions contained usability and motivation aspects, e.g. ‘To what extent did the installation motivate you to touch the canvas?’ and ‘Do you know how you can influence the music?’

The choreography of interaction was evaluated by analysing the participants’ movements, using direct observations and the captured video material. Subsequently, participants were asked during the interviews why they moved the way they did, while at the same time showing them the video material to get an insight into what caused and motivated their actions.

The actual user experience was evaluated by interview questions and the Self-Assessment Manikin (SAM) scales [11]. The SAM scales, an often-used method for measuring human emotion responses, consisted of three graphical Likert scales, covering pleasure, arousal and dominance. The interview involved questions about enjoyment, motivation, social (inter)action and meetings, e.g.: did you feel like you were meeting someone?

4.3 Results

As much as 80% of the people did not understand how to compose or activate music at all. In addition, people primarily focussed on trying to understand how to make music, without focussing on making bodily contact through the canvas.

Participants told us that the pattern on the screen suggested cues for interacting with the screen. Observations showed that the person on the solid coloured side was more focussed on the impressions created by the other user than the person on the pattern side. Most people interacted with their hands only; interviews revealed that hands-only interaction already required a lot of attention. People who did experiment with involving other body parts stopped their extra efforts, because this in itself did not change the music, as the other person needed to touch that area as well and they felt that involving that extra body part was useless. In the scarce occasions that sensors were found, people kept a hold of them or remembered their positions, which led to a static, rigid and non-explorative form of bodily contact. We also observed that the style of music samples influenced people’s movements, e.g. a heavy beat caused rhythmic and rough dance movements. Surprisingly, several participants indicated that they had the feeling they really ‘met’ someone. When the participants finally saw each other for the first time after the evaluation, they often kept talking at least for 15 min (then we had to ask them to leave) in an animated and familiar way.

4.4 Discussion

The most apparent result was that most people did not understand the concept and only rarely managed to activate music. People focussed on discovering how to make music, instead of discovering each other through touch and music exploration. With hindsight, the design of the interactive canvas directly caused the following: by placing several hidden sensors inside the canvas, we motivated people to search for the music through touch. This resulted in users trying to discover how to use the canvas and to create music. They did not succeed in this, because the musical output seemed inconsistent to them, as there was music only at (seemingly) random moments: the moments when both the participants touched at the correct place. At other moments, when they also touched each other, there was no music at all. This inconsistency caused misunderstanding and, thus, other movements than we had intended.

This conclusion is confirmed by other recent studies, where it is found that people need immediate apprehensibility: they need to experience success early, feel competent and have to understand the purpose, scope and properties of an object immediately if they are going to continue the interaction [1, 19].

To solve this issue, the canvas should give immediate audio feedback at any mutual touch, instead of giving feedback when touched at fixed positions only. This way, people get rewarded for their mutual touch instantly, they hopefully get a better understanding of the use of the canvas, and a search through music would be created, instead of the search for music that happened now.

The fact that people kept hold of the irregularly spread and hidden sensors once having found them activated a rigid, directed way of moving: people eventually started sudden movements from sensor to sensor instead of the intended dynamic explorative change of contact. Again with hindsight, we can see that we measure fragmented movements, as we only measure at small parts of the canvas. This fragmented measurement is translated into fragmented feedback and causes fragmented movements. Continuous measurement, throughout the entire canvas, should be translated into continuous feedback, and consequently motivate continuous movements. In addition, bodily contact should be mapped to music in a direct way: now, touching a small area at the correct position was translated in a rather rich sample. Touching a small area should result in ‘small’ music, e.g. a subtle and restricted sample.

The style of music and dance movements that people wanted to make sometimes interfered with the subtle, continuous and melodic movements required to make bodily contact. Movements logically are influenced by rhythmic sounds, so probably less leading rhythms in samples are required to induce fluent and subtle movements.

The use of hands only was probably caused by two reasons. Most people did not understand the canvas at all and started to explore with their hands, where most of our human manipulations start. They did not reach the stage of discovering the possibilities with more and other body parts, because even contact through hands was (seemingly) not rewarded. Second, the fragility of the frame and thickness of the canvas offered hands-only use: using other body parts to make contact through the canvas required both more stability of the frame (enabling to lean against it) and more ‘trans-tangibility’ of the canvas (enabling to feel the other person with more detail).

The pattern on the canvas was seen as a cue for interacting with the screen, which removed the focus from the other user. Since even a static, non-informative pattern attracted so much focus, this confirmed our decision to use audio feedback instead of visual feedback.

The initial explanation provided to the participants (‘you can make music together by making bodily contact with each other through the canvas’) seemed to influence their actions as well. People interpreted ‘make bodily contact’ and ‘make music’ in different ways, and therefore had differences in their focus on touch and music, which influenced their actions again. For example, one couple interpreted it as making music themselves, and started tapping the screen to create a rhythm.

Positive was that when people were aware of the musical change, they tried to keep in contact while moving. Playing a simple music sample proved to be rewarding enough to keep strangers touching each other for more than 5 min, and people stated that they would be willing to make more bodily contact with more body parts through the canvas.

The familiar and animated conversations between the participants after the evaluation were promising as well: there really seemed to be a common ground for conversation and follow-up meetings. People reported that they felt they had really ‘met’ someone, which was exactly what we intended. Although the interactive canvas hardly motivated the intended movements, and the intimate dynamic meeting choreography only scarcely took place, the overall user experience of getting acquainted to a stranger seemed to be there. Proof of the concept was evident, but a better developed design was needed, as we believe that the experience could have been stronger if users had performed the intended choreography of interaction.

4.5 Design recommendations

The former results and discussion aspects were translated into the following design recommendations for the next iteration:

  1. 1.

    The canvas should give musical output as soon as any bodily contact is made.

  2. 2.

    The sensor system should be able to detect bodily contact throughout the whole canvas.

  3. 3.

    The measured variables of bodily contact should be directly mapped to musical variables in such way that both are perceived as unity: the music of making bodily contact.

  4. 4.

    The style of the music samples should elicit movements that support and fit the movements of exploring bodily contact.

  5. 5.

    The canvas should have no visual patterns at all.

  6. 6.

    The assignment for the second test should be adjusted in such way that ‘making music’ and ‘making bodily contact’ would not be misinterpreted.

5 TouchMeDare prototype

As seen in the first implementation, the proof of concept is given for getting strangers acquainted with each other through bodily contact, by means of music. However, the interactive canvas needs to be further developed before it elicits our choreography of interaction properly. We believe that in case the installation was used with the right movements, it would create an even better experience. Further developments are described in the following iteration.

5.1 Installation design

The TouchMeDare prototype consisted of a canvas of 3 × 2.5 m, to allow more space to explore. The canvas was uniformly stretched by strong elastics on a robust metal frame of 4 × 4 m, in which all electronics and technology were hidden and protected (see Fig. 9). The increased robustness allowed people to lean against the installation and thus enabled the involvement of more parts of the body than hands alone.

Fig. 9
figure 9

The TouchMeDare installation, a canvas stretched in a metal frame, in which all electronics are hidden

Full size image

The sensor system incorporated a grid of 12 × 10 sensors of the same kind as in the first installation, with a size of 20 by 20 cm each. This grid covered the entire canvas (see Fig. 10). This sensor system allowed continuous detection of bodily contact, all over the canvas. Due to limitations in the sensor technology, and the necessity for a robust canvas, our intentions of making the canvas thinner were not fulfilled.

Fig. 10
figure 10

The sensor system, consisting of a grid of 12 × 10 sensors

Full size image

The new sensor system was able to detect different variables of bodily contact (e.g. contact surface size, contact position, contact shift), which allowed us to experiment: by choosing different variables of bodily contact and mapping these to different variables of music (e.g. volume, pitch), several possibilities could quickly be evaluated. Eventually, two different mappings were created, called A and B. Both mappings involved the following basic translations:

  • If only one person touches the canvas, a noise will play, indicating that the installation reacts to contact.

  • Once two persons make bodily contact through the canvas (and only then), the installation will immediately play music.

  • The amount of areas of mutual bodily contact creates the same amount of music samples: for each mutual bodily contact area, a new sample is played.

  • Activated samples are purely a result of bodily contact and explicitly not related to a location in the canvas.

Mapping A (see Fig. 11) focussed primarily on the amount and shift of bodily contact, which determined the musical composition. The content of the musical composition could be varied by the shift of bodily contact. The music collection consisted of nine families of music samples, each having one basic sample and eight variations of it. If an area of bodily contact shifted to a adjacent sensor in any direction, the initial sample changed into a more interesting variation of the sample. Shifting further changed the sample into yet another variation. The volume of the total music composition was determined by the total amount of activated sensors.

Fig. 11
figure 11

The coupling of bodily contact variables and music variables for mapping A, and the reaction this triggered from the sensor system

Full size image

Mapping B (see Fig. 12) focussed on variation possibilities of the shape of the bodily contact area. First of all, the stereo balance of each sample could be changed by a horizontal shift of each area of bodily contact (shift to the left changed the balance gradually to the left speakers, and vice versa). Secondly, the volume was linked to the samples separately: the size of each bodily contact area determined the volume of the sample.

Fig. 12
figure 12

The coupling of bodily contact variables and music variables for mapping B, and the reaction this triggered from the sensor system

Full size image

5.2 Evaluation of the prototype

The primary goal of this second evaluation was to learn the extent to which we had reached our aim of getting strangers acquainted with each other through bodily contact, in an enjoyable way. Since this is highly interdependent on the choreography of interaction and the interactive canvas, these two aspects were evaluated as well. Both mapping A and mapping B were evaluated, to discover the influence of different mappings on the elicited movements and experience.

The evaluation of the prototype was divided into a quantitative and a qualitative part. Quantitatively, we evaluated whether we had achieved our goal of getting strangers acquainted with each other through bodily contact in an enjoyable way.

Experienced enjoyment, meeting intensity and relatedness were measured, because we decided that these three aspects are indicators of an enjoyable meeting. Enjoyment is the amount of joy/interest someone experiences during an activity. Meeting intensity is the degree of closeness that someone experiences during a meeting. Relatedness is the experienced connectedness, and the feeling of ‘really knowing’ someone.

This resulted in the following quantitative research questions:

  1. 1.

    What is the experienced enjoyment of the activity for each of the mappings?

  2. 2.

    What is the experienced intensity of meeting through the activity for each of the mappings?

  3. 3.

    What is the experienced relatedness between the two persons after having participated in the activity?

In search for valid methods to measure the enjoyment of the activity, we found amongst others Attrakdiff [18], PACES [32] and IMI [38, 40]. Attrakdiff and PACES do not report to what extent people enjoy an activity, they report about products (e.g. attractiveness) or physical activities (e.g. physically feeling good). We chose to use the Intrinsic Motivation Inventory (IMI), which measures intrinsic motivation, with a subscale in measuring interest/enjoyment in an activity, by the means of a seven-statement Likert scale. An example statement is: This activity was fun to do.

To measure the intensity of meeting, we found the relationship closeness inventory [6], the subjective closeness index [6] and the inclusion of other in the self (IOS) scale [3]. The first two were not useful for our situation, since they test intensity or closeness of people who already know one another. We used the IOS scale and additionally a non-validated method developed by ourselves: a three-statement Likert scale with statements about the awareness of bodily contact, the feeling of cooperation and the unity of intentions and dynamics with the other person. An example statement is: I felt that I really co-operated with the other person.

The relatedness was measured by using a subscale of the above-mentioned intrinsic motivation inventory [38, 40]. This eight-statement Likert scale reports the relatedness experienced by a person after person–person interaction; it is often used in studies that have to do with interperson interaction, friendship formation and so on. An example statement is: I felt like I could really trust this person. As this scale measures relatedness between people, it is logically not applicable to strangers who have not yet met each other, as is the case in our evaluation. For strangers, there simply is no relationship and thus no relatedness score (or, one could argue, a score of 1).

Qualitatively, the acquaintance level, choreography of interaction and the interactive canvas were evaluated. Focus points were the understanding of the participants and their actual movements.

A combination of observations and interviews was used for the qualitative evaluation. For the observations, we recorded the users’ movements from two directions per user: from behind and from the side. Simultaneously, the sensor detection was recorded, allowing us to see the sensory pattern, triggered by bodily contact during the evaluation (see Fig. 13). At the end of each interaction, the participant was interviewed using a semi-closed interview setup, with questions regarding their understanding of the canvas, their opinion about the other person and the reasons why they made certain movements.

Fig. 13
figure 13

Observations during the second evaluation, consisting of video captures with the detected sensors as overlay

Full size image

The independent variable in the evaluation was the mapping, where a mapping without music output acted as the control condition. The dependent variables were the experienced enjoyment, meeting intensity and relatedness during the meeting activity. To remove learning and adaptation influences, the evaluation was executed in two groups: each group started with the mapping without sound output. Group A first worked with mapping A, followed by mapping B. Group B first evaluated mapping B, followed by mapping A (see Table 1). Before the evaluation, after each mapping and at the end of the evaluation, each user filled in a questionnaire. The users were randomly selected from a pool of self-reported festival visitors. Precautions were taken to prevent visual or auditory contact between the participants before and during the evaluation, to focus the interaction on bodily contact only. The camera setup was similar to the one in the first evaluation, except that an extra video camera was added. This way, each side view was recorded, since the installation’s frame was too broad to film two users simultaneously from the side with just one camera.

Table 1 Scheme with the order in the second user evaluation
Full size table

5.3 Quantitative and qualitative results

In total, 34 persons (who did not know each other) participated as couples in the user evaluation. The Pearson product–moment correlation coefficient analysis showed that there was no significant correlation in experience for people in the same couple (rsame_couples = 0.16, rdifferent_couples = 0.21), so all persons’ measurements could be analysed as individuals.

The questionnaires consisted of different validated Likert scales, except for the intensity of meeting attribute, which contained three non-validated scales. A reliability analysis across all factors was conducted. The resulting Cronbach’s Alpha for the enjoyment (0.896) and intensity of meeting (0.690) gave us sufficient confidence in the reliability of the questionnaire.

Next, we will discuss the quantitative results for each of the research questions.

1. What is the experienced enjoyment of the activity for all three mappings?

First, the average value for the enjoyment attribute was calculated, per user, per mapping. These values were then used to calculate the average enjoyment per mapping. To discover if the mappings had significant differences in experienced enjoyment, dependent group T-tests were executed. A significant difference (t(33) = −2.09, p = 0.04) existed in experienced enjoyment between the activity without musical output (mean = 4.3, SD = 1.10) and with musical output (mean = 4.6, SD = 0.89). No significant difference (t(33) = 1.92, p = 0.06) existed in experienced enjoyment between mapping A (mean = 4.5, SD = 0.99) and mapping B (mean = 4.8, SD = 0.92).

2. What is the experienced intensity of meeting of the activity for all three mappings?

Again, the average per mapping was calculated, followed by dependent group T-tests. No significant difference (t(33) = −1.45, p = 0.16) existed in experienced meeting intensity between the activity without musical output (mean = 3.9, SD = 0.98) and with musical output (mean = 4.1, SD = 0.98). No significant difference (t(33) = 0.40, p = 0.69) existed in experienced meeting intensity between mapping A (mean = 4.1, SD = 1.04) and mapping B (mean = 4.1, SD = 1.00).

3. What is the experienced relatedness between the two persons after having participated in the activity?

Relatedness was measured only at the end of the evaluation, and so no comparison was available. The experienced relatedness (mean = 3.9, SD = 0.86) at the end of the evaluation was relatively high on a scale from 1 to 7, especially for people who had just met each other (and had never seen or talked to each other). For comparison, a recent study [28], where relatedness was measured within college classes, showed relatedness scores between 3.5 and 3.8, depending on the class and school.

These results show that the addition of (the ability to create) musical output positively influenced the experienced enjoyment. This addition, however, did not significantly influence the meeting intensity: apparently, the act of touching each other through the canvas created a high meeting intensity already, independent of the creation of music. Finally, the difference between the mappings showed no significant influence on the user experience. This was in sharp contrast with our expectations, as the two mappings differed quite a lot in musical output, so we had expected that this would influence the user experience and movements.

In addition to the quantitative results, we now shift to the qualitative results. We observed that people were more focussed on the other person while interacting with TouchMeDare than with the concept demonstrator. In some cases the focus and effort of following and touching the other person was so high that the participant was barely aware of the music and musical changes. In addition to the change of focus, people involved more body parts to vary bodily contact and sometimes even used their whole body by leaning against the canvas and the other person. However, the continuous, smooth and ‘stroking’ movements, in the search through the music, were still missing: there was no clear focus on making multiple and shifting areas of bodily contact, as was required to properly influence the music. Instead, a more random cooperation existed, resulting in more random musical output as well.

In the interviews after the interaction, we discovered that people’s understanding varied: some were unaware of their influence on the music, while most understood that they influenced the music, but did not know how. The link between more bodily contact and more music was made several times, but the influence of movements on e.g. the volume or sample sort was unnoticed. The ignorance of users about their influence on the music seemed odd to us, as the music only started when mutual touch was made. Krueger [25], who did a lot of exploration with strangers interacting through virtual, projected screens, noticed something similar: ‘Some people do not listen to what their experience is telling them. They move their arms in a herky-jerky manner, rather than synchronising their movements with the rhythm of the current feedback relationship.’ Most probably, according to him and us, the effort and experience of interacting in such a novel way is so large that the feedback is simply ignored. And indeed, when we told the participants after the evaluation how the canvas really worked, most immediately started to interact again, and with hindsight clearly noticed their influence.

With the manipulation possibilities of music not being evident to people, one interesting observation was made: we were now able to look at the way people made bodily contact through the canvas with their own ‘natural’ movements. Possibly, variables in movement of which we were unaware could be found this way. Indeed, a follow-up analysis by Vegt [43] showed that people clearly used both hands as continuous reference points to keep ‘in touch’ with each other. Different bodily contacts were explored, by adding different body parts, but the hands always kept touching in order to keep in contact with each other.

Participants were also asked for their opinions and thoughts about the other person, e.g. what gender would the other person be, what sort of character, etc. Surprisingly, people often gave a specific and detailed description of the other person, which sometimes did match, but often did not. Things were said like ‘I think that the other person would be a great teammate to work with’. The fact that people did express such detailed characteristics was interesting in relation to our goal of meeting without visual prejudice. After the evaluation, both participants were brought into contact with each other, and their initial reaction and further conversation were observed. People acted like they knew each other well; they kept, just as in the first iteration, sharing experiences and kept talking for 15 min, until we asked them to leave.

6 In-context case study

Lowlands is a 3-day Dutch festival, with over 50,000 visitors, showcasing various activities including music, cabaret, literature and art. For a major part, the experiences of the visitors depend on the atmosphere at the festival; this atmosphere is partly based on the organisation and its events, but the social interactions among the festival goers themselves play an important role. Stimulating the visitors to get acquainted with each other in an expressive way fits exactly with such an atmosphere.

The TouchMeDare prototype was placed in Lowlands 2007 (see Fig. 14), as a first in-context implementation. We evaluated this case study by observation, informal interviewing and meeting with people through TouchMeDare ourselves.

Fig. 14
figure 14

The TouchMeDare installation at the Lowlands Festival, where a case study was conducted

Full size image

In Lowlands, people used and enjoyed TouchMeDare as an excuse to literally get in touch with strangers. People acted in a rough manner: they hit, bumped and jumped into the canvas, which was in stark contrast with the prototype evaluation at the university under laboratory conditions.

Corresponding with the preceding evaluation, people showed minimal understanding of how they could manipulate the music through bodily contact. People stated to have the impression that the music would play by making bodily contact through the canvas, no matter how they would do this. They understood that the music would be louder and richer if they made more intensive contact through the canvas. But again, they expressed no understanding of the required kind of intensity; they, for example, just pushed and hit harder.

Four different sets of music samples were created by three DJs. Each of these sets created a specific atmosphere, e.g. the movements and ambience with a world-music set were totally different from an experimental set. In the world-music case, which contained a lot of rhythmic samples, people started fast, almost exotic dances, while touching the screen and the other person at the same time. The experimental set, which included vague melodies and voices, evoked a lot of slow movements, with people dropping to their knees while leaning against the screen. These observations not only strengthened our earlier conclusion that the rhythms of the samples influence the movements, but also that the ‘style’ of the music heavily influences the movements.

We also observed how the installation functioned in this busy context, having multiple people taking part at a time. The focus on making bodily contact through the canvas sometimes changed, since participants were also in contact with people standing next to them and with spectators around the canvas. Group conversations with strangers, people running around the canvas to look at both sides, and spectators who were directing the participants to move in a certain way, together created a friendly and open community atmosphere. During the festival, people returned on multiple occasions, often dragging others along to interact with, and asking strangers to join them while they were interacting.

Subsequently, due to this mass-usage, understanding of how the canvas worked became more difficult. Individual changes were hardly audible, as so many people (and thus samples) interacted at the same time. For festival contexts, the usage should be restricted to one person on each side or, more suitably, the feedback mapping should be altered.

7 Discussion

The study presented in this paper raised some points for discussion, including evaluation methodology, installation design and afforded meeting activity aspects. These three points are discussed below.

7.1 Methodology

In our search for the appropriate assignment of (the focus of) our evaluation, we argued where the optimum is: between (a) being able to measure the actions that are offered by the canvas, and (b) being able to measure the meeting effect of the meeting movements we intend to offer by explaining people how the installation works. This proved to be a major complication, but one that would be useful for future evaluations. During the second evaluation, the assignment was adjusted three times. In the first assignment, people were told that they could explore each other with multiple body parts through the canvas. Most participants were unaware of their influence on the music and interpreted the assignment in abstract ways. In the second assignment, we added that they could create music by exploring each other through the canvas. Now, most participants started to use the installation as a percussion-like instrument, e.g. by slapping on the canvas, again being unaware of their influence on the music. In the final version of the assignment, we added a movie of two persons performing the intended way of making bodily contact. This steered people to make the intended actions, but did not tell them what the exact influence on the music was. This assignment worked reasonably well, so the study was reset and continued with this combination of text and video. Measuring the effect of our intended meeting activity, while at the same time evaluating the affordability of the canvas, appeared to be an inevitable complex entanglement of interaction design elements.

As a second issue, the order of with and without music conditions was not randomised in the prototype evaluation. We expected to find significant differences in experience between the two different mappings. Evaluation without music was intended to function as a control condition and was not intended to measure the general difference between with and without music. Unexpectedly, no significant differences were found on comparing the two mappings. Therefore, we looked at the differences between the conditions of with and without music, even though the order was not counterbalanced.

7.2 The affordance of the meeting

User evaluation of the meeting showed that participants experienced fun and collaboration by means of bodily contact and music, and that they reached a relatively high level of relatedness. Music positively influenced the feeling of fun and collaboration, but remarkably users were not aware of their exact influence on the music or of their influence at all.

From the observations, questionnaires and interviews, we can conclude that our main goal has been achieved: people got acquainted with each other through bodily contact, in an enjoyable way. However, it might seem that our intended (and not yet provided) physical and dynamic aspects of bodily contact were not needed to reach the effect of getting acquainted. Perhaps, the act of interacting through a canvas was sufficient to bring strangers into close contact. We argue against this, for two main reasons. First of all, the thought of making music together provides an excuse for participating and for touching another person. Just as asking someone for a dance can be an excuse for getting close to a person, so can the thought of making music together be an incentive to start participating. Second, we believe that the level of specific variations in movements (e.g. the stroking movements required to alter the music samples) provide nuances in the acquaintance making. We believe that the meeting activity would become even more specific and personal, if we could reach the intended level.

7.3 Challenges in future work

Looking back on both evaluations and the case study, we gained several insights that might be challenges to future work.

A large effort is required to interact with another person in such a direct and intimate way. Focussing on the other person and maintaining bodily contact requires so much attention that the feedback given is often ignored. A possible solution for this is to make the feedback more distinctive. In the case of sound feedback, we would have to think in the direction of continuous tones or other basic elements, as rich music samples proved to show indistinguishable differences.

We found that the balance between a meaningful mapping and eliciting the intended movements is a complex trade-off. The mapping, which basically is the translation of movements into music, should ‘guide’ the movements in the intended way. Before this guidance can take place, certain correct movements have to be made in order that the mapping is understood. In more concrete terms, this means that users only get feedback on action and have to gain understanding about the product through trial and error. This is necessary to keep the interaction exploratory and discovering, since concrete feedforward could eliminate this. The main challenge is that the feedback should be given in such a way that it will also serve as a kind of abstract feedforward to guide the user into the right movements. If the initial mapping is too simple, people will soon know everything about the canvas and the discovering will diminish. In the current situation, the mapping was too complex: people did not understand anything at all. We believe that the key point of this balance is the coupling of the music directly to the movements: one type of movement should have similar effects on the music. Implementing this proved to be difficult, because the canvas was not designed to be an instrument. People should not be making the music themselves. Instead, they have to activate samples, composing music. We still do not know how to do this exactly, but we believe that basic, simplistic samples are a solution: the resulting music might be less ‘interesting’, but we expect the user experience of composing that music to increase.

Participants felt the concept demonstrator to be fragile and therefore limited their movements. In contrast, the TouchMeDare prototype was robust and appeared robust as well. It seemed to encourage rough behaviour, with people jumping into the screen and hitting it. Probably, the installation directly offered this: people could not see it as an installation-accommodating technology; they just saw strong frames with a taut, trampoline-like canvas. Mueller et al. [34] experienced the same in their remote impact game, where people had to make full body contact by jumping into a sensitive wall to ‘shadowbox’ over a distance. In our situation, where delicate and gentle movements were intended, we should adopt a middle course: the installation should offer these gentle and delicate movements. Perhaps faking fragility or showing the electronics could help to induce this behaviour.

We believe to have met a rich array of interaction design issues in this project. Even though we have gained many insights, we also feel that we have just got started. A clear view of the project shows that design is not about separate issues, but the essence lies in the way all these issues come together.

8 Conclusions

The design study presented in this paper aimed at getting people acquainted with each other in an enjoyable and exploratory way. The TouchMeDare installation was designed for a meeting experience, using choreography of interaction, which should lead to a physical product. The result was an interactive installation, which should elicit dynamic exploration of mutual bodily contact while using the canvas as a mediator. The installation consisted of a flexible, opaque canvas that played different compositions of pre-recorded music samples, based on the variation of bodily contacts. In a series of design research iterations, this installation was developed and evaluated.

The evaluation and the case study showed that people did not yet dynamically compose and explore the surface of contact with the other person in the way we had intended in our choreography of interaction. The most difficult part was mapping bodily contact variables to music variables. It appeared difficult to use and change natural human movements in relation to music, e.g. rhythmic samples elicited tapping on the canvas. Therefore, participants only got a basic understanding of how bodily contact influenced the musical composition.

Although participants did not always interact in the intended way and the interaction was not completely understood, they experienced an enjoyable meeting. Participants experienced a high level of relatedness and reported to have clear images of each other: describing gender, character and personality traits. Summarising, we can conclude that we reached our goal to get people acquainted with each other.