In this paper, we compare the use of physical and digital sticky notes in collaborative ideation. Inspired by a case study in a design company, we focus on a collaborative ideation task, which is often part of pair-wise brainstorming in design. For comparison and to focus on the different materiality, we developed a digital sticky notes setup designed to be as close to the physical setup as possible, not adding any advanced digital features, even though technology has reached a stage where more sophisticated use of digital sticky notes on digital boards is possible. In this paper, we present a study of ideation among pairs of experienced sticky note users. The ideation sessions were video recorded and analyzed to focus on how collaboration is supported across the two setups. Based on quantitative analyses of the participants’ interactions with the artefacts, talking patterns, position and attention during the sessions, we qualify how the differences and similarities between the 2 setups have an impact on note handling, ideation techniques, group dynamics and socio-spatial configuration, e.g. the use of the room, the boards and tables. We conclude that, while the physical setup seems more appropriate for creating notes and posting notes, the digital setup invites more note interaction. Nevertheless, we did not find significant differences in the ideation outcome (e.g., number of notes created) or how participants collaborated between the 2 setups. Hence, we argue that collaborative ideation can successfully be supported in a digital setup as well. Consequently, we believe that the next step in a technological setup is not an either or, but should bring the best of the two worlds together.
1 Introduction
In professional design companies, research and student groups, as well as many others, sticky notes (or Post It® notes) are often the first choice for collaborative ideation. Collaborative design methods extend from future workshops (Kensing and Madsen 1992) to e.g., affinity diagramming or paper prototyping (Beyer and Holtzblatt 1997). Within these methods, sticky notes are used for externalization of ideas, or representation of requirements, among other things. The inexpensive and disposable nature of sticky notes is in part what makes them valuable as a design tool and their tangibility is highlighted as one of their inherent qualities (Harboe and Huang 2015). Despite these qualities, sticky notes have well-known shortcomings, limiting their use in professional design processes (see also Jensen et al. 2018). These limitations include; (i) poor support for documentation, (ii) moving a board with sticky notes can throw the work in disarray, and (iii) no support for remote collaboration on the same board.
Many contributions in the HCI field have attempted to overcome these limitations by emulating (Geyer et al. 2011a, Hilliges et al. 2007, Tse et al. 2008), augmenting (Probst et al. 2011), or digitizing (Klemmer et al. 2001) sticky notes. However, physical sticky notes are still the most prevalent material in design processes (Harboe and Huang 2015). This may be because the digital systems proposed so far; (i) offer additional digital features that eventually introduce complexity to the user, (ii) require an extensive setup of cameras, displays or projectors or (iii) require specialized hardware with a potential risk of long-term hardware deprecation, or simply because the digital medium cannot replace the physical in this particular work situation. Many of these setups share issues similar to those experienced in classical meeting-room setups, known from CSCW–challenges with multiple authors, records over time, limitations in resolution and structure of the boards and surfaces upon which the brainstorm, design and notetaking happen (see e.g. Cognoter (Stefik et al. 1987) and Tivoli (Pedersen et al. 1993)). In contrast to this body of previous work, our intention is not to fully, or even partially, resolve the problems and shortcomings of sticky notes nor of digital systems supporting (co-located) group ideations (cf. Nunamaker et al. 1996) by designing and evaluating potential replacements. Instead, we are interested in the impact of the material (i.e., physical paper in contrast to digital applications) on interaction patterns and collaboration in ideation contexts. In that regard, we also refrain from claiming that sticky notes are the ideal tool for ideation. Rather, based on the fact that they are a popular choice for design processes and ideation, and a considerable amount of literature exists on them, we use sticky notes as a concrete example to investigate differences in affordance and use of digital and physical artefacts for collaboration (see also Badram and Houben 2018). As design is a ubiquitous, collaborative and highly material activity (Vyas et al. 2009), our motivation for doing this work is to understand how changing the material used in a design process changes the design and collaborative practice, and whether these changes explain the limited use of digital sticky note systems in current design teams.
Anzeige
In this study, we make use of a setup of physical sticky notes and a no-frills1 digital setup, using modern off-the-shelf hardware (Jensen et al. 2018). In order to be able to compare the two setups, we focus on a rather straight-forward task where groups of two designers collaborate to generate ideas in response to similar collaborate design tasks, using sticky notes over 3 phases; a brainstorming phase, a clustering phase and a labelling phase. We are inspired by Jornet and Roth (2017), who study a collaborative design case through the lens of what they call ‘the irreducible unit of thinking and communication’. They focus on the conceptual design trajectory that happens in such processes by looking at the history of the production of descriptions. In our case, the sticky notes are the important feature in the design trajectory. Sticky notes are produced, moved and discussed by the team as the design process unfolds.
With this work, we contribute to a detailed understanding of how physical and digital material can both support and influence collaborative ideation. To explore the similarities and differences between the use of physical and digital sticky notes, we use video recordings to analyze in detail how participants create the notes, place them on boards, move and group them, while carrying out their irreducible unit of thinking and communication among themselves. We further investigate how participants move, talk, and where they focus, across the two setups in order to understand how the digital and physical affect collaboration. We present our findings as a combination of statistical analyses of the various interaction phenomena, and concrete examples from the video material. This analysis is a continuation of the work presented by Jensen et al. (2018). However, where their focus was on the different user experience between the 2 kinds of material unfolded from retrospective qualitative interviews, the focus of this paper is on how collaboration is facilitated and appropriated in each of the two conditions through video analysis. Both papers provide insights on the influence material (as the only variable) has on interactions, collaboration and user experience from a comparative point of view, which is relatively underexplored in current literature. In this paper, we study 12 quantitative metrics, which we believe are definitive for creative sticky note collaboration, showing differences and similarities of actual use between the 2 materials. With this research, we hope to support future work in digitizing tools for collaborative ideation and creativity, in general.
1.1 Background
The motivation for this study emerged from the CIBIS project (http://pit.au.dk/research-projects/cibis-creativity-in-blended-interaction-spaces/) where we, with a large number of collaborators, have studied creative and design processes both among professional designers and among high-school students. Analysing design thinking, and the forms of collaboration and materials supporting this, has been a main focus in the project (Dove et al. 2017, Biskjaer et al. 2017, Hansen et al. 2017). Along with exploring the blends of physical and digital materials–be these sticky notes, Lego bricks or other design materials (Bødker and Klokmose 2016; Klokmose et al. 2015).
Specifically, a participatory design case with an international design company (Dove et al. 2017, Jensen et al. 2018) worked with potentials and barriers in digitizing current use of physical tools and materials (e.g., paper, whiteboards, sticky notes) for collaborative work. As discussed by Jensen et al. (2018), the participating designers have repeatedly stated that tangibility and materiality are essential for their design processes and should be maintained in future systems. However, due to the physical nature of the sticky notes, the designers experienced difficulties with documenting procedures and results of design sessions that employ sticky notes, as well as distributing and transferring boards with sticky notes to new locations. By observing their use of sticky notes during workshops and analyzing videos from internal design sessions at the company, it was unclear why the tangibility and materiality of the sticky notes were considered important and indispensable for collaboration and externalizations of design suggestions. The designers did not seem to utilize the physicality of the sticky notes in particular. Therefore, we designed a digital setup that aims to be as close as possible to their physical setup as a means for investigating the impact on the design process, in particular the collaboration and interaction between designers, that follows from a digitization of (one of their design) tools. From this we investigate insights on the presumed advantages of physical sticky notes and what merits the digital version might bring. We do not argue that sticky notes are the ultimate design tool nor that it is impossible to create a digital system that is superior. However, since sticky notes are ubiquitous in design processes and there have been a plethora of attempts to digitize sticky notes (often providing additional functionality or a cumbersome setup) with limited success, our aim with this paper is to highlight any obstacles or issues that are inherent within the digital or any non-replicable advantages of the physical paper notes.
Anzeige
For our study setup, we use Jensen et al.’s (2018) use case, where two designers conduct an ideation session for the relaunch of a company website using sticky notes. Both designers create notes at the table, some contain ideas that emerge from their discussion and some state requirements from the client that they look at on a laptop. Whenever a sticky note is created, it is posted on their shared whiteboard. When they are satisfied with the notes on the board, they start to cluster them in groups and eventually these groups are given titles, which are written on the whiteboard with a marker. The two designers often distribute the work between them to focus on different elements of their project. We are aware that this is just one of the many design-related uses of sticky notes, however, as this was a concrete example observed within a design company, we decided to use it to structure the user study presented in this paper, as a basis for comparing physical and digital sticky notes as design tools.
1.2 Collaborative Ideation
In this work, we specifically focus on collaborative ideation that takes place in the context of a creative task where ideas are generated, sorted and specified. With the term collaborative, we refer to a setting where two or more persons (in this study two) work together in a joint effort as opposed to competing or challenging each other. In this case, the people are co-located in one room. The spatiality of the collaborative ideation is not restricted, meaning that they are free to move around and use the entire room. How the room is actually used and the role of spatiality in collaborative ideation is part of our analysis in this paper.
In the context of this work, a collaborative ideation task consists of a set of activities involving the generation of new ideas and solutions that are then organized and assessed regarding their potential and usefulness to extend or improve an existing service or product. Being common practice in design processes (Harboe and Huang 2015), (sticky) notes were used to document ideas, issues and thoughts.
We explore collaborative ideation as an activity that involves 3 subtasks; brainstorming, clustering and labelling. In the beginning, ideas that involve solving, extending or improving the task at hand are brainstormed (= brainstorming). With reference to Rietzschel et al.’s work (2006), collaboration in brainstorming can mainly take two forms. In one, participants work together closely, where ideas are presented, discussed or contested before or while writing them down. This first type is referred to as interactive brainstorming. In the second, nominal brainstorming, participants are mostly silent during the actual brainstorming and create ideas separately from other team members. According to an intuitive understanding of the term, there is not much collaboration in nominal brainstorming.
All documented ideas and thoughts in the form of notes are collected on a shared, vertical board. Posting the notes to the board can either take place during the brainstorming process, for instance whenever an idea has been documented on a note, or after the brainstorming. For the latter, notes would have been collected individually and particularly in the case of nominal brainstorming privately in the sense that they have not been shared before. For this analysis, we are interested in the nature of collaboration, meaning how, when and for how long people interacted with each other (i.e. group dynamics and turn-taking) but also with the artefacts provided (i.e. the board, physical and digital notes). In that respect, it is also interesting to analyze the process of posting, which could to some extent be understood as transitioning from a private to a shared state. This transition does not necessarily take place when notes are first posted on the board. Literature has documented instances where individuals prefer to maintain a private space when interacting with an otherwise shared interface (Greenberg et al. 1999; Whalen 2003). In a collaborative brainstorming scenario, such as the one we are focusing on here, notes would in that case be posted separately from each other (e.g. creating individual zones) and the actual sharing would then happen in the next phase.
After brainstorming, the ideation consists of sorting and clustering into groups, on the board, following a bottom-up approach. In the third phase, notes are then labelled and associations between them drawn. These 3 sub-tasks do not necessarily happen sequentially. They can be intertwined, iterative or take place simultaneously/in parallel. For instance, one group member might choose to post notes right after creation, while the other(s) keeps them to her/his self. Or the brainstorming and clustering can be merged, when posting the notes in groups from the start. We are interested in how collaboration unfolds in creative design processes and how the physical versus digital material, in this case sticky notes, affect collaboration. This paper further explores what role notes play during collaborative ideation and whether their mediation affects the nature of the collaboration or the outcome of the process (i.e. structure, interactivity).
2 Related Work
CSCW has a long tradition of addressing meeting rooms (virtual or augmented), shared documents and design processes as example collaborative processes. Specifically, Herrmann (2008) discusses computer/meeting room supported creativity in collaborative ideation or brainstorming. He discusses the role of materials and digital components in a variety of creative processes. Among other issues, he discusses the malleability and openness of the material, which represents design concepts and how this malleability is essential for dynamic idea generation. He addresses turn-taking and how the technology should invite people to make changes without being afraid that these changes may have destructive effects. His design heuristics point to features and options supporting creative ideation that are only feasible (or easier) in digital setups. Herrmann (2008) makes reference to the classical meeting-room setups, known from CSCW, such as Cognoter (Stefik et al. 1987) and Tivoli (Pedersen et al. 1993). The iLand project similarly worked early on wider landscapes of creativity (Streitz et al. 1999). In our current work, we share this focus on the material and digital components and in particular, how collaboration is happening with the material. We look at situations where groups, who are experienced in using sticky notes for brainstorming, do so, either with physical or with digital sticky notes, in a meeting room with a shared large surface (either a whiteboard or a shared screen). This form of collaboration and turn-taking over shared materials (i.e. sticky notes) further relates to a classical CSCW issue, namely the differences between synchronous vs. asynchronous communication that dates back to DeSanctis and Brent Gallupe (1987). In the context of analysis for this paper, we use the term turn-taking as referring to how and when users take turns handling notes and talking (i.e. speaker shifts). Turn-taking therefore describes the division of the interaction and conversation between users, and their dominance (or prominence) in each.
Greenberg and his collaborators have done extensive work on shared workspaces and toolkits, in particular Gutwin and Greenberg (1998) focus on trade-offs between individual and group space in collaboration. In Greenberg et al.’s (1999) work with SharedNotes, they observed how people move from individual to group work, how personal artefacts and public artefacts are handled and moved back and forth. In our current project, by considering the placement of notes we will further look at how sticky notes are sometimes shared, sometimes not, as well as the ownership of the notes that have been put in public on the board, and/or have been activated in the discussion.
We are certainly not alone in deploying shared digital screens in such settings, and in trying to understand how they work in collaborative situations together with physical materials. In recent years, we have seen work on collaborative tabletops (Geyer et al. 2011b), blended interaction (Jetter et al. 2014) and also distributed teams who collaborate around blended interaction spaces (O'Hara et al. 2011; Broughton et al. 2009). Wellner (1993) did some of the earliest work on paper and digital desks. Mackay et al. (1995) worked with paper as an interface to collaborative systems, and Rekimoto and Saitoh (1999) worked with shared continuous work space to examine interaction techniques for hyper-dragging between paper and virtual representations. The Designer’s Outpost combines physical sticky notes and digital technologies (Klemmer et al. 2001). The system allows users to manipulate sticky notes with physical interactions, which are then used as input for a digital display through camera tracking. Quickies presents a different approach to digitizing the physical sticky note (Mistry and Maes 2008) where digital pen hardware is used to capture what is written on notes, so that the data can be processed and searched, particularly for digital reminders.
With regard to usage of space in collaborative settings, the literature points towards the necessity to consider how privacy-related themes influence behavior. Morris et al. (2006) worked with collaborative gesturing for co-located groupware, and several authors have worked on the boundaries between shared and individual artefacts in collaborative sessions: While large vertical displays are mainly used for public information, small handheld devices (e.g. tablets) tend to be perceived as private artefacts with which only one user interacts (Magerkurth et al. 2004). The latter are predominantly used to enter or administer private data or enter private commands. Whereas studies on the use of tabletops suggest a demarcation of individual and shared space on displays (e.g. Whalen 2003). In their work on the socio-spatial aspects of computing, Krogh et al. (2017) argue that every technology design has a spatial agency that can influence socio-spatial configurations, e.g. how people move around, position and orient themselves, an aspect that we will look at further in our analysis.
In our work, we are also inspired by the many studies of detailed handovers from ethnomethodology and conversation analysis, that are familiar to CSCW, e.g. (Goodwin 1994; Heath et al. 1993). Heath et al. (1993) present a foundational example of such detailed studies of collaboration, hand-over/turn-taking through artefacts (which in this case could be sticky notes), pointing towards the need and desire of CSCW to understand technology-mediated interaction and collaboration at this minute-to-minute level. In relation to turn-taking, we further look at how the digital versus physical material affects alternation between individual and group idea generation, as mentioned by Korde and Paulus (2017). Our work is also influenced by Tang et al. (2011) and Chulvi et al. (2017), who investigate how moving from a physical to a digital setup influences creativity in different sketching environments, and by work on artefact mediation, and affordances for interaction at large (Bærentsen and Trettvik 2002, Gaver 1992, Badram and Houben 2018).
Similarly, we build on research addressing how artefacts are used to discuss and generate ideas. Bødker (1998) discuss representations in design in general and several later papers have discussed the use of game pieces in room settings (e.g. Bødker and Buur 2002), scenarios (Bødker and Christiansen 1997), inspiration cards (Halskov and Dalsgaard 2006), situation cards (Mogensen and Trigg 1992)), pointing towards the importance of the materials and materiality in these creative design activities. Hartmann et al. (2010) propose a setup for creative collaboration where physical and digital pictures can be combined and annotated. In our particular case, we focus on a setup where we look at a design process, mediated by sticky notes within a design team of two, collocated designers, and address in particular the role of the sticky notes in this process. In the current study, we are interested in the different phases of collaborative ideation (brainstorming, clustering and labelling) and the passing and handling of sticky notes in these phases. Are there different strategies for when to post notes during e.g. brainstorming, and is there a difference between the digital and physical material in this? How do groups post and create notes, how and when do they post them to the board, how is the note creation of the two interleaved, and how do they discuss the notes? In general, we investigate how the material affects these questions.
3 Digital Sticky Notes
To compare how physical and digital sticky notes affect collaboration between people in idea generation sessions, we used the digital sticky note setup as designed and developed by Jensen et al. (2018). The digital setup, presented in Jensen et al. (2018) and used here, is designed to be as close as possible to the physical setup of a collaborative ideation session, where users can use sticky notes, pens and markers and a (white)board. In this no-frill implementation, there were choices made and the two set-ups are different, as we shall see. The digital system is designed with off-the-shelf technologies and has no complicated hardware installation requirements, e.g. no camera-tracking nor projection mapping setup. This was to make installation as straight-forward as possible and to lower the barrier of integrating digital sticky note technologies in everyday practices.
The setup consisted of two main parts (see Figure 1); a web application, called NoteCreator, running on iPad Minis for creating sticky notes (corresponding to pads of physical notes) and a web application, called NoteCanvas, running on a Microsoft Surface Hub for posting and sorting notes (corresponding to the whiteboard). Both web applications were developed using CodeStrates (Rädle et al. 2017) on top of the Webstrates platform (Klokmose et al. 2015).
Figure 1.
Left: The NoteCreator interface on an iPad Mini. Right: The Note Canvas interface on a Microsoft Surface Hub
×
The NoteCreator presents a simple layout to the user. They can create new notes, pick the note color, draw on the notes (either using finger or a capacitive stylus), select line color, delete notes and flick through already created notes. To stay as true to the physical setup as possible, changing the color of the note is only possible before the user starts drawing and strokes cannot be deleted, meaning that users have to create a new note if mistakes are made.
The NoteCanvas is a full-screen, white canvas, where users can pick pen-color, draw on the canvas (using the Surface Pen) and erase strokes using their fingers (as on a traditional whiteboards). Users can move and rotate notes that are posted to the canvas. The digital notes have a virtual adhesive area corresponding to the physical notes, making it possible to move and rotate several overlaying sticky notes. Users can still write on the notes once they have been posted to the canvas.
3.1 Posting and Pulling
Posting from the NoteCreator to the NoteCanvas (see Figure 2) is done in the following way: First, the user finds or creates a note intended to be posted. Then a long press on a clear space on the NoteCanvas, enables a “post”-button on the NoteCreator interface. By tapping this enabled button, the note will transfer from the NoteCreator to the current finger position on the NoteCanvas. This design was driven by two requirements: 1) We wanted the user to be within arm’s reach in order to post, as with physical notes, and 2) we wanted the intention (the long press enabling a post) to happen in the users’ shared space, i.e. on the Microsoft Surface Hub and not on the individual iPad Minis. A pull can be performed in a similar way, as a long press on a note posted on the NoteCanvas, enables a “pull”-button on the NoteCreator interface, which upon clicking will remove a note from the canvas.
Figure 2.
A sequence illustrating how to post a digital sticky note using the proposed system.
×
3.2 Tradeoffs and Restrictions to the Physical Setup
Mapping the use of physical sticky notes in collaborative ideation sessions to a digital setup required some tradeoffs in the design, as well as the introduction of a few restrictions to the physical use.
The first tradeoff is the pen switching required in the digital setup, as a result of the iPad Mini and the Microsoft Surface Hub not supporting the same styluses (passive versus active). With physical sticky notes, board markers can be used to draw on the notes as well as on the whiteboard. The second tradeoff is that the decreasing stickiness of physical sticky notes was not implemented in the digital setup, i.e. that the adhesive area gets weaker during use, eventually causing the note to fall off the board. We decided not to implement this behavior as it (1) might be incomprehensible to users, (2) is unclear where a fallen digital note should go, and (3) is complex to simulate when an adhesive area gets too weak to stick. The third tradeoff is posting and pulling sticky notes to and from the board. In the physical setup, the notes are tangible and can be grasped, which makes these actions embodied and natural. However, to transfer the tangibility to the digital version, we would have to develop small embedded notes with an active display, pen input sensing, and the ability to stick to a wall, or use a camera and projector setup allowing for projection mapping. These options went against our initial design goals of using off-the-shelf technologies and a simple setup, and hence, we decided on the posting and pulling operations as described above.
Further, to enable a comparison between physical and digital sticky notes, we added a few restrictions to the physical setup. First, we limited the whiteboard by applying a frame to it with the same size and in the same height as the Microsoft Surface Hub. Second, the users were only allowed to put sticky notes on the whiteboard, on themselves or back on the pads, and not on the table. This restriction was included to enable a fair comparison with the digital setup. If we wanted to allow users to place sticky notes on the table, it would require an implementation of a tabletop interface in the digital setup as well, thus increasing complexity. Further, if this restriction is the main difference between the physical and digital setups, we were confident that the study participants would emphasize this, as we only recruited experienced sticky note users.
3.3 Caveat
The mapping from the physical to the digital can never be completely direct and numerous design choices have been made. Comparing a physical practice with a digital setup is then, to some degree, a comparison of apples and oranges. However, we believe the comparison is important in order to understand why the physical notes are still prevalent in professional design processes, despite their shortcomings (as listed in the introduction). Using the experimental vocabulary as presented by Hornbæk (2013), we want to compare our key ideas (constructs), namely, the use of physical and digital sticky notes in ideation sessions. In this section, we have explained how we have operationalized these constructs into comparable and representative conditions (the physical and digital setup) by a number of design choices, tradeoffs and restrictions. We further argue that we are not presenting a “Straw Man Comparison” that only compares new technology to outdated work or processes (Munzner 2008), but that the physical setup is a strong baseline, as the use of physical sticky notes is still ubiquitous in design methods.
4 User Study
To understand how the choice of material (physical or digital) of sticky notes affects collaboration, we conducted a comparative study using a within-subject design. In pairs, participants used both physical sticky notes and our digital sticky notes setup. The conditions (physical and digital setup; order of tasks) were counterbalanced using a Greco-Latin Square to avoid carry-over effects.
Jensen et al. (2018) work with the same empirical study as we do here, to the extent that they analyse the same pairs of users, working in the two settings described. However, the previous paper primarily presents a qualitative, comparative analysis based on post-session interviews with participants, combined with a presentation of the digital set-up. Jensen et al. (2018) use the activity theoretical Human-Artefact Model to highlight expected, and unexpected, differences and similarities between the setups when it comes to the handling of sticky notes and the underlying setup. Jensen et al. (2018) conclude that: “Despite users’ general preference for creating physical sticky notes, the digital technologies have reached a point where easier handling of digital sticky notes on interactive screens and the potential of proper design documentation makes it an alternative to their physical counterparts.” With the analysis presented in this paper, we focus on measureable collaboration metrics using video analysis of what the users actually do (in contrast to how they feel, as explored in Jensen et al. 2018) to understand how the two materials (physical and digital) influence how experienced sticky note users collaborate.
4.1 Participants
Seven pairs (10 male, 4 female) used both setups to carry out a collaborative ideation task. The participants were recruited in pairs, with the requirements that they were experienced in using sticky notes in creative tasks and/or design processes and were used to working together using sticky notes. Their average experience of using sticky notes in design processes were 5.3 years (σ = 2.5); nine reported weekly use and five reported monthly use of sticky notes for various creative tasks. They all had at least a bachelor’s degree in a relevant field (e.g., Information Studies or IT Product Development), and were either master students or IT consultants, UX designers, and one was a IT Lab Manager.
4.2 Procedure and Data Collection
Before working on the collaborative ideation tasks, participants were briefed about the agenda and purpose of the study. Participants further received an introduction to the digital setup. The core part of the study involved solving two tasks; one with a physical setup and the other with our digital setup. Each task consisted of designing features and functions for a concept: a service called Milk&CookiesNow and a Smart Wardrobe App, respectively.2 The tasks both involved 3 phases: (i) brainstorming features, functions, or potential challenges (= brainstorming phase), (ii) clustering created notes by sorting them in groups using a bottom-up approach (= clustering phase), and (iii) labelling the groups and drawing associations between them (= labelling phase).
In order to have similar conditions among the groups, the 3 phases were timed and participants were asked to continue with the next ideation phase when the time was up. The participants worked creatively with each setup for 16 min (6 min brainstorming, 5 min clustering, 5 min labelling) during the study. To create a more natural environment, we did not require groups to continue when they wanted to move on to the next phase, and they were allowed to finish discussions and note creation that were initiated before the phase finished. In total, including introduction to the study, participants reading the provided scenarios and user stories, the creative work and a follow-up interview (results reported in: Jensen et al. 2018), the study lasted around 80 min.
Participants gave their consent to have the tasks video recorded. For each setup, two cameras captured participants’ interactions. One camera was mounted on the whiteboard or Microsoft Surface Hub respectively, capturing the entire scene (e.g. how participants interacted with each other and the tablets/sticky notes). The second camera focused solely on the board or screen respectively generating footage of participants’ interaction with the physical and digital sticky notes as well as the board/screen. At the end of the study, participants filled in a questionnaire about their demographics and sticky note experience.
4.3 Analysis
Participants’ interaction and collaboration while working on the ideation tasks was analysed using video-based interaction analysis as outlined by Jordan and Henderson (1995). To that end, interactional phenomena such as the position and focus of attention as well as the use of physical and digital artefacts were coded.
With reference to previous studies in collaborative settings that are at least partially mediated by digital artefacts, we assumed that we would observe differences regarding the use of digital and physical artefacts and how those might influence collaboration and ideation. We thus applied a deductive approach to our video analysis. Being open to other aspects that might appear, we also followed up with a whole-to-part approach (Erickson 2006) in which 3 researchers began by watching the entire videos before progressively increasing the depth. We focused on themes that emerged from aspects mentioned by the participants during the follow-up interviews and interesting snippets observed and noted during study. Group viewing and re-iterating categories further defined our themes (Engle et al. 2007). In the coding process, attention was focused on when participants interacted with each other as well as how they interacted with both the physical and digital artefacts (i.e. tablets, sticky notes, board and screen). To facilitate analysis and get a better overview of the scene, videos from the two cameras were synchronised and viewed next to each other. We further looked for patterns in the participants’ routines; for instance, how and whether they collaborated in the brainstorming part. Identified patterns were then checked against other sequences and overlap with other themes.
To code the location and focus of participants, sticky note actions, and the verbal communication between participants, we used ChronoViz3 (Fouse et al. 2011). To analyse the example snippets, we used video transcriptions. We coded the videos using 3 schemes, each focusing on a different aspect, relating to a research question. First, we coded the participants’ interactions with the notes, asking; does the digital material change the way people work and interact with the notes? Does it change their division of labour? Second, we coded the participants’ talk and speech, asking; does the setup change turn-taking and social relations between participants? Does it impact the way they discuss? Third, we coded the participants’ focus, i.e. the direction of their gaze, and their position in the room, asking; does the screen capture the users’ attention, making them less focused on each other? Does the digital setup impact the socio-spatial configuration, e.g. how people move and act in the room? Further, by combining these insights, we also asked; does the setup impact the ideation techniques used by the participants? With these questions in hand, we conducted our analysis.
In this section, we present the results from our video analysis. The quantitative data originates from the coded annotations of the individual study recordings. Where appropriate, we illustrate the quantitative findings by outlining and presenting interaction examples from the video recordings. We seek to unfold the influence that digitizing a design tool has on collaboration by examining the differences and similarities in terms of; (1) How participants use and handle notes, (2) how they approach the ideation in terms of technique and communication, (3) the group dynamics, e.g. their relations and division of labour, and (4) how they situate themselves and act in the room, e.g. the socio-spatial configuration. Together, these four themes span the main aspects of collaboration.
5.1 Handling Physical and Digital Notes
Being interested in whether digital technology would impact the number of notes created, we counted the notes created in each setup. In the physical setup, 13.86 (SD = 5.67) notes were created and 13.57 (SD = 6.88) notes in the digital setup. Considering that the phases were timed, some groups would have produced more notes had we allowed them to continue brainstorming. A t-test showed that there was no significant difference between the 2, t(6) = 0.34, p = 0.744, indicating that the material does not affect the quantity of posts created. In order to investigate the differences in the use of physical and digital artefacts, we coded interactions with notes. We identified and coded four types of interactions:
a.
Point/touch: participants point to a note or touch it (without moving it).
b.
Move: participants move a note on the board/screen.
c.
Hold in-hand: participants took a note from the board and held it in their hands (this interaction also counts as at least two moves; one to the hand and one back to the board; those are included in the total move count).
d.
Post: participants posted a note to the board.
Although technically possible with the digital setup, the participants only held notes in their hand when using the physical setup. This was done either to draw attention to the note, or to limit attention to it by removing it from the board.
We investigated the differences between participants’ interactions with digital and physical notes by conducting t-tests. Overall, we found that participants interacted more with digital notes (M = 135.71 actions, SD = 43.87) than with physical notes (M = 98.86 actions, SD = 37.16), t(6) = 2.667, p = .037. Although the number of sticky notes created is not significantly different between the physical and the digital setups, there is a difference in the interaction with the notes that we explore further.
While we did not find significant differences between pointing and posting for the two setups, there were significant differences for moving and holding notes in the hand. The latter is unsurprising, since holding notes in the hand did not occur in the digital setup. Digital notes (M = 87.29 moves, SD = 35.47) were moved significantly more than physical sticky notes (M = 37.86 moves, SD = 19.24), t(6) = 4,44, p = .004. This difference can be explained by 3 practices that we observed:
1)
When posting notes to the digital board, most participants place them in one spot first and then – after it appears on the screen – move the note to another location (see Example 4). For most groups, this move after initial posting was not connected to sorting or grouping notes. To some extent, one could say that this move was redundant as the participants could just have posted the note in the second location to start with.
2)
When drawing attention to a specific note in the digital setup, some participants would move the note around or “shake” it (cf. Example 2), whereas in the physical setup participants would take it into their hands (c.) or just point towards it (a.) (cf. Example 3).
3)
Participants generally “fiddled” more with the digital sticky notes, restructuring them again and again. In some examples, participants clearly just played around with the notes, e.g. moving notes simultaneously with both hands or moving clusters of adherent notes, as this was so easy to do in the digital setup (cf. Example 2).
While using digital technology does not result in the creation of more or fewer notes, it does lead to more interaction with them, as illustrated in Figure 3. It seems that the digital setup encourages playful (cf. 3.) and redundant (cf. 1.) interaction that does not necessarily contribute to the underlying design or ideation process.
Figure 3
. This graphic shows note interactions for each group and how they are distributed across the 2 setups (physical and digital) divided into the four categories (Point/touch, Move, Hold in hand and Post). The y-axis indicates {digital/physical}_{group letter}
×
Since previous research on tabletops and large display collaboration has shown that users tend to divide the available screen space into private and public areas (e.g. Whalen 2003), we were interested to see whether this applies to our physical and digital setups. Instances where participants placed their notes only on one side of the board, marking the side as private, were coded as separated posting of notes (see Example 1), whereas instances where participants did not seem to distinguish between sides or areas of the board and placed notes anywhere5 on the screen were coded as mixed posting of notes (see Example 6 and Example 5).
Contrasting previous findings, most groups tended to consider the entire screen as shared space and posted notes anywhere on the screen, not distinguishing between individual sides (see Table 1). In the physical setup, participants seemed to prefer to place their notes on a designated side first, before moving them to other parts of the board in the clustering and labelling phase. In two cases, participants combined the brainstorming with the clustering phase. For those cases, it was difficult to determine whether each participant had their own side as notes got instantly moved or posted near the group that the note belonged to, and hence they were counted as mixed as well.
Table 1
Overview of techniques used during the brainstorming part and placement of notes in a private area (separated) or considering the entire board/screen as shared space (mixed)
Brainstorming Technique \ Notes Placement
Separated
Mixed
Nominal Brainstorming
p_B; p_D
d_B; d_D
Interactive Brainstorming
p_A; p_C; d_G
d_A; d_C; d_E; p_F; p_G
We noticed no difference in articulated ownership of notes between the physical and digital setup. In both setups, participants referred to the notes both with “mine” (Example 4) and “yours” as well as “it” (Example 3). When working with the notes (i.e. during the clustering phase), participants seemed not to distinguish between “their” notes and the ones their partner had created. As far as the outcome of the ideation, it did not seem to matter who had created the notes. Overall, participants mainly focused on the contents of the note (e.g., idea, issue) as opposed to the physical/digital artefact itself. Occasionally, participants would point out who had created the note (e.g.in Example 4: D.2: “that’s a good one” – D.1: “It’s mine”). In that respect, articulated ownership matches participants’ usage of space in particular for the case of the digital setup in the sense that participants perceived notes as being held in common.
In summary, participants seem to handle and use the notes similarly in the digital and the physical setup, with the same approach to posting, space and note ownership. The main difference between the 2 setups were the amount of general note actions and moves, which were higher in the digital version. Part of these extra interactions can be explained by system novelty. However, it also seems that the digital material invites additional actions, possibly due to the absence of glue issues, known from physical sticky notes (Harboe and Huang 2015; Jensen et al. 2018). Nevertheless, there is no evidence pointing towards that the participants changed their ways of collaborating remarkably.
×
×
×
×
5.2 Brainstorming Technique
We identified two brainstorming techniques that participants used during the first part of the tasks which are in line with Rietzschel et al.’s (2006) definitions. The first technique matches the definition of interactive brainstorming, where participants interact with each other switching between discussing ideas, creating sticky notes and re-visiting the instructions. The second technique relates to nominal brainstorming, where participants spend most of part one of the task creating sticky notes in silence, occasionally referring back to the instructions (see Example 5). Due to the almost non-existent communication in this technique, it could also be referred to as individual, silent or solitude brainstorming. In Example 4 above, the group has just moved from writing a bunch of sticky notes individually that they have then posted on the digital board. Two of the groups (B and D) never spoke a word to each other during the entire ideation generation.
On average, the groups spent 13.83 min in the digital setup and 13.48 min in the physical setup working on the entire experimental part of the study (i.e. all 3 phases; ideation, clustering, labelling). In the digital setup, participants on average talked for 5.96 min (SD = 166,263; corresponding to 42.33% of the entire task duration) and in the physical setup for 6.35 min (SD = 153,141; corresponding to 46.94% of the entire task duration). There was no significant difference between the amount of conversation between the 2 setups, t(6) = .548, p = .603, indicating that using technology during collaborative ideation tasks does not decrease conversation. These durations only account for conversation pertaining to the actual ideation task (e.g. discussions about individual notes). During the study, participants also talked and made comments about the set of artefacts they were given to carry out the task. There was significantly more talk about the artefacts in the digital setup (M = 12,894, SD = 3619) than in the physical setup (M = 2161, SD = 2928), t(6) = 6.785, p = .001. The 95% confidence interval regarding the effect of more talk about the digital artefacts occurring is between 0.11 s and 0.24 s. In fact, only 3 groups talked about the physical artefacts at all (i.e. sticky notes, pens, board). These numbers further point towards the difference in durations between the digital and physical setup, potentially explaining why on average groups spent more time on the experimental part in the digital setup.
The amount of talking occurring in the two setups reflects the type of brainstorming chosen during part one of the study. In both setups, groups having opted for an interactive brainstorming talked more than those having opted for nominal brainstorming. However, this difference proved only significant in the digital setup, t(6) = −2.67, p = .044. On average, participants talked 4.35 s longer in the interactive brainstorming condition than in the nominal condition (Figure 4).
Figure 4
. Overview of speech during the study grouped by topic, where “Talk” refers to talk related to the idea generation task and “Talk about setup” when participants made comments about either the digital artefacts (i.e. tablets, digital board) or the physical artefacts (i.e. sticky notes, physical board)
×
Table 1 provides an overview of which groups chose which technique in which setup. In both the physical and digital setup, participants seemed to prefer the interactive brainstorming approach. The two groups having opted for the nominal (or silent) brainstorming, used this approach in both setups. In fact, none of the groups changed their brainstorming technique when switching setups.
An interesting observation is that both groups doing nominal brainstorms used a separated initial posting approach in the physical setup and a mixed initial posting approach in the digital (see Table 1). One group posted their physical notes as they created them, causing them to divide the board between them in order to avoid hindering the work of the other. In the digital setup, however, they kept all notes on the iPads until the first part of the task was over. Having two users post a combined 28 notes simultaneously (as was the case in group D) was not accounted for in the cross-device object movement strategy. This resulted in participants helping each other post, with one pressing the screen and the other tapping the iPad, causing a natural mix of the notes (even though it would have been possible to separate them).
In the other group doing nominal brainstorming, the participants put their physical notes on their arms and then went to a side of the board each to post them. In the digital setup, they posted their notes subsequently, which seemed to invite them to spatially mix the notes on the board. From this it seems that the physical setup encourages separate simultaneous note posting, while the digital setup encourages mixed posting. However, it is unclear why this is, as the two groups used different approaches in all cases.
In summary, the 2 setups did not have an impact on the choice of brainstorming technique (interactive or nominal) nor were there any indications that the digital material impacted the discussions or collaboration. The participants did talk more about the digital artefacts, which seems natural given the novelty of them working with them. Moreover, it does seem that the digital setup changes the way participants post notes simultaneously, even if that might be due to the implemented procedure for transferring notes. This could potentially impact the collaboration, as the order in which notes are talked about could be more random when posted with a mixed approach. Particularly for digital notes, it might be harder to distinguish between different authors of notes. In that regard, a mixed set of notes might reduce barriers and bias when discussing other people’s notes, leading to a more open and genuine reflection on them. However, in this study, we have not encountered any concrete indications of this.
×
5.3 Group Dynamics and Turn-Taking
The dynamics of brainstorming are further influenced by the behavior of the group members in relation to one another, e.g. whether one was more dominant in terms of speaking, initiative and turn-taking. In order to define whether any of the participants showed dominant behavior or accounted for the majority of speech, we defined a threshold of activity/speech where one participant undertakes more than 60 pct. of all the interactions or is responsible of more than 60 pct. of speech.
In 3 of the seven groups, one participant could be identified as dominant pertaining to speech. In both group C and E, it was the same participant who spoke more than the other participant in both setups. In cases of no dominance, both participants had about the same amount of speaking time, within the 60 pct. Threshold. With reference to Figure 5, it seems the physical/digital material does not have an impact on dominance in speech behavior.
Figure 5.
Summary of how much each participant (X.1 and X.2) talks during the ideation sessions (digital = d, physical = p) in the study. X.1 and X.2 refers to participant 1 and participant 2 of group X, for example; the blue bar in d_D represents the percentage of talk done by participant 1 in group D (D.1) in the digital setup
×
Taking into consideration that only about 50 pct. of communication is verbal (Goffman 1959), we also looked at the number of note actions undertaken per participant as indicator for dominant behavior. Figure 6 summarizes participants’ interaction with notes in both setups. According to note action, with exception of group C and group G, each group had a dominant participant in at least one setup. In group D, participants switched roles across setups; where participant one was interacting more with notes in the digital setup, participant two was more active in the physical setup. Only in group A, the same participant displayed a dominant behavior regarding note action in both setups. In four cases a participant was dominant in the digital setup and in 3 cases a participant was clearly more active in the physical setup. Judging by the amount of note actions, we could not identify a tendency as to which setup leads to more dominant behavior. Hence, it seems that the dominance displayed by the participants was not influenced by the two setups.
Figure 6
. This graphic shows the division of the total number of note actions that occurred in each trial; sorted by participant (X.1 and X.2)
×
In four of the groups there was an almost equal division of labor between the participants, whereas in the other 3 groups, one participant carried out a majority of the note actions. We investigated whether this division of labor differed between the 2 setups. Looking at the ratio of total note actions for participants, we did not find a significant difference between division of labor, t(6) = .883, p = .411. There was only a visible change of dominance between the 2 setups for group D and group F; one participant was clearly more active in the digital setup, while the other participant was more active in the physical setup. While none of the participants showed dominant behavior in group C, their roles also changed between setups. This indicates that participants were not less or more active in either of the setups and that the digital version does not encourage dominant behavior.
Being interested in how collaboration was organized and structured, we compared dominant behavior with dominant speech. We found that only for group E and F, dominant behavior and dominant speech matched. In group E the same participant was dominant in both setups and accounted for the majority of note actions as well as was speaking the most often and longest. Overall, we found seven instances of dominant behavior when looking at note actions (digital: 4, physical: 3), but only five instances of dominant speech (digital: 2, physical: 3).
We further investigated whether the amount of speech correlates with the number of note actions. For the digital setup, there was no relation between the number of note actions (M = 135.71, SD = 43.87) and the amount of speech (M = 357,627, SD = 166,263) that relate to the ideation task, r = .361, p = .426. Also, in the physical setup, there were no significant correlations between the amount of speech and the amount of moving, focusing and posting notes. We did, however, find a strong, positive correlation between pointing at physical notes (M = 37.43, SD = 17.50) and talking about their content (M = 380,991, SD = 3619), r = .875, p = .010.
We further noted different strategies as to when notes get posted during the first part of the collaborative ideation, i.e. the brainstorm (see Table 2). One approach can be described as posting and creating notes alternately, meaning participants would create a note, then post it to the board, return to the table, create another note, post it to the board, and so on. We refer to this strategy as alternate posting. The second approach, sequential posting, was to create all notes first and then post them to the board at the end of the brainstorming session.
Table 2
Summary of strategies for initial posting of notes to the board/screen during (alternate) or after (sequential) the brainstorming phase
Alternate [d: 3; p: 6]
Sequential [d: 4 p: 1]
Groups
p_A; d_A; p_C; d_C; p_D; p_E; d_F; p_F; p_G
d_B; p_B; d_D; d_E; d_G
Some groups doing alternate posting did so while standing in front of the board, while others sat back down at the table after they posted a note to the board. In the digital setup, group A combined those two approaches. They started off by creating notes at the table, then after about a minute get up and start creating notes while standing in front of the screen, posting notes alternately. This points towards a further concern for the spatial setup and use of the room.
In this section, we have identified a number of turn-taking strategies that differ over the phases of the collaborative ideation, across groups, across speech and action, and across material of the sticky notes. In some groups, members take turns leading, whereas in others they do not, but this is not influenced by the setup. In one of the groups, the same participant was dominant in both setups and accounted for the majority of note actions and speaking. Again, it seems like the digital setup influences how notes are posted to the board, but it does not seem to impact the division of labour nor the roles between the participants.
×
5.4 Spatiality, Orientation and Attention
The spatial setups were as follows: the meeting room has a table where the group members can sit across from each other and look up at the board or the screen, which are both placed 1 m from the end of the table (see Example 7). The digital board is slightly more in the corner of the room than the physical one (see Example 6). This means that the group members either work both seated, one seated, one standing, or both standing next to each other. Standing may happen at the board or closer to the table as we see in Example 7.
We investigated where participants were located when working on the creative tasks and whether their use of the space differed between the 2 setups. On average, participants spent about 49.32 pct. of the entire task duration in front of the board and 40.60 pct. at the table. The sum of these durations does not equal the entire time used for the tasks, as occasionally participants would hover between what we defined as the board and table area, or they were simply moving between the table and the board. The data show that in the physical setup (t(6) = .956, p = .376) and in the digital setup (t(6) = 656, p = .536) participants did not spend significantly more or less time in front of the board/screen or the table. There was also no difference in the amount of time participants spent in front of the board/screen, t(6) = .003, p = .998 or the table, t(6) = −.134, p = .897, between the 2 setups. This suggests that the digital artefact did not have a drawing-in effect, encouraging participants to step closer. Thus, digital artefacts do not necessarily imbibe attention from the task at hand or the social aspect of collaboration. Our participants only focused on the board when there was a pragmatic reason for doing so.
As a next step, we addressed participants’ focus towards each other and towards the artefacts in the room. For this purpose, we coded 3 targets of attention: i) the partner/other group member (as seen in Example 7 a-b), ii) sticky notes (as seen in Example 6 a and d), and iii) the board (as seen in Example 2). We identified a focus whenever participants were visually focused on one of the 3 targets (Bødker 1996).
The majority of the time, participants were focusing on either the screen (average 43.42 pct.) or the board (average 43.11 pct.). In line with findings from previous studies (Argyle and Graham 1977; Anderson et al. 1997), participants spent the least time focusing on their partner. While we did not find significant differences for the total durations of focusing on either the sticky notes or the board/screen, participants spent significantly less time gazing at the other participant in the digital setup (M = 204,2616, SD = 151,340) than in the physical setup (M = 304,044, SD = 172,359), t(6) = −2.579, p = .042. In that respect, the durations of participants looking at the other were also significantly shorter in the digital setup. In other words, not only did participants look less often at each other, but also the time they spent looking at each other was less in the digital setup. Furthermore, the amount of time spent focusing on digital notes (M = 459,275, SD = 192,806) is significantly longer than on physical sticky notes (M = 315,454, SD = 143,532), t(6) = 3.303, p = .016. This increased attention to the notes on the pad in the digital setup is caused by the creation process, as the participants were not familiar with using a stylus and an iPad interface instead of pen and paper. This extra time spent looking at the iPad is probably also the reason participants look less at each other (Figures 7 and 8).
Figure 7.
Summary of where participants were standing in the room while doing the creative brainstorming tasks
Figure 8
. The graphic gives an overview on what participants focused during the creative task. The bottom line describes attention focus in the physical, the upper line describes the digital setup
×
×
There was no difference in the individual time spans spent looking at digital notes on the tablets (M = 6306, SD = 3311) and physical notes (M = 7415, SD = 2348), t(6) = −.918, p = .394. The higher variance for the digital notes can be explained by considering that when posting a digital note to the screen, users had to touch the screen and tap a button on the tablet. This requires at least a short glance at the tablet, which occurred whenever a participant was posting a note to the screen, explaining both the shorter time intervals and the high variance in the attention spans when using digital notes. Hence, the finding of participants spending more time focusing on digital notes is probably a result of the design of the specific cross device object movement technique, more than it is an effect of the digital material as such.
In summary, participants look less at each other and more on their artefacts in the digital setup, and it seems that the design of the digital setup may explain why, at least in parts. In their work on socio-spatial aspects of computing, Krogh et al. (2017) argue that every technology design has a spatial agency that can influence socio-spatial configurations. Based on our result, however, the choice of digital and physical setup does not appear to have an influence on the users’ spatial behavior and how they move around in the room during the brainstorms. This finding is important when comparing the collaborative aspects across the two setups, as socio-spatial configuration is a quintessential part of collaboration (Krogh et al. 2017). In other words, except for the more limited amount of gazing at their partner, especially when creating notes in the digital setup, the socio-spatial configuration plays exactly the same role for the collaboration in the 2 setups.
×
6 Discussion
In the following, we will discuss the results presented above. First, we will provide an overview of the findings that resulted from our analysis of participant collaboration and artefact use, and discuss them in relation to the findings on user experience as presented by Jensen et al. (2018). Second, we will discuss the implications of moving on from a no-frills remediation by highlighting potential pitfalls and opportunities for other materials than sticky notes.
6.1 Differences between Digital and Physical Notes
Our analysis granted us insights into how the digital setup influences various aspects of collaboration and use of artefacts during ideation as compared to the physical setup (for an overview see Table 3).
Table 3
Overview of findings, sorted by whether or not there was a significant difference between the physical and digital setup
Metric
Description
Difference
Explanation
Interactions with notes
Moving notes around on board and in hand
Different
There were more interactions with the digital notes
Talk about artefacts
The conversation not related to the task but the tools
Different
There was more conversation focusing on the digital tools.
Orientation
The time spend looking at the other participant, the notes and the board
Different
Participants were more oriented towards the digital notes during the creation and less oriented towards the other participant.
Use of the board
The participants use of the digital and physical boards for posting
Different
The setups did influence where (mixed vs. separate) and when (sequential vs alternate) notes were placed on the board.
Quantity of notes
The number of notes created
No Difference
There was no difference in the average number of notes created
Choice of technique
The brainstorming technique used across the setups
No Difference
The setups did not influence the groups’ choice between nominal and interactive brainstorms in general
Ownership
The way participants talk about the notes, i.e. mine vs. ours
No Difference
The participant referred similarly to the physical and digital notes
Conversation and discussions
The amount of conversation during the session
No Difference
The setup did not influence the amount of time each participant spoke
Actions and collaboration
The number of note interactions for each participant
No Difference
The setups did not influence the number of the participants’ interactions
Speech dominance
Was one participant more dominant in the conversation depending on the setup?
No Difference
The setup did not have an impact on which participant was dominant in the conversations
Action dominance
The setups’ effect on participant dominance in terms of note interactions
No Difference
The setup did not influence which participant made most interactions
Spatiality
The amount of time spent in different place and positions in the room
No Difference
There was no difference in the time participants spend at the digital and physical board and by the table.
Our remediation of sticky notes did not lead to an increased number of ideas/notes created in a collaborative ideation session, and from Jensen et al. (2018) we know that the majority of the participants felt equally creative and productive when working with the digital and physical notes. We did, however, see that participants interacted significantly more with the digital notes, in particular moving those around. While moving physical notes around on the board seemed to happen less (possibly to avoid notes from falling off), digital notes were playfully shuffled around multiple times (i.e. recurring re-arranging of notes after initial post). Furthermore, “shaking” digital notes replaced the relatively cumbersome action of taking notes from the board (i.e. pulling and holding in hand) in order to, for instance, draw attention to them. As this interaction was used in all groups, it points toward it being afforded by the digital notes. This is supported by Jensen et al. (2018), as they find that participants in general prefer to sort and cluster the notes in the digital setup, as it is easier to move the notes around and they do not have to worry about the known glue issues (Harboe and Huang 2015). However, this difference does not seem to have any direct influence on creativity, nor on productivity in particular. This corresponds with the work of Tang et al. (2011) and Chulvi et al. (2017), who found that digital setups did not significantly influence measurable creativity in sketching.
The digital setup did not have any impact on brainstorming techniques, group dynamics (dominance), socio-spatial configurations or discussions related to the topic at hand. It does, however, influence the posting procedure (alternate vs. sequential and mixed vs. separate).
However, it is not clear what the consequences of this might be, and this is a question for future creativity research. Based on literature suggesting that tablets encourage more individual work (Magerkurth et al. 2004), participants could be expected to lean towards the nominal brainstorming approach, where ideas are created independently from other group members. Since the amount of conversation did not differ between the 2 setups, we conclude that the presence of tablets did not discourage collaborative interaction (i.e. brainstorming technique, amount of conversation). On the other hand, the fact that participants spent less time looking at each other in the digital setup may indicate that digital artefacts such as tablets are visual distractions during collaboration. This is in line with findings from Zagermann et al. (2016), who showed that larger tabletops divert users’ attention away from their collaborators. Yet, this distraction did not seem to have negative consequences on the overall ideation in the current study (cf. no difference in number of notes produced). While the iPads seemed to draw visual attention, the Surface Hub did not attract spatial attention in the sense that there was no difference in where participants positioned themselves in the room. In general, the digital artefacts did not impact the socio-spatial configuration, e.g. where and how in the room the participants positioned themselves, which is considered an essential factor when evaluating collaboration (Krogh et al. 2017).
The majority of users tended to consider the entire screen as a shared space and did not distinguish between individual or shared areas when initially posting notes. This finding stands in contrast to Whalen’s (2003) observations on collaborative use of tabletops. In line with Magerkurth et al. (2004), the tablets on the other hand could indeed be seen as private artefacts. Tablets were never shared, as in given to the partner, nor was content on the tablet shown to the partner. This observation also applies to the physical notes. Hence, notes only became public after they had been posted on the board or screen. Jensen et al. (2018) showed that the participants did feel greater awareness about each other’s work when using physical notes, as they were able to see the color of the notes and the writing on them, in contrast to the iPad. In both setups, posted notes were considered public artefacts, supported by the notion that participants generally referred to these as belonging to neither individual but to the group.
In general, we did not find any evidence that the setups impacted the nature of collaboration, again supported by the participants’ experiences, where the majority stated that their collaboration did not change across the setups (Jensen et al. 2018). Yet, while there was a similar amount of conversation, participants did talk more about the digital artefacts. This could be attributed partly to the novelty of the system, and partly due to the fact that breakdowns occurred more often in the digital setup. For example, when participants forgot that the stylus used for the iPads could not be used on the Surface Hub. The lack of pen homogeneity in the setup was pointed out as an important challenge to address (Jensen et al. 2018). Our findings that participants did spend more time focusing on digital notes compared to physical notes support that. However, it is worth considering the effect that the novelty of the system had on participants, in particular the implemented cross-device-object-movement technique, which required users to look at the iPad whenever a note needed to be posted or pulled, causing an overhead and requiring more attention than the embodied and natural movement of “sticking” a note to the board. This cross-device object movement technique was also highlighted as an important challenge for future systems by Jensen et al. (2018).
In summary, the majority of the metrics indicate that the physical and digital setup equally support collaboration in many facets (cf. Table 3). This conclusion is also supported by the participants’ own experiences, as presented by Jensen et al. (2018). Nevertheless, we found significant differences in the brainstorming phase, when notes are created and posted, due to stylus and posting issues. This was highlighted by the participants when asked (Jensen et al. 2018), but further consolidated in the current analysis, as participants were more focused on the devices than on each other and as they posted the majority of digital notes in a mixed order. As such, the qualitative experience of the participants was also reflected in the quantitative data. The increased attention to the technology points to the merits of physical sticky notes, yet previous literature has highlighted problems with physical notes such as the glue used to attach sticky notes to surfaces (Jensen et al. 2018, Harboe and Huang 2015), especially in terms of durability and documentation. Overall, this means that it makes sense to support physical sticky notes for brainstorming and note creation, with the hope of transferring them seamlessly to a digital form rather than gluing them to the board.
In the clustering phase, sorting notes was done more freely and easily in the digital version, as notes could be moved on the board with minimal effort, and “shaken” to be brought to the attention of the collaborators. Also, when notes are given labels and connections in the labelling phase, the easier movement of notes seemed to be in favor of the digital setup, as indicated by the additional notes interactions and supported by the participants experiences (Jensen et al. 2018). Largely this means that the clustering and labelling phases of collaborative ideation are potentially better supported in the digital setup, despite the more cumbersome creation and posting of the notes in the first place. These controversies led us to the conclusion that it does not always have to be an “either/or” decision and that we should focus less on “versus” but more on merits of the available options.
Therefore, we want to explore the following 3 issues in future work:
1)
Enable easy ways of digitizing the physical sticky notes after brainstorming.
2)
Provide more appropriate ways to post digital notes.
3)
Create augmented sticky notes that remain physical but have some features that are replaced or supplemented digitally.
With these 3 future work areas, it is certainly also possible to explore many other merits of extended digital setups, e.g. the possibilities for remote collaboration, for saving shared work spaces and for including other materials, inspired by existing work (Geyer et al. 2011a, Hilliges et al. 2007, Tsu et al. 2007, Probst et al. 2011, Klemmer et al. 2001), thereby deviating from the purist and no-frills mapping of the physical to the digital. How this may work is a concern for later work, but we are convinced that starting off with a translation that allows users’ skills to transfer from their physical work space provides a solid foundation. It is important to implement this before adding additional features, as such features could otherwise clutter the interface, rendering it overwhelming or confusing for the user.
6.2 Moving on from a no-Frills Remediation
When relying more on digital features and infrastructure, we need to be aware of the possible consequences. Bødker and Christiansen (2006) discuss how Grudin (2002) presents new challenges to CSCW in the ubiquitous situation, in particular how Grudin talks about social encounters as being ‘in the making’ and hence “jeopardized if–by means of IT–the ‘here–and now’ is substituted with ‘everywhere–and forever.’” This means that the challenges for digital setups like ours lie in the ephemerality of face-to-face meetings, as well as in the space where participants explore and think privately (e.g. by writing on sticky notes not visible to others than those present). As there is an inherent persistence in the digital setup and a disclosure (with possible loss of control) in the shared board, it may cause some challenges to our digital design, possibly reducing productivity and creativity. Grudin (2002) argues that we need to be able to work without being constantly monitored. This relates to Gutwin and Greenberg (1998)‘s focus on the trade-offs between individual and group space in collaboration, particularly regarding the ownership of the notes that have been put in public on the board, as well as Herrmann (2008)‘s work, where the issue of possibly destroying the contributions of others gets raised. In our current study, we did not find examples of these concerns, and we can only speculate whether this may be because the participants are used to collaborating, or because the tasks to be solved were exercises. We believe that the participants overcame these well-known obstacles because of their prior experience of working together with physical sticky notes, and that their developed skills seem to have transferred to the digital setup as well. Ultimately, these considerations point to challenges that we have not explored, and hence they may only surface in more extended digital setups.
Ideation may include a much wider set of materials than sticky notes, e.g. game pieces (e.g. Bødker and Buur 2002), scenarios (Bødker and Christiansen 1997), inspiration cards (Halskov and Dalsgaard 2006), situation cards (Mogensen and Trigg 1992), physical and virtual pictures (Hartmann et al. 2010) and for that matter Lego bricks (Dove et al. 2017). Our current study does not explore the impact of other materials on a setup like ours. However, many of these are actually more easily dealt with in a digital form (with a clear exception of the Lego bricks), and we do not see much in our study that would talk against an extension of the types of materials for further studies. We believe that minimalist implementations of such remediations provide a solid outset. This is because the minimalist ‘no-frills’ approach appears to enable a high degree of skill transfer between setups, as supported in the presented study.
A remaining issue in our study is the size of the group collaborating: Are our findings generally useful for larger groups, as we know them from collaborative meeting settings, etc.? On one hand, it is quite evident that the detailed ideation processes that we explored would look rather different had we studied larger groups, e.g. doing Future Workshops or other forms of discussion meetings (Mogensen and Trigg 1992). On the other hand, there can indeed be situations where similar (technical) challenges due to the presence of multiple users can arise in digital setups. The posting by several users via iPads is probably the least of them, but e.g. the Microsoft Surface Hub used in our setup only deploys two pens. Otherwise, the challenges are mainly based on the structuring of the board space and the ideation process to support more participants. This does not specifically point to the difference between the physical and the digital setup. Another limitation is certainly the introduction of artificial constraints in the physical setup (e.g. limited posting space). In order to make them comparable, both setups were limited in their inherent affordances. We acknowledge that, in particular, our findings pertaining to spatiality would have differed if those constraints had not been in place. There may also be a difference in group dynamics and handling notes.
To summarize, both physical and digital materials have their advantages when supporting collaborative ideation. Future work points towards a possible merger of the two platforms, in order to get the best of both worlds. We have also raised concerns that might follow from increased digitization in situations that we have not studied. At the same time, we find a lot of merit in aiming for no-frill implementations for collaborative setups–a concept that has not been explored much in CSCW.
7 Conclusion
This paper compared the use of physical and digital sticky notes in an ideation setting to investigate how the physical and digital material affects collaboration. While our study showed some differences regarding the participants interactions with the two types of notes (physical and digital), we also found that ideation and collaboration can largely be supported by physical or digital sticky notes, for groups of users who are experienced in working together and using sticky notes. The brainstorming technique, group dynamics, and socio-spatial setup are not significantly influenced by the sticky note setup. However, the digital sticky notes still require additional attention when it comes to creating them and posting them on the board, in contrast to the physical sticky notes, where these actions are more seamless and embodied. Contrary, handling the digital notes are easier, as illustrated by the increased note interactions, and as such, the clustering and labelling phases of ideation are potentially better supported in the digital setup, despite the more cumbersome posting procedure. In other words, we observed differences between the uses of the 2 presented setups, showing how they provide different strengthes and weaknesses, respectively. However, in the end, both setups can support ideation and collaboration.
This points towards different qualities of the two materials that may be best brought out in a combined solution where sticky notes are produced (mainly) physically but transformed to the digital setup before, or as part of, the clustering and labelling phases. This introduces new challenges for the research ahead in terms of working towards seamless transfers from physical to digital sticky notes and boards.
As a next step in our research, we plan to explore ways to transfer qualities from digital notes to physical notes in an attempt to better support the clustering phases of collaborative ideation, and in particular moving notes. Accordingly, our future work will focus on establishing easy ways to (partially) digitize physical sticky notes after brainstorming (e.g. by creating augmented sticky notes), as well as providing more appropriate ways to post digital notes.
Acknowledgements
We thank the entire CIBIS project, and in particular Clemens Nylandsted Klokmose and Roman Rädle for their contributions to this project. The work is sponsored by Innovation Fund Denmark grant 1311-00001B, CIBIS.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
In the following, we use a naming convention, where the participating pairs are referred to as group A-G. Specific participants are then referenced using X.1 and X.2, where the X is the group letter and the number points to a specific group member, meaning that D.2 is participant 2 of group D. Further, we occasionally refer to the groups’ use of the digital and physical setup using d_X and p_X (again X is the group letter) respectively, meaning that d_B refers to group B using the digital setup.
