Skip to main content

Über dieses Buch

This book explores the technological advances and social interactions between interactive spaces, surfaces and devices, aiming to provide new insights into emerging social protocols that arise from the experimentation and long-term usage of interactive surfaces.

This edited volume brings together researchers from around the world who investigate interactive surfaces and interaction techniques within large displays, wearable devices, software development, security and emergency management. Providing both theory and practical case studies, the authors look at current developments and challenges into 3D visualization, large surfaces, the interplay of mobile phone devices and large displays, wearable systems and head mounted displays (HMD’S), remote proxemics and interactive wall displays and how these can be employed throughout the home and work spaces.

Collaboration Meets Interactive Spaces is both for researchers and industry practitioners, providing readers with a coherent narrative into the current state-of-the-art within interactive surfaces and pervasive display technology, providing necessary tools and techniques as interactive media increasingly permeates everyday contexts.



Chapter 1. An Introduction to Collaboration Meets Interactive Spaces

Interactive Surfaces and Spaces have become ever more pervasive in the past decade. Indeed, the current explosion of media that pervades our everyday lives invades our senses through (increasingly) interactive displays surfaces in all sizes, shapes and formats. Indeed, interactive walls, tables, mobiles (tablets and phones), as well as wearables change the way human beings interact with information and collaborate with one another. At the same time, these surfaces and devices are redesigned and reinvented through new social protocols and collaborative work styles that arise from the experimentation and long-term usage of novel people/device ecologies. The book reflects a high interest among researchers and practitioners about this particular approach and the challenges it entails. It offers an up to date and comprehensive scientific overview of the new generation of devices and their myriad combinations. While pervasive display technologies are changing the way we relate to media, people and society are also shaping and adapting new techniques, methods and idioms. Our purpose is to update both researchers and practitioners with exciting new work around the emergence of social protocols that arise from the experimentation and long time usage of interactive surfaces and also includes numerous case studies, based on recent work.
C. Anslow, P. Campos, J. Jorge

Devices and Techniques for Collaboration Through Interactive Surfaces


Chapter 2. Tabletop 3D Object Manipulation with Touch and Tangibles

Tabletop environments are ideal for collaborative activities that involve moving and arranging objects. However, manipulating 3D virtual objects through the 2D interface is challenging because users’ 2D actions must be translated into 3D actions. We explore the use of touch and tangibles to aid collaboration and 3D object manipulation. Our user study shows that using touch and tangible interaction together has advantages 3D object manipulation. While most users preferred touch due to its familiarity, the tangibles were favored for some tasks.
Beryl Plimmer, Ben Brown, James Diprose, Simon Du Preez, Andrew Luxton-Reilly

Chapter 3. Spontaneous Gesture Production Patterns on Multi-touch Interactive Surfaces

Expressivity of hand movements is much greater than what current interaction techniques enable in touch-screen input. Especially for collaboration, hands are used to interact but also to express intentions, point to the physical space in which collaboration takes place, and communicate meaningful actions to collaborators. Various types of interaction are enabled by multi-touch surfaces (singe and both hands, single and multiple fingers, etc.), and standard approaches to tactile interactive systems usually fail in handling such complexity of expresion. The diversity of multi-touch input also makes designing multi-touch gestures a difficult task. We believe that one cause for this design challenge is our limited understanding of variability in multi-touch gesture articulation, which affects users’ opportunities to use gestures effectively in current multi-touch interfaces. A better understanding of multi-touch gesture variability can also lead to more robust design to support different users’ gesture preferences. In this chapter we present our results on multi-touch gesture variability. We are mainly concerned with understanding variability in multi-touch gestures articulation from a pure user-centric perspective. We present a comprehensive investigation on how users vary their gestures in multi-touch gestures even under unconstrained articulation conditions. We conducted two experiments from which we collected 6669 multi-touch gestures from 46 participants. We performed a qualitative analysis of user gesture variability to derive a taxonomy for users’ multi-touch gestures that complements other existing taxonomies. We also provide a comprehensive analysis on the strategies employed by users to create different gesture articulation variations for the same gesture type.
Yosra Rekik, Radu-Daniel Vatavu, Laurent Grisoni

Chapter 4. Remote Proxemics

Virtual meetings have become increasingly common with modern video-conference and collaborative software. While they allow obvious savings in time and resources, current technologies add unproductive layers of protocol to the flow of communication between participants, rendering the interactions far from seamless. In this work we describe in detail Remote Proxemics, an extension of proxemics aimed at bringing the syntax of co-located proximal interactions to virtual meetings. We also describe the role of Eery Space as a shared virtual locus that results from merging multiple remote areas, where meeting participants’ are located side-by-side as if they shared the same physical location. Thus rendering Remote Proxemics possible. Results from user evaluation on the proposed presence awareness techniques suggest that our approach is effective at enhancing mutual awareness between participants and sufficient to initiate proximal exchanges regardless of their geolocation, while promoting smooth interactions between local and remote people alike.
Maurício Sousa, Daniel Mendes, Daniel Medeiros, Alfredo Ferreira, João Madeiras Pereira, Joaquim Jorge

Chapter 5. Content Sharing Between Spatially-Aware Mobile Phones and Large Vertical Displays Supporting Collaborative Work

Large vertical displays are increasingly widespread, and content sharing between them and personal mobile devices is central to many collaborative usage scenarios. In this chapter we present FlowTransfer, bidirectional transfer techniques which make use of the mobile phone’s position and orientation. We focus on three main aspects: multi-item transfer and layout, the dichotomy of casual versus precise interaction, and support for physical navigation. Our five techniques explore these aspects in addition to being contributions in their own right. They leverage physical navigation, allowing seamless transitions between different distances to the display, while also supporting arranging content and copying entire layouts within the transfer process. This is enabled by a novel distance-dependent pointing cursor that supports coarse pointing from distance as well as precise positioning at close range. We fully implemented all techniques and conducted a qualitative study documenting their benefits. Finally, based on a literature review and our holistic approach in designing the techniques, we also contribute an analysis of the underlying design space.
Ricardo Langner, Ulrich von Zadow, Tom Horak, Annett Mitschick, Raimund Dachselt

Chapter 6. Interactive Exploration of Three-Dimensional Scientific Visualizations on Large Display Surfaces

The chapter surveys the different approaches investigated to interact with scientific visualizations on large surfaces such as tables and walls. The chapter particularly does not focus on VR-based interaction or tangible input but on those interaction techniques where the input is provided on the surface itself or where it is focused on the surface. In particular, tactile interaction techniques are covered and the challenges of gestural input as well as of combining touch input with stereoscopic rendering are discussed. Where possible, connections to collaborative interaction scenarios are pointed out, even though most publications to date focus on single-user interaction.
Tobias Isenberg

Chapter 7. CubIT: Design and Evaluation of a Collaboration-Tool for Large Interactive Wall Surfaces

In this book chapter we describe the design and evaluation of CubIT, a multi-user presentation and collaboration system installed at the Queensland University of Technology’s (QUT) Cube facility. The ‘Cube’ is an interactive visualisation facility made up of five very large-scale interactive multi-panel wall displays, each consisting of up to twelve 55-inch multi-touch screens (48 screens in total) and additional very large projected display screens situated above the display panels. The chapter outlines the unique design challenges, features, implementation and evaluation of CubIT. The system was built to make the Cube facility accessible to QUT’s academic and student population. CubIT enables users to easily upload and share their own media content, and allows multiple users to simultaneously interact with the Cube’s wall displays. The features of CubIT are made available via three user interfaces, a multi-touch interface working on the wall displays, a mobile phone and tablet application and a web-based content management system. Each of these interfaces play different roles and offers different interaction mechanisms, appropriate to the underlying platform. Through its interfaces CubIT supports a wide range of collaborative features including multi-user shared workspaces, drag and drop upload and sharing between users, session management and dynamic state control between different parts of the system. The results of our longitudinal evaluation study showed that CubIT was successfully used for a variety of tasks, but also highlighted specific challenges with regards to user expectations as well as issues arising from public use.
Markus Rittenbruch

Chapter 8. Shared Façades: Surface-Embedded Layout Management for Ad Hoc Collaboration Using Head-Worn Displays

Collaboration is a necessary, everyday human activity, yet computing environments specifically designed to support collaborative tasks have typically been aimed toward groups of experts in extensive, purpose-built environments. The cost constraints and design complexities of fully-networked, multi-display environments have left everyday computer users in the lurch. However, the age of ubiquitous networking and wearable technologies has been accompanied by functional head-worn displays (HWDs), which are now capable of creating rich, interactive environments by overlaying virtual content onto real-world objects and surfaces. These immersive interfaces can be leveraged to transform the abundance of ordinary surfaces in our built environment into ad hoc collaborative multi-display environments. This paper introduces an approach for distributing virtual information displays for multiple users. We first describe a method for producing spatially-constant virtual window layouts in the context of single users. This method applies a random walk algorithm to balance multiple constraints, such as spatial constancy of displayed information, visual saliency of the background, surface-fit, occlusion and relative position of multiple windows, to produce layouts that remain consistent across multiple environments while respecting the local geometric features of the surroundings. We then describe how this method can be generalized to include additional constraints from multiple users. For example, the algorithm can take the relative poses of two or more users into account, to prevent information from being occluded by objects in the environment from the perspective of each participant. In this paper, we however focus on describing how to make the content spatially-constant for one user, and discuss how it scales from one to multiple closely confined users. We provide an initial validation of this approach including quantitative and qualitative data in a user study. We evaluate weighting schemes with contrasting emphasis on spatial constancy and visual saliency, to determine how easily a user can locate spatially-situated information within the restricted viewing field of current head-worn display technology. Results show that our balanced constraint weighting schema produces better results than schemas that consider spatial constancy or visual saliency alone, when applied to models of two real-world test environments. Finally, we discuss our plans for future work, which will apply our window layout method in collaborative environments, to assist wearable technology users to engage in ad hoc collaboration with everyday analytic tasks.
Barrett Ens, Eyal Ofek, Neil Bruce, Pourang Irani

Chapter 9. Is It in Your Eyes? Explorations in Using Gaze Cues for Remote Collaboration

According to previous research, head mounted displays (HMDs) and head worn cameras (HWCs) are useful for remote collaboration. These systems can be especially helpful for remote assistance on physical tasks, when a remote expert can see the workspace of the local user and provide feedback. However, a HWC often has a wide field of view and so it may be difficult to know exactly where the local user is looking. In this chapter we explore how head mounted eye-tracking can be used to convey gaze cues to a remote collaborator. We describe two prototypes developed that integrate an eye-tracker with a HWC and see-through HMD, and results from user studies conducted with the systems. Overall, we found that showing gaze cues on a shared video appears to be better than just providing the video on its own, and combining gaze and pointing cues is the most effective interface for remote collaboration among the conditions tested. We also discuss the limitations of this work and present directions for future research.
Mark Billinghurst, Kunal Gupta, Masai Katsutoshi, Youngho Lee, Gun Lee, Kai Kunze, Maki Sugimoto

Case Studies and Applications


Chapter 10. Usage of Interactive Event Timelines in Collaborative Digital Tabletops Involving Automation

Tabletop computers are increasingly being used for complex, collaborative scenarios, such as emergency response. In such scenarios, maintaining situation awareness of dynamic changes automated by the system is crucial for users to make optimal decisions. If the system does not provide users with appropriate feedback, they can become confused and “out-of-the-loop” about the current system state, leading to suboptimal decisions or actions. To enhance situation awareness of dynamic changes occurring in the collaborative tabletop environment, we designed an interactive event timeline to enable exploration of historical system events. We conducted a user study to understand how various design alternatives of interactive event timelines impacted situation awareness in the context of a cooperative tabletop game. Our initial results showed that, on average, all groups had a high combined level of situation awareness, regardless of the given timeline designs. To better understand what role the timelines played for the groups, we conducted an in-depth video analysis. Participants used the timelines mostly for perceiving new changes by interacting with the detailed information. The analysis also revealed the benefits of the high-level information presented in the timelines for projecting future system states. The information presented in the timelines was considered as the correct historical account and was used to negotiate the knowledge of automated changes. We also report on how other system features, in addition to the timelines, were used for situation awareness maintenance. Finally, we discuss implications for designing interactive event timelines for co-located collaborative systems involving automated events.
Y.-L. Betty Chang, Stacey D. Scott, Mark Hancock

Chapter 11. Activity-Based Collaboration for Interactive Spaces

Activity-based computing (ABC) is a conceptual and technological framework for designing interactive systems that offers a better mapping between the activities people conduct and the digital entities they use. In ABC, rather than interacting directly with lower-level technical entities like files, folder, documents, etc., users are able to interact with ‘activities’ which encapsulate files and other low-level resources. In ABC an ‘activity’ can be shared between collaborating users and can be accessed on different devices. As such, ABC is a framework that suits the requirements of designing interactive spaces. This chapter provides an overview of ABC with a special focus on its support for collaboration (‘Activity Sharing’) and multiple devices (‘Activity Roaming’). These ABC concepts are illustrated as implemented in two different interactive spaces technologies; ReticularSpaces [1] and the eLabBench [2, 3]. The chapter discusses the benefits of activity-based collaboration support for these interactive spaces, while also discussing limitations and challenges to be addressed in further research.
Jakob E. Bardram, Morten Esbensen, Aurélien Tabard

Chapter 12. Collaborative Business Process Modeling in Multi-surface Environments

Analyzing and redesigning business processes is a complex task which requires the collaboration of multiple actors such as process stakeholders, domain experts and others. Current collaborative modeling approaches mainly focus on modeling workshops where participants verbally contribute their perspective on a process along with ideas on how to improve it. These workshops are supported by modeling experts who facilitate the workshop and translate participants’ verbal contributions into a process model. Being limited to verbal contributions however might negatively affect the motivation of participants to actively contribute. Interactive technology such as smartphones, tablets, digital tabletops and interactive walls can provide opportunities for participants to directly interact with process models. Multi surface environments where different interactive technologies (e.g. display walls, tabletops, tablets, and mobiles) are combined also allow for orchestrating different modes of collaboration. In this chapter we describe an approach that combines different styles of collaboration using various interactive surfaces in a multi surface environment. Testing this approach in three different settings we found indications that interactive technology not only improves involvement by participants but also speeds up workshops and improves the quality of collaboration outcomes. The studies also revealed means for improving the proposed approach.
Alexander Nolte, Ross Brown, Craig Anslow, Moritz Wiechers, Artem Polyvyanyy, Thomas Herrmann

Chapter 13. Interactive Digital Cardwalls for Agile Software Development

Agile software development is characterized by very intensive communication and collaboration among members of the software development team and external stakeholders. In this context, we look specifically at cardwalls, noting that despite the wide availability of digital cardwalls, most Agile teams still use physical cardwalls to support their collaborative events. This is true even though a physical cardwall hinders efficient distributed software development and causes extra effort to capture story artefacts into digital tools to meet traceability and persistence requirements. We conducted two empirical studies in industry to understand the use of existing digital Agile cardwalls and to find out the needs for an ideal digital Agile cardwall. The first study was with eight Agile teams of committed digital cardwall users. The study showed the reasons why some teams use projected digital cardwalls and their detailed experiences with them. The study showed that most digital cardwalls seem not be sufficient for the highly interactive and collaborative Agile workstyle. The second study was with eleven Agile companies. The study comprised of the development of aWall, a software prototype of a large interactive high-resolution multi-touch display that supports varied Agile meetings where cardwalls are used. The results of the study emerged with design considerations for digital Agile cardwalls from the evaluation of aWall in a user workshop. Both studies, which were conducted concurrently, began with an interest in new large interactive surface technologies which might have the potential to provide not only the required interaction possibilities to support intensive collaboration, but also the required large display format necessary for a collaborative space. The results of the studies collectively seem to confirm our assumption, that large interactive surface technologies could bring the support for the collaboration of Agile teams to a new level, potentially making the teams more productive.
Martin Kropp, Judith M. Brown, Craig Anslow, Stevenson Gossage, Magdalena Mateescu, Robert Biddle

Chapter 14. Collaborative Interaction with Geospatial Data—A Comparison of Paper Maps, Desktop GIS and Interactive Tabletops

Over the last two decades, researchers have thoroughly investigated the benefits and challenges of large interactive surfaces, highlighting in particular their potential for efficient co-located collaboration and coping with rich content (complex diagrams, multi-layer digital maps, etc.). However, comparative studies that actually evaluate the same tasks on tabletops and other types of systems are still scarce. We have identified crisis management (CM) as promising application context, in which to study such tasks. In CM, people from different organizations use, among others, large paper maps to establish a common understanding of a critical situation, and plan and coordinate appropriate countermeasures. What sets CM apart from other application areas are the very formalized (and different) user roles, and the variations in completeness of the operational picture between involved organizations, both necessitating regular information exchange and collaboration in planning. Based on these characteristics, we have designed a system for interactive tabletops that facilitates collaborative situation analysis and planning by users having different information and planning functionality available. We have then conducted a comparative study, in which 30 participants performed tasks reflecting actual CM work on the tabletop system, classical paper maps and an off-the-shelf desktop GIS. Our goal was to quantify the benefits of tabletops w.r.t. performance, usability, and teamwork quality. We found that users were most efficient using the tabletop and perceived its UX as superior; also, the tabletop offered a teamwork quality comparable to classical paper maps. This indicates that tabletops may indeed be a valuable tool for collaboration in crisis management, and, more generally, for all application areas in which users with different roles collaborate around geospatial data.
Sebastian Döweling, Tarik Tahiri, Jan Riemann, Max Mühlhäuser

Chapter 15. Envisioning the Emergency Operations Centre of the Future

Emergencies, crises, and disasters happen frequently, with significant impact on the lives of countless people. To respond to these events, many organizations including the Police, EMS, and Fire departments work together in a collaborative effort to mitigate the effects of these events. In addition, these agencies are often joined by third-party organizations such as the Red Cross or utility companies. For all of these groups to work together, an Emergency Operations Centre (EOC) acts as a hub for centralized communication and planning. Despite the significant role of the EOC, many existing EOCs still rely on aging technologies, leaving many potential improvements available by adopting new technologies. Considering the impact of emergencies on human lives and lost resources, and the scale of these emergencies, even a minor improvement can lead to significant benefits and cost-savings. Emergency Operations Centre of the Future (EOC-F) is an exploration into the integration of various novel technologies in EOC design, in an effort to make emergency response more efficient and collaborative. We have built a multi-surface environment (MSE) which utilizes various digital surfaces including display walls, tabletops, tablet devices, and mobile/wearable computing devices. Within this multi-surface EOC, we look at proxemic interactions and augmented reality as useful ways to transfer and access information. We also discuss how analysis of information gathered within the EOC, as well as social media, can lead to more informed decision making during emergency response.
Edwin Chan, Craig Anslow, Teddy Seyed, Frank Maurer

Chapter 16. Security in User Interfaces Distributed Amongst Dynamic Sets of Devices and Users

Given the increasing availability of many devices in our daily environments, distributed user interfaces are becoming more and more used. However, they raise many issues related to security, which current frameworks for distributed user interfaces have yet to address adequately. We present a solution for this purpose, able to exploit public key certificates for authentication and encryption as well. The presented solution consists of a reference software architecture for secure distributed user interfaces and a corresponding implementation. We also report on an example application for a city guide supporting collaborative cross-device user interfaces.
Luca Frosini, Fabio Paternò

Chapter 17. Surface Applications for Security Analysis

This chapter relates to human factors in computer security, and how surface technology might support security analysis. This specific domain allowed us to investigate surface application design and development in an established context, and thus learn how the real needs of the domain might best be supported. Throughout, we were fortunate to have partners in industry and government working in the domain who were able to give us advice and feedback. A number of projects were conducted over the span of our research program, each one offering findings that informed later projects. In this chapter, we provide an outline of our work, summarizing each of the main projects and their findings. We cover: (1) a literature review. (2) Ethnographic studies of firstly operators and technicians in seven operations centres, and secondly a team of ten professional analysts working in the security domain; (3) ACH Walkthrough, a collaborative web-based decision-making tool; (4) Ra, a tool that supports rollback, playback and other explorative actions when using web applications like ACH Walkthrough; and (5) Strata, a tool that allows for the annotation of web applications, enabling the work of collaborative teams.
Judith M. Brown, Jeff Wilson, Peter Simonyi, Miran Mirza, Robert Biddle

Chapter 18. Collaboration Around an Interactive Tabletop in Rehabilitation Settings

Acquired brain injury, mostly after a stroke or an accident, is a hard cut in a person’s life and often followed by a long process of rehabilitation with many ups and downs. Therapy can be perceived as monotonous due to the (therapeutically necessary) repetitive nature of the tasks. Therapy nowadays does not only involve conventional settings but often additional computer-based exercises that allow the computer to take over time-consuming routine tasks. In addition to the time factor, computer-assisted therapy can lead to higher patient motivation. Especially, when computer-based rehabilitation allows for collaborative settings, a positive effect on motivation can be noted. Collaboration can be easily facilitated with tabletop computers because they can be interacted with by multiple people in parallel. Modern tabletops can process a high number of concurrent interactions and the user interfaces can be designed in a way that allows for relevant (interactive) elements to be aligned towards different directions. This chapter presents an approach towards rehabilitation using an interactive tabletop in collaborative settings, covering the therapeutic motivation behind as well as aspects related to interaction design and modalities.
Mirjam Augstein, Thomas Neumayr, Renate Ruckser-Scherb, Sabine Dielacher

Chapter 19. Visual to Non-visual Collaboration on a Dynamic Tactile Graphics Display

In this article, a collaborative workstation for sighted and visually impaired users is proposed. It can be used for creating tactile graphics in a collaborative manner. The workstation consists of a classical drawing application extended with tools supporting the design of tactile graphics as well as a non-visual interface to the drawing application. As a result, blind users get both auditory and tactile feedback from the workstation through a dynamic planar tactile pin-matrix device. We also introduce supporting features as well as discuss problems in collaboration. Ultimately, we provide a set of recommendations for building a collaborative system with these different interface modalities.
Jens Bornschein, Denise Prescher

Chapter 20. Rich Digital Collaborations in a Small Rural Community

In this chapter we describe experience in the design and installation of a low-cost multi-touch table in a rural island community. We discuss the creation of the table including: pragmatic challenges of installation, and then re-installation as the physical fabric of the multi-purpose building (café, cinema, meeting area and cattle market) altered; technical challenges of using off-the-shelf components to create state-of-the art multi-touch interactions and tactile BYOD (bring your own device) end-user programming; design challenges of creating high-production value bespoke mountings and furniture using digital fabrication in an environment that could include sewing needles, ketchup laden sandwiches and cow manure. The resulting installation has been used in semi-in-the-wild studies of bespoke applications, leading to understandings of the way small communities could use advanced interactions. More broadly this sits within a context of related studies of information technology in rural developments and a desire to understand how communities can become users of the rich streams of open data now available, and, perhaps more important, offer ways in which small communities can become empowered through the creation and control of their own data.
Alan Dix, Alessio Malizia, Tommaso Turchi, Steve Gill, Gareth Loudon, Richard Morris, Alan Chamberlain, Andrea Bellucci
Weitere Informationen