Elsevier

Design Studies

Volume 35, Issue 6, November 2014, Pages 559-592
Design Studies

A comparative analysis of multimodal communication during design sketching in co-located and distributed environments

https://doi.org/10.1016/j.destud.2014.04.002Get rights and content

Highlights

  • We characterized multimodal communication during distributed and co-located design sketching.

  • Our theoretical framework treats gesturing as a communication channel and a thinking medium.

  • When gesturing is restricted, graphical communication might be leveraged to compensate.

  • There is a design phase dependent interaction between gestural and graphical communication.

  • “Cross-gesturing” relates sketch content of different authors and facilitates shared understanding.

This study extends our understanding of multimodal communication during design sketching. Building on the literature, the theoretical dimension frames gesturing as a communication channel and a thinking medium, and postulates an interplay between gesturing and other channels. The empirical dimension explores the theoretical propositions in the context of co-located and distributed sketching. Quantitative analyses suggest that when gesturing is restricted, graphical communication is leveraged to compensate, and that verbal communication is incessant in both collaboration environments. They also highlight a non-compensatory design phase dependent interaction between gestural and graphical communication. Moreover, they reveal differences in the communication structure used in the two environments. Qualitative analyses identify a behavior termed “cross-gesturing,” which informs how gesturing facilitates shared understanding during collaborative sketching.

Section snippets

Sketches as design representations that mediate shared understanding

Representations are instrumental in design activity; representation construction has even been argued to define designing (Visser, 2006). Representations can reveal much about how designers structure and process information, and can be categorized according to several differentiating principles. Two principles relevant to this study are the location of the representation, and its intended audience.

When considering the first differentiating principle, a common and useful distinction is the

Protocol study

Building on the theoretical framing introduced in the previous section, we designed and conducted a protocol study in order to systematically investigate and compare multimodal communication during design sketching in co-located and distributed environments.

Data analysis framework

We analyzed the data by using video interaction analysis techniques. We watched the footage individually and as a group, and made qualitative observations. Our initial observations helped us to identify a specific behavior related to the use of deictic gestures in constructing shared understanding, and to determine how to decompose the sketching sessions into a series of communication activities over time. In consultation with the literature, we arrived at the coding scheme described below, and

Results

Each group was able to finish the design task in each condition in approximately 30 min. As also confirmed by the exit interviews, participants did not exhibit any significant difficulty in utilizing the DSS, and their transition from the whiteboard to the system was much smoother than we anticipated. This is mainly due to the simplicity, size and resolution of the system, and the use of the same sketching tools (a whiteboard marker and eraser) in both environments.

Gesturing and sketch content creation durations

The 53% decrease in the gesturing duration in the distributed condition supports our first hypothesis. Based on our qualitative observations and participants' post-session reflections, this seems to be an outcome of the decreased visual connectivity to the remote designer in the distributed setting, which is to be expected. If one cannot confirm that one's gestures are being observed and understood by the other party, one is less motivated to gesture. In the co-located setting, we observed that

Conclusion

The findings of the study extend our knowledge of the role gesturing plays during design sketching.

First, the study provides a comparative quantitative characterization of multimodal design communication during sketching in co-located and distributed environments, and serves a descriptive purpose. It identifies speech as an incessant communication channel in both co-located and distributed design settings. It also shows that hand gestures communicate information unique to the gestural channel

References (46)

  • E. Arias et al.

    Transcending the individual human mind—creating shared understanding through collaborative design

    ACM Transaction on Human-Computer Interaction

    (2000)
  • H. Arrow et al.

    Small groups as complex systems: Formation, coordination, development, and adaptation

    (2000)
  • P. Badke-Schaub et al.

    Mental models in design teams: a valid approach to performance in design collaboration?

    CoDesign

    (2007)
  • R. Bakeman et al.

    Observer agreement for timed-event sequential data: a comparison of time-based and event-based algorithms

    Behavior Research Methods

    (2009)
  • M.M. Bekker et al.

    Analysis of gestures in face-to-face design teams provides guidance for how to use groupware in design

  • A.S. Bly

    A use of drawing surfaces in different collaborative settings

  • J. Cassell

    A framework for gesture generation and interpretation

  • J. Donovan et al.

    Getting the point: the role of gesture in managing intersubjectivity in a design activity

    Artificial Intelligence for Engineering Design, Analysis and Manufacturing

    (2011)
  • C. Dym et al.

    Engineering design thinking, teaching, and learning

    Journal of Engineering Education, Special Edition

    (2005)
  • N. D'souza

    The metaphor of an ensemble: architectural design and skill convergence

    Journal of Design Research

    (2012)
  • P. Ekman et al.

    The repertoire of nonverbal behavior: categories, origins, usage and coding

    Semiotica

    (1969)
  • R. Faste

    Innovations in mechanical engineering curricula for the 1990s

    (1994)
  • E.S. Ferguson

    Engineering and the mind's eye

    (1992)
  • Cited by (70)

    • The impacts of digital design platforms on design cognition during remote collaboration: A systematic review of protocol studies

      2022, Heliyon
      Citation Excerpt :

      For example, some common cognitive comparisons are novice vs expert designers, manual (pen-and-paper, sketch) vs digital (or CAD) tools, and conventional CAD vs emerging CAD (parametric or BIM) applications. Distributed design team processes can further consider the difference between synchronous and asynchronous actions (Rahman et al., 2013), and co-located or distributed environments (Eris et al., 2014; Gül and Maher, 2009; Tang et al., 2011). Furthermore, using protocol analysis a specific cognitive process or pattern developed from a group of participants can be compared with the results of other groups undertaking the same experiment to identify similarities or differences in cognitive operations and thereby propose new, or examine existing theories about human cognitive behaviours.

    • Lost in imagined space: A psychoanalysis of participatory design

      2022, Design Studies
      Citation Excerpt :

      Design communication uses visual and verbal means, and a range of tools have been employed to inspire and engage participants, including pen, paper, prototype models, and web-based platforms. Eris et al. (2014) illustrate that the primary means of communication between participants and a designer/facilitator is in the verbal realm overlapping 90% of the time with gestures and 30–40% with graphics. Given the role of verbal communication in PD, studies have sought to understand the use of language (Ungureanu & Hartmann, 2021; Wiegers et al., 2011) and have shown verbal communications (language) to be ‘ambiguous’ and ‘vague’ (Durrant et al., 2018; Wiegers et al., 2011).

    View all citing articles on Scopus
    View full text