Nelson Wong
ABSTRACT
Pointing gestures -- particularly deictic references -- are ubiquitous in face-to-face communication. However, deictic pointing can be much more difficult in collaborative virtual environments (CVEs) than in everyday life -- early studies found that the 'fragmentation' caused by the environment greatly complicated object-based communication. In the fifteen years since these studies appeared, the technologies used in CVEs have improved substantially, and several techniques for improving pointing have been proposed or implemented. What these advances mean for the problems of fragmentation and deictic gesture, however, is not clear. To find out, we conducted a new observational study of deictic pointing in a CVE with several techniques that may reduce fragmentation: extra-wide and third-person views, precise control over an avatar's pointing arm, and visual enhancements such as object highlighting and laser pointing. Our study shows that although pointing has come a long way, problems of fragmentation still occur, and that visual and view enhancements can cause new problems for collaboration, even as they solve others. In addition, the visibility of a gesture's preparatory actions remained important to study participants, even when pointing was augmented. These results provide a richer understanding of the subtlety in avatar-based deictic communication, and of the ways that this critical communication mechanism can be better supported in CVEs.
Supplemental Material
- Bekker, M., Olson, J., Olson, G. Analysis of gestures in face-to-face design teams provides guidance for how to use groupware in design. Proc. DIS, 1995, 157--166. Google Scholar
Digital Library
- Benford, S., Bowers, J., Fahlén, L.E., Greenhalgh, C., Snowdon, D. User embodiment in collaborative virtual environments. Proc. CHI, 1995, 242--249. Google ScholarDigital Library
- Bowers, J., Pycock, J., O'Brien, J. Talk and embodiment in CVEs. Proc. CHI, 1996, 58--65. Google ScholarDigital Library
- Ducheneaut, N., Wen, M.-H., Yee, N., Wadley, G. Body and mind: a study of avatar personalization in three virtual worlds. Proc. CHI, 2009, 1151--1160. Google ScholarDigital Library
- Fraser, M., Benford, S., Hindmarsh, J., Heath, C. Supporting awareness and interaction through collaborative virtual interfaces. Proc. UIST 1999, 27--36. Google ScholarDigital Library
- Fraser, M., Benford, S. Interaction effects of virtual structures. Proc. CVE, 2002, 128--134. Google ScholarDigital Library
- Goodwin, C., Goodwin, M. Seeing as a situated activity: Formulating planes. In Y. Engeström and D. Middleton, eds., Cognition and Communication at Work. Cambridge University Press, 1996, 61--95.Google ScholarCross Ref
- Goodwin, C. Action and embodiment within situated human interaction. J. Prag. 32, 10, 2000, 1489--1522.Google ScholarCross Ref
- Goodwin, C. Pointing as situated practice. In S. Kita, ed., Pointing: Where language, culture and cognition meet. Psychology Press, 2003, 217--241.Google Scholar
- Greenberg, S., Gutwin, C., Roseman, M. Semantic telepointers for groupware. Proc. OzCHI, 1996, 54--61. Google ScholarDigital Library
- Gutwin, C., Penner, R. Improving interpretation of remote gestures with telepointer traces. Proc. CSCW, 2002, 49--57. Google ScholarDigital Library
- Heath, C., Hindmarsh, J. Configuring Action in Objects: From Mutual Space to Media Space. Mind, Culture, and Activity 7, 1, 2000, 81--104.Google Scholar
- Hindmarsh, J., Fraser, M., Heath, C., Benford, S., and Greenhalgh, C. Fragmented interaction: establishing mutual orientation in virtual environments. Proc. CSCW, 1998, 217--226. Google ScholarDigital Library
- Hindmarsh, J., Fraser, M., Heath, C., Benford, S., Object-focused interaction in collaborative virtual environments. ToCHI 7, 4, 2000, 477--509. Google ScholarDigital Library
- Hindmarsh, J., Heath, C. Embodied reference: A study of deixis in workplace interaction. Journal of Pragmatics 32, 12, 2000, 1855--1878.Google ScholarCross Ref
- Kirk, D., Rodden, T., Fraser, D.S. Turn it this way: grounding collaborative action with remote gestures. Proc. CHI, 2007, 1039--1048. Google ScholarDigital Library
- Krauss, R.M., Fussell, S.R. Mutual knowledge and communicative effectiveness. In Intellectual teamwork: social and technological foundations of cooperative work. L. Erlbaum Associates Inc., 1990, 111--145. Google ScholarDigital Library
- Luff, P., Heath, C., Kuzuoka, H., Yamazaki, K., Yamashita, J. Handling documents and discriminating objects in hybrid spaces. Proc. CHI, 2006, 561--570. Google ScholarDigital Library
- Luff, P., Yamashita, N., Kuzuoka, H., Heath, C. Hands on hitchcock: embodied reference to a moving scene. Proc. CHI, 2011, 43--52. Google ScholarDigital Library
- McNeill, D. Hand and mind. Univ. Chicago. 1992.Google Scholar
- Moore, R.J., Ducheneaut, N., Nickell, E. Doing Virtually Nothing: Awareness and Accountability in Massively Multiplayer Online Worlds. Proc. CSCW, 2007, 265--305. Google ScholarDigital Library
- Tang, J.C. Findings from observational studies of collaborative work. In International Journal of Man Machine Studies. Academic Press Ltd., 1991, 11--28. Google ScholarDigital Library
- Tang, J.C., Minneman, S. VideoWhiteboard: video shadows to support remote collaboration. Proc. CHI, 1991, 315--322. Google ScholarDigital Library
- Wong, N., Gutwin, C. Where are you pointing? the accuracy of deictic pointing in CVEs. Proc. CHI, 2010, 1029--1038. Google ScholarDigital Library
- Wong, N. and Gutwin, C. Controlling an Avatar's Pointing Gestures in Desktop Collaborative Virtual Environments. Proc. Group, 2012, 21--30. Google ScholarDigital Library
Index Terms
- Support for deictic pointing in CVEs: still fragmented after all these years'
Recommendations
Where are you pointing?: the accuracy of deictic pointing in CVEs
CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsDeictic reference -- pointing at things during conversation -- is ubiquitous in human communication, and should also be an important tool in distributed collaborative virtual environments (CVEs). Pointing gestures can be complex and subtle, however, and ...
Controlling an avatar's pointing gestures in desktop collaborative virtual environments
GROUP '12: Proceedings of the 2012 ACM International Conference on Supporting Group WorkCollaborative Virtual Environments (CVEs) allow people to interact with each other in virtual worlds through computer-generated avatars. Avatars are much less expressive than real bodies, and one main limitation is their lack of support for non-verbal ...
Accuracy of interpreting pointing gestures in egocentric view
UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous ComputingCommunicating spatial information by pointing is ubiquitous in human interactions. With the growing use of head-mounted cameras for collaborative purposes, it is important to assess how accurately viewers of the resulting egocentric videos can interpret ...
Comments