Jiazhi Ou
Susan R. Fussell
ABSTRACT
We present a system integrating gesture and live video to support collaboration on physical tasks. The architecture combines network IP cameras, desktop PCs, and tablet PCs to allow a remote helper to draw on a video feed of a workspace as he/she provides task instructions. A gesture recognition component enables the system both to normalize freehand drawings to facilitate communication with remote partners and to use pen-based input as a camera control device. Results of a preliminary user study suggest that our gesture over video communication system enhances task performance over traditional video-only systems. Implications for the design of multimodal systems to support collaborative physical tasks are also discussed.
- Bekker, M. M., Olson, J. S., & Olson, G. M. (1995). Analysis of gestures in face-to-face design teams provides guidance for how to use groupware in design. Proceedings of DIS 95. NY: ACM Press.]] Google Scholar
Digital Library
- Flor, N. V. (1998). Side-by-side collaboration: A case study. International Journal of Human-Computer Studies, 49, 201--222.]] Google ScholarDigital Library
- Fussell, S. R., Kraut, R. E., & Siegel, J. (2000). Coordination of communication: Effects of shared visual context on collaborative work. Proceedings of CSCW 2000 (pp. 21--30). NY: ACM Press.]] Google ScholarDigital Library
- Fussell, S. R., Setlock, L. D., & Kraut, R. E. (2003). Effects of head-mounted and scene-oriented video systems on remote collaboration on physical tasks. Proceedings of CHI 2003. NY: ACM Press.]] Google ScholarDigital Library
- Fussell, S., Setlock, L., Parker, E., & Yang, J. (2003). Assessing the value of a cursor pointing device for remote collaboration on physical tasks. Proceedings of CHI '2003. NY: ACM Press.]] Google ScholarDigital Library
- Fussell, S. R., Setlock, L. D., Yang, J., Ou, J., & Parker, E. M. (under review). Gestures over video streams to support remote collaboration on physical tasks.]]Google Scholar
- Gutwin, C., & Penner, R. (2002). Improving interpretation of remote gestures with telepointer traces. Proceedings of CSCW 2002. (pp.49-57). NY: ACM Press.]] Google ScholarDigital Library
- Ishii, H., Kobayashi, M., & Grudin, J. (1993). Integration of interpersonal space and shared workspace: ClearBoard design and experiments. ACM Transactions on Information Systems, 11, 349--375.]] Google ScholarDigital Library
- Jin, X., Liu, W., Sun, J., & Sun, Z. (2002). On-line Graphics Recognition. Proceedings of PG'02 (pp. 256--265).]] Google ScholarDigital Library
- Jorge, J., & Fonseca, M. (1999). A Simple Approach to Recognise Geometric Shapes Interactively. Proceedings of GREC'99 (pp. 266--276).]] Google ScholarDigital Library
- Kraut, R. E., Fussell, S. R., & Siegel, J. (2003). Visual information as a conversational resource in collaborative physical tasks. Human-Computer Interaction, 18, 13--49.]]Google ScholarDigital Library
- Kraut, R. E., Miller, M. D., & Siegel, J. (1996) Collaboration in performance of physical tasks: Effects on outcomes and communication, Proceedings of CSCW'96 (57--66). NY: ACM Press.]] Google ScholarDigital Library
- Kuzuoka, H., & Shoji, H. (1994). Results of observational studies of spatial workspace collaboration. Electronics and Communications in Japan, 77, 58--68.]]Google ScholarCross Ref
- Kuzuoka, H., Kosuge, T., & Tanaka, K. (1994) GestureCam: A video communication system for sympathetic remote collaboration. Proceedings of CSCW 94 (35--43). NY: ACM Press.]] Google ScholarDigital Library
- Kuzuoka, H., Oyama, S., Yamazaki, K., Suzuki, K., & Mitsuishi, M. (2000). GestureMan: A mobile robot that embodies a remote instructor's actions. Proceedings of CSCW 2000 (pp. 155--162). NY: ACM Press]] Google ScholarDigital Library
- McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.]]Google Scholar
- Ou, J., Chen, X., Fussell, S. R., & Yang, J. (in press). DOVE: Drawing over Video Environment. Multimedia 2003 Demonstrations.]] Google ScholarDigital Library
- Rubine, D. (1991). Specifying gestures by example. Computer Graphics, 25, 329--337.]] Google ScholarDigital Library
- Sezgin, M., Stahovich, T., & Davis, R. (2001). Sketch based interfaces: Early processing for sketch understanding. Proceedings of PUI-2001. NY: ACM Press.]] Google ScholarDigital Library
- Tang, J. C. (1991). Findings from observational studies of collaborative work. International Journal of Man-Machine Studies, 34, 143--160.]] Google ScholarDigital Library
Index Terms
- Gestural communication over video stream: supporting multimodal interaction for remote collaborative physical tasks
Recommendations
DOVE: drawing over video environment
MULTIMEDIA '03: Proceedings of the eleventh ACM international conference on MultimediaWe demonstrate a multimedia system that integrates pen-based gesture and live video to support collaboration on physical tasks. The system combines network IP cameras, desktop PCs, and tablet PCs (or PDAs) to allow a remote helper to draw on a video ...
An exploratory analysis of partner action and camera control in a video-mediated collaborative task
CSCW '06: Proceedings of the 2006 20th anniversary conference on Computer supported cooperative workThis paper reports on an exploratory experimental study of the relationships between physical movement and desired visual information in the performance of video-mediated collaborative tasks in the real world by geographically distributed groups. Twenty-...
Weak gaze awareness in video-mediated communication
CHI EA '05: CHI '05 Extended Abstracts on Human Factors in Computing SystemsWe present a video mediated communication system that conveys gaze information to a remote location. Unlike existing video mediated communication system, this system does not send visual information directly, only gaze position and face direction. The ...
Comments