Nitin Sawhney
Chris Schmandt
Abstract
Mobile workers need seamless access to communication and information services while on the move. However, current solutions overwhelm users with intrusive interfaces and ambiguous notifications. This article discusses the interaction techniques developed for Nomadic Radio, a wearable computing platform for managing voice and text-based messages in a nomadic environment. Nomadic Radio employs an auditory user interface, which synchronizes speech recognition, speech synthesis, nonspeech audio, and spatial presentation of digital audio, for navigating among messages as well as asynchonous notific ation of nely arrived messages. Emphasis is placed on an auditory modality as Nomadic Radio is designed to be used while performing other tasks in a user's everyday environment; a range of auditory cues provides peripheral awareness of incoming messages. Notification is adaptive and cntext sensitive; messages are presented as more or less obtrsive based on importance inferred from content filtering, whether the user is engaged in conversation and his or her own recent responses to prior messages. Auditory notifications are dynamically scaled from ambient sound through recorded voice cues up to message summaries. Iterative design and a preliminary user evaluation suggest that audio is an appropriate medium for mobile messaging, but that care must be taken to minimally intrude on the wearer's social and physical environment.
- ACKERMAN, M. S., STARR, B., HINDUS, D., AND MAINWARING, S. D. 1997. Hanging on the 'Wire: A field study of an audio-only media space. ACM Trans. Comput. Hum. Interact. 4, 1, 39-66. Google Scholar
- AKOI, S., MITSUHASHI, K., NISHINO, Y., AND MURAKAMI, K. 1998. Noise-Suppressing Compact Microphone/Receiver Unit. NTT Rev. 10, 6, 102.Google Scholar
- ARONS, B. 1992. A Review of the Cocktail Party Effect. J. Amer. Voic. 12.Google Scholar
- ARONS, B. 1997. SpeechSkimmer: a system for interactively skimming recorded speech. ACM Trans. Comput. Hum. Interact. 4, 1, 3-38. Google Scholar
- BEDERSON, B. B. 1995. Audio augmented reality: A prototype automated tour guide. In Conference Companion for the ACM Conference on Human Factors in Computing Systems (CHI '95, Denver, CO, May 7-11), I. Katz, R. Mack, and L. Marks, Eds. ACM Press, New York, NY, 210-211. Google Scholar
- CHALFONTE, B. L., FISH, R. S., AND KRAUT, R. E. 1991. Expressive richness: a comparison of speech and text as media for revision. In Proceedings of the Conference on Human Factors in Computing Systems: Reaching through Technology (CHI '91, New Orleans, LA, Apr. 27-May 2), S. P. Robertson, G. M. Olson, and J. S. Olson, Eds. ACM Press, New York, NY, 21-26. Google Scholar
- CLARKSON, B., SAWHNEY, N., AND PENTLAND, A. 1998. Auditory Context Awareness via Wearable Computing. In Proceedings on Perceptual User Interfaces Workshop (PIU '98, San Francisco, CA, Nov. 5-6), M. Turk, Ed. 37. http://research.microsoft.com/ PUIWorkshop/ PUI98.htm.Google Scholar
- COHEN, J. 1994. Monitoring Background Activities. In Auditory Display: Sonification, Audification, and Auditory Interfaces. Addison-Wesley, Reading, MA.Google Scholar
- FUKUMOTO, M. AND TONOMURA, Y. 1999. Whisper: A Wristwatch Style Wearable Handset. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '99, Pittsburgh, PA, May). ACM Press, New York, NY, 112. Google Scholar
- GARDNER, W. G. AND MARTIN, K. D. 1995. HRTF measurements of a KEMAR. J. Acoust. Soc. Amer. 97, 6, 3907.Google Scholar
- GAVER, W. W., SMITH, R. B., AND O'SHEA, T. 1991. Effective sounds in complex systems: The ARKOLA simulation. In Proceedings of the Conference on Human Factors in Computing Systems: Reaching through Technology (CHI '91, New Orleans, LA, Apr. 27-May 2), S. P. Robertson, G. M. Olson, and J. S. Olson, Eds. ACM Press, New York, NY, 85-90. Google Scholar
- HORVITZ, E. AND JED, L. 1997. Perception, Attention, and Resources: A Decision-Theoretic Approach to Graphics Rendering. In Proceedings on Uncertainty in Artificial Intelligence. 238. Google Scholar
- HUDSON, S. E. AND SMITH, I. 1996. Electronic mail previews using non-speech audio. In Proceedings of the CHI '96 Conference Companion on Human Factors in Computing Systems: Common Ground (CHI '96, Vancouver, British Columbia, Canada, Apr. 13-18), M. J. Tauber, Ed. ACM Press, New York, NY, 237-238. Google Scholar
- KAELBLING, L. P. AND LITTMAN, M. L. 1996. Reinforcement Learning: A Survey. J. Artif. Intell. Res. 4, 237. Google Scholar
- KOBAYASHI, M. AND SCHMANDT, C. 1997. Dynamic Soundscape: Mapping time to space for audio browsing. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '97, Atlanta, GA, Mar. 22-27), S. Pemberton, Ed. ACM Press, New York, NY, 194-201. Google Scholar
- MARTIN, G. L. 1989. The utility of speech input in user-computer interfaces. Int. J. Man-Mach. Stud. 30, 4 (Apr.), 355-375. Google Scholar
- MARX, M. AND SCHMANDT, C. 1996. CLUES: dynamic personalized message filtering. In Proceedings of the 1996 ACM Conference on Computer-Spported Cooperative Work (CSCW '96, Boston, MA, Nov. 16-20), M. S. Ackerman, Ed. ACM Press, New York, NY, 113-121. Google Scholar
- MYNATT, E. D., BACK, M., WANT, R., BAER, M., AND ELLIS, J. B. 1998. Designing audio aura. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '98, Los Angeles, CA, Apr. 18-23), C.-M. Karat, A. Lund, J. Coutaz, and J. Karat, Eds. ACM Press/Addison-Wesley Publ. Co., New York, NY, 566-573. Google Scholar
- O'CONAILL, B. AND FROHLICH, D. 1995. Timespace in the workplace: dealing with interruptions. In Conference Companion for the ACM Conference on Human Factors in Computing Systems (CHI '95, Denver, CO, May 7-11), I. Katz, R. Mack, and L. Marks, Eds. ACM Press, New York, NY, 262-263. Google Scholar
- ROY, D. K. AND SCHMANDT, C. 1996. NewsComm: a hand-held interface for interactive access to structured audio. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '96, Vancouver, B.C., Canada, Apr. 13-18), M. J. Tauber, Ed. ACM Press, New York, NY, 173-180. Google Scholar
- RUDNICKY, A., REED, S., AND THAYER, E. 1996. SpeechWear: A mobile speech system. In Proceedings on ICSLP '96 (ICSLP '96).Google Scholar
- SAWHNEY, N. 1997. Situational Awareness from Environmental Sounds.Google Scholar
- SAWHNEY, N. AND SCHMANDT, C. 1997. Speaking and Listening on the Run: Design for Wearable Audio Computing. In Proceedings of the IEEE International Symposium on Wearable Computing (Oct.). IEEE Computer Society, Washington, DC, 108. Google Scholar
- SAWHNEY, N. AND SCHMANDT, C. 1999. Nomadic Radio: Scaleable and Contextual Notification for Wearable Audio Messaging. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '99, Pittsburgh, PA, May). ACM Press, New York, NY, 96. Google Scholar
- SCHMANDT, C. 1994. Multimedia Nomadic Services on Today's Hardware. IEEE Network (Sept./Oct.), 12.Google Scholar
- SCHMANDT, C. AND MULLINS, A. 1995. AudioStreamer: exploiting simultaneity for listening. In Conference Companion for the ACM Conference on Human Factors in Computing Systems (CHI '95, Denver, CO, May 7-11), I. Katz, R. Mack, and L. Marks, Eds. ACM Press, New York, NY, 218-219. Google Scholar
- STARNER, T., MANN, S., RHODES, B., LEVINE, J., HEALEY, J., KIRSCH, D., PICARD, R., ANDGoogle Scholar
- PENTLAND, A. 1997. Augmented reality through wearable computing. Presence: Teleoper. Virtual Environ. 6, 4.Google Scholar
- STIFELMAN, L. J., ARONS, B., SCHMANDT, C., AND HULTEEN, E. A. 1993. VoiceNotes: A speech interface for a hand-held voice notetaker. In Proceedings of the ACM Conference on Human Factors in Computing (INTERCHI '93, Amsterdam, The Netherlands, Apr. 24-29), S. Ashlund, A. Henderson, E. Hollnagel, K. Mullet, and T. White, Eds. ACM Press, New York, NY, 179-186. Google Scholar
- SUZUKI, Y., NAKADAI, Y., SHIMAMURA, Y., AND NISHINO, Y. 1998. Development of an Integrated Wristwatch-type PHS Telephone. NTT Rev. 10, 6, 93.Google Scholar
- WENZEL, E. M. 1992. Localization in virtual acoustic displays. Presence: Teleoper. Virtual Environ. 1, 1 (Winter), 80-107. Google Scholar
- WHITTAKER, S., HIRSCHBERG, J., AND NAKATANI, C. H. 1998. All talk and all action: strategies for managing voicemail messages. In Proceedings of the CHI 98 summary conference on CHI 98 summary: human factors in computing systems (CHI '98, Los Angeles, CA, Apr. 18-23), C.-M. Karat and A. Lund, Chairs. ACM Press, New York, NY, 249-250. Google Scholar
- YANKELOVICH, N. 1994. Talking vs. Taking: Speech Access to Remote Computers. In Proceedings of the ACM Conference on Human Factors in Computing Systems: "Celebrating Interdependence" (CHI '94, Boston, MA, Apr. 24-28). ACM Press, New York, NY, 275. Google Scholar
Index Terms
- Nomadic radio: speech and audio interaction for contextual messaging in nomadic environments
Recommendations
Nomadic radio: scaleable and contextual notification for wearable audio messaging
CHI '99: Proceedings of the SIGCHI conference on Human Factors in Computing SystemsMobile workers need seamless access to communication and information services on portable devices. However current solutions overwhelm users with intrusive and ambiguous notifications. In this paper, we describe scaleable auditory techniques and a ...
Nomadic Radio: A Spatialized Audio Environment for Wearable Computing
ISWC '97: Proceedings of the 1st IEEE International Symposium on Wearable ComputersThis paper describes an on-going research project at the MIT Media Lab, exploring the use of auditory I/O as a primary interaction modality for wearable computing. Nomadic Radio is a framework developed for use on a wearable audio platform. It presents ...
Using nonspeech sounds to provide navigation cues
This article describes 3 experiments that investigate the possibiity of using structured nonspeech audio messages called earcons to provide navigational cues in a menu hierarchy. A hierarchy of 27 nodes and 4 levels was created with an earcon for each ...
Comments