Skip to main content

2017 | OriginalPaper | Buchkapitel

Vouch-T: Multimodal Text Input for Mobile Devices Using Voice and Touch

verfasst von : Minyoung Lee, Gerard J. Kim

Erschienen in: Human-Computer Interaction. Interaction Contexts

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Entering text on a small mobile device is not easy due to the relatively small screen space and the fat finger problem. We consider a multimodal text input method, called Vouch-T (Voice + tOUCH - Text) combining the touch and voice input in complementary way. With Vouch-T, the user makes an approximate touch among densely distributed alphabetic keys, accompanied with the voice input to effectively disambiguate the target among possible candidates. We have assessed the potential of Vouch-T in terms of the usability and performance compared to the conventional touch-only based method. We considered two types of text input layout, namely, the QWERTY and 3 × 4 telephone keypad on two sizes of mobile devices, the smart phone and smart watch. The comparative simulation experiment validated that the multimodal approach of Vouch-T improves the input performance and usability for the smaller-sized smart watch, but only marginally for the larger smart phone.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
In a previous work, we have suggested for the same basic idea, combining touch and voice, for digit input on smart watches [27].
 
Literatur
1.
Zurück zum Zitat Albinsson, P.A., Zhai, S.: High precision touch screen interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2003), pp. 105–112. ACM (2003) Albinsson, P.A., Zhai, S.: High precision touch screen interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2003), pp. 105–112. ACM (2003)
2.
Zurück zum Zitat Baudisch, P., Chu, G.: Back-of-device interaction allows creating very small touch devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2009), pp. 1923–1932. ACM (2009) Baudisch, P., Chu, G.: Back-of-device interaction allows creating very small touch devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2009), pp. 1923–1932. ACM (2009)
3.
Zurück zum Zitat Bolt, R.A.: “Put-that-there”: Voice and Gesture at the Graphics Interface. ACM, New York (1980)CrossRef Bolt, R.A.: “Put-that-there”: Voice and Gesture at the Graphics Interface. ACM, New York (1980)CrossRef
4.
Zurück zum Zitat Cha, J.M., Choi, E., Lim, J.: Virtual sliding QWERTY: a new text entry method for smartwatches using Tap-N-Drag. Appl. Ergon. 51, 263–272 (2015)CrossRef Cha, J.M., Choi, E., Lim, J.: Virtual sliding QWERTY: a new text entry method for smartwatches using Tap-N-Drag. Appl. Ergon. 51, 263–272 (2015)CrossRef
5.
Zurück zum Zitat Chen, X.A., Grossman, T., Fitzmaurice, G.: Swipeboard: a text entry technique for ultra-small interfaces that supports novice to expert transitions. In: Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST 2014), pp. 615–620. ACM (2014) Chen, X.A., Grossman, T., Fitzmaurice, G.: Swipeboard: a text entry technique for ultra-small interfaces that supports novice to expert transitions. In: Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST 2014), pp. 615–620. ACM (2014)
6.
Zurück zum Zitat Cho, H., Kim, M., Seo, K.: A text entry technique for wrist-worn watches with tiny touchscreens. In: Proceedings of the Adjunct Publication of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST 2014), pp. 79–80. ACM (2014) Cho, H., Kim, M., Seo, K.: A text entry technique for wrist-worn watches with tiny touchscreens. In: Proceedings of the Adjunct Publication of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST 2014), pp. 79–80. ACM (2014)
7.
Zurück zum Zitat Darbara, R., Senb, P. K., Dasha, P., Samantaa, D.: Using hall effect sensors for 3D space text entry on smartwatches. In: Proceedings of the 7th International Conference on Intelligent Human Computer Interaction (IHCI 2015) (2015) Darbara, R., Senb, P. K., Dasha, P., Samantaa, D.: Using hall effect sensors for 3D space text entry on smartwatches. In: Proceedings of the 7th International Conference on Intelligent Human Computer Interaction (IHCI 2015) (2015)
8.
Zurück zum Zitat Dunlop, M.D., Komninos, A., Durga, N.: Towards high quality text entry on smartwatches. In: CHI 2014 Extended Abstracts on Human Factors in Computing Systems, pp. 2365–2370. ACM (2014) Dunlop, M.D., Komninos, A., Durga, N.: Towards high quality text entry on smartwatches. In: CHI 2014 Extended Abstracts on Human Factors in Computing Systems, pp. 2365–2370. ACM (2014)
9.
Zurück zum Zitat Dusan, S., Gadbois, G. J., Flanagan, J.L.: Multimodal interaction on PDA’s integrating speech and pen inputs. In: INTERSPEECH (2003) Dusan, S., Gadbois, G. J., Flanagan, J.L.: Multimodal interaction on PDA’s integrating speech and pen inputs. In: INTERSPEECH (2003)
10.
Zurück zum Zitat Funk, M., Sahami, A., Henze, N., Schmidt, A.: Using a touch-sensitive wristband for text entry on smart watches. In: CHI 2014 Extended Abstracts on Human Factors in Computing Systems, pp. 2305–2310. ACM (2014) Funk, M., Sahami, A., Henze, N., Schmidt, A.: Using a touch-sensitive wristband for text entry on smart watches. In: CHI 2014 Extended Abstracts on Human Factors in Computing Systems, pp. 2305–2310. ACM (2014)
11.
Zurück zum Zitat Holzinger, A.: Finger instead of mouse: touch screens as a means of enhancing universal access. In: Carbonell, N., Stephanidis, C. (eds.) UI4ALL 2002. LNCS, vol. 2615, pp. 387–397. Springer, Heidelberg (2003). doi:10.1007/3-540-36572-9_30 CrossRef Holzinger, A.: Finger instead of mouse: touch screens as a means of enhancing universal access. In: Carbonell, N., Stephanidis, C. (eds.) UI4ALL 2002. LNCS, vol. 2615, pp. 387–397. Springer, Heidelberg (2003). doi:10.​1007/​3-540-36572-9_​30 CrossRef
12.
Zurück zum Zitat Hong, J., Heo, S., Isokoski, P., Lee, G.: SplitBoard: a simple split soft keyboard for wristwatch-sized touch screens. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2015), pp. 1233–1236. ACM (2015) Hong, J., Heo, S., Isokoski, P., Lee, G.: SplitBoard: a simple split soft keyboard for wristwatch-sized touch screens. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2015), pp. 1233–1236. ACM (2015)
13.
Zurück zum Zitat Kurihara, K., Goto, M., Ogata, J., Igarashi, T.: Speech pen: predictive handwriting based on ambient multimodal recognition. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2006), pp. 851–860. ACM (2006) Kurihara, K., Goto, M., Ogata, J., Igarashi, T.: Speech pen: predictive handwriting based on ambient multimodal recognition. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2006), pp. 851–860. ACM (2006)
14.
Zurück zum Zitat Kwon, S., Choi, E., Chung, M.K.: Effect of control-to-display gain and movement direction of information spaces on the usability of navigation on small touch-screen interfaces using Tap-N-Drag. J. Ind. Ergon. 41(3), 322–330 (2011)CrossRef Kwon, S., Choi, E., Chung, M.K.: Effect of control-to-display gain and movement direction of information spaces on the usability of navigation on small touch-screen interfaces using Tap-N-Drag. J. Ind. Ergon. 41(3), 322–330 (2011)CrossRef
15.
Zurück zum Zitat Leiva, L.A., Sahami, A., Catalá, A., Henze, N., Schmidt, A.: text entry on tiny QWERTY soft keyboards. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2015), pp. 669–678. ACM (2015) Leiva, L.A., Sahami, A., Catalá, A., Henze, N., Schmidt, A.: text entry on tiny QWERTY soft keyboards. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2015), pp. 669–678. ACM (2015)
16.
Zurück zum Zitat Nebeling, M., Guo, A., Murray, K., Tostengard, A., Giannopoulos, A., Mihajlov, M., Bigham, J.P.: WearWrite: orchestrating the crowd to complete complex tasks from wearables. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology (UIST 2015), pp. 39–40. ACM (2015) Nebeling, M., Guo, A., Murray, K., Tostengard, A., Giannopoulos, A., Mihajlov, M., Bigham, J.P.: WearWrite: orchestrating the crowd to complete complex tasks from wearables. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology (UIST 2015), pp. 39–40. ACM (2015)
17.
Zurück zum Zitat Oney, S., Harrison, C., Ogan, A., Wiese, J.: ZoomBoard: a diminutive QWERTY soft keyboard using iterative zooming for ultra-small devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2013), pp. 2799–2802. ACM (2013) Oney, S., Harrison, C., Ogan, A., Wiese, J.: ZoomBoard: a diminutive QWERTY soft keyboard using iterative zooming for ultra-small devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2013), pp. 2799–2802. ACM (2013)
18.
Zurück zum Zitat Oviatt, S.: Mutual disambiguation of recognition errors in a multimodel architecture. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 1999). pp. 576–583. ACM (1999) Oviatt, S.: Mutual disambiguation of recognition errors in a multimodel architecture. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 1999). pp. 576–583. ACM (1999)
19.
Zurück zum Zitat Paliwal, K., Basu, A.A.: Speech enhancement method based on Kalman filtering. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Process (ICASSP), pp. 177–180 (1987) Paliwal, K., Basu, A.A.: Speech enhancement method based on Kalman filtering. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Process (ICASSP), pp. 177–180 (1987)
20.
Zurück zum Zitat Roudaut, A., Huot, S., Lecolinet, E.: TapTap and MagStick: improving one-handed target acquisition on small touch-screens. In: Proceedings of the Working Conference on Advanced Visual Interfaces, pp. 146–153. ACM (2008) Roudaut, A., Huot, S., Lecolinet, E.: TapTap and MagStick: improving one-handed target acquisition on small touch-screens. In: Proceedings of the Working Conference on Advanced Visual Interfaces, pp. 146–153. ACM (2008)
21.
Zurück zum Zitat Schüssel, F., Honold, F., Schmidt, M., Bubalo, N., Huckauf, A., Weber, M.: Multimodal interaction history and its use in error detection and recovery. In: Proceedings of the 16th International Conference on Multimodal Interaction, pp. 164–171. ACM (2014) Schüssel, F., Honold, F., Schmidt, M., Bubalo, N., Huckauf, A., Weber, M.: Multimodal interaction history and its use in error detection and recovery. In: Proceedings of the 16th International Conference on Multimodal Interaction, pp. 164–171. ACM (2014)
22.
Zurück zum Zitat Siek, K.A., Rogers, Y., Connelly, K.H.: Fat finger worries: how older and younger users physically interact with PDAs. In: Costabile, M.F., Paternò, F. (eds.) INTERACT 2005. LNCS, vol. 3585, pp. 267–280. Springer, Heidelberg (2005). doi:10.1007/11555261_24 CrossRef Siek, K.A., Rogers, Y., Connelly, K.H.: Fat finger worries: how older and younger users physically interact with PDAs. In: Costabile, M.F., Paternò, F. (eds.) INTERACT 2005. LNCS, vol. 3585, pp. 267–280. Springer, Heidelberg (2005). doi:10.​1007/​11555261_​24 CrossRef
23.
Zurück zum Zitat Tsourakis, N.: Using hand gestures to control mobile spoken dialogue systems. Univ. Access Inf. Soc. 13(3), 257–275 (2014)CrossRef Tsourakis, N.: Using hand gestures to control mobile spoken dialogue systems. Univ. Access Inf. Soc. 13(3), 257–275 (2014)CrossRef
24.
Zurück zum Zitat Turunen, M., Hurtig, T., Hakulinen, J., Virtanen, A., Koskinen, S.: Mobile speech-based and multimodal public transport information services. In: Proceedings of MobileHCI 2006 Workshop on Speech in Mobile and Pervasive Environments (2006) Turunen, M., Hurtig, T., Hakulinen, J., Virtanen, A., Koskinen, S.: Mobile speech-based and multimodal public transport information services. In: Proceedings of MobileHCI 2006 Workshop on Speech in Mobile and Pervasive Environments (2006)
27.
Zurück zum Zitat Lee, J., Kim, G.J.: Vouch: multimodal input for smart watches under difficult operating conditions using touch and voice. J. Multimodal Interfaces (2016, submitted) Lee, J., Kim, G.J.: Vouch: multimodal input for smart watches under difficult operating conditions using touch and voice. J. Multimodal Interfaces (2016, submitted)
28.
Zurück zum Zitat MacKenzie, I.S., Soukoreff, R.W.: Phrase sets for evaluating text entry techniques. Ext. Abstracts CHI 2003, pp. 754–755. ACM Press (2003) MacKenzie, I.S., Soukoreff, R.W.: Phrase sets for evaluating text entry techniques. Ext. Abstracts CHI 2003, pp. 754–755. ACM Press (2003)
Metadaten
Titel
Vouch-T: Multimodal Text Input for Mobile Devices Using Voice and Touch
verfasst von
Minyoung Lee
Gerard J. Kim
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-58077-7_17

Neuer Inhalt