Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben
Journal on Multimodal User Interfaces
Erschienen in: Journal on Multimodal User Interfaces 1-2/2012

01.07.2012 | Original Paper

Measuring the Quality of Service and Quality of Experience of multimodal human–machine interaction

verfasst von: Ina Wechsung, Klaus-Peter Engelbrecht, Christine Kühnel, Sebastian Möller, Benjamin Weiss

Erschienen in: Journal on Multimodal User Interfaces | Ausgabe 1-2/2012

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Quality of Service (QoS) and Quality of Experience (QoE) have to be considered when designing, building and maintaining services involving multimodal human–machine interaction. In order to guide the assessment and evaluation of such services, we first develop a taxonomy of the most relevant QoS and QoE aspects which result from multimodal human–machine interactions. It consists of three layers: (1) The quality factors influencing QoS and QoE related to the user, the system, and the context of use; (2) the QoS interaction performance aspects describing user and system behavior and performance; and (3) the QoE aspects related to the quality perception and judgment processes taking place within the user. For each of these layers, we then provide metrics which are able to capture the QoS and QoE aspects in a quantitative way, either via questionnaires or performance measures. The metrics are meant to guide system evaluation and make it more systematic and comparable.
Vorheriger Artikel Studying and enhancing talking condition recognition in stressful and emotional talking environments based on HMMs, CHMM2s and SPHMMs
Nächster Artikel Sound/tracks: artistic real-time sonification of train journeys

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Literatur
1.
Lamel L, Bennacef S, Gauvain JL, Dartigues H, Temem JN (2002) User evaluation of the MASK kiosk. Speech Commun 38:131–139 MATHCrossRef
2.
Bernsen NO, Dybkjær L, Kiilerich S (2004) Evaluating conversation with Hans Christian Andersen. In: Proc LREC, vol 3, Lisbon, pp 1011–1014
3.
Wahlster W (2006) SmartKom: foundations of multimodal dialogue systems. Springer, Berlin CrossRef
4.
Turunen M, Hakulinen J, Ståhl O, Gambäck B, Hansen P, Rodriguez Gancedo M, Santos de la Camara R, Smith C, Charlton D, Cavazza M (2011) Multimodal and mobile conversational health and fitness companions. Comput Speech Lang 25:192–209 CrossRef
5.
Cassell J, Kopp S, Tepper P, Ferriman K, Striegnitz K (2007) Trading spaces: how humans and humanoids use speech and gesture to give directions. In: Nishida T (ed) Conversational informatics. Wiley, New York, pp 133–160
6.
Gibbon D, Mertins I, Moore RK (eds) (2000) Handbook of multimodal and spoken dialogue systems. Kluwer Academic, Boston
7.
Coutaz J, Nigay L, Salber D, Blandford AE, May J, Young RM (1995) Four easy pieces for assessing the usability of multimodal interaction: the CARE properties. In: Nordby K, Helmersen PH, Gilmore DJ, Arnesen SA (eds) Human–computer interaction, proc interact 1995. Chapman & Hall, London, pp 115–120
8.
Wickens CD (2008) Multiple resources and mental workload. Hum Factors 50:449–455 CrossRef
9.
Bernsen NO, Dybkjær L (2010) Multimodal usability. Human–computer interaction series. Springer, Berlin CrossRef
10.
ISO 25000 (2005) Software engineering—software product quality requirements and evaluation (SQuaRE)—guide to SQuaRE. International Organization for Standardization, Geneva
11.
Gray WD, Salzman MC (1998) Damaged merchandise? A review of experiments that compare usability evaluation methods. Hum-Comput Interact 13(3):203–261 CrossRef
12.
Olson GM, Moran TP (1998) Commentary on Damaged Merchandise? Hum-Comput Interact 13(3):263–323 CrossRef
13.
Hornbæk K (2006) Current practice in measuring usability: challenges to usability studies and research. Int J Man-Mach Stud 64(2):79–102
14.
Hartson HR, Andre TS, Williges RC (2003) Criteria for evaluating usability evaluation methods. Int J Hum-Comput Interact 15(1):145–181 CrossRef
15.
ITU-T Rec. E.800 (1994) Terms and definitions related to Quality of Service and network performance including dependability. International Telecommunication Union, Geneva
16.
ETSI TR 102 643 (2010) Human factors (HF); Quality of Experience (QoE) requirements for real-time communication services. European Telecommunications Standards Institute, Sophia Antipolis
17.
Brooks P, Hestnes B (2010) User measures of quality of experience: why being objective and quantitative is important. IEEE Netw 24:8–13 CrossRef
18.
ITU-T Rec. P.10 (2007) Vocabulary for performance and Quality of Service. International Telecommunication Union, Geneva
19.
Jekosch U (2005) Voice and speech quality perception. Assessment and evaluation. Springer, Berlin
20.
Fraser N (1997) Assessment of interactive systems. In: Gibbon D, Moore R, Winski R (eds) Handbook on standards and resources for spoken language systems. Mouton de Gruyter, Berlin, pp 564–615
21.
ITU-T Suppl. 24 to P-Series Rec. (2005) Parameters describing the interaction with spoken dialogue systems. International Telecommunication Union, Geneva
22.
ITU-T Rec. P.851 (2003) Subjective quality evaluation of telephone services based on spoken dialogue systems. International Telecommunication Union, Geneva
23.
Hone KS, Graham R (2000) Towards a tool for the Subjective Assessment of Speech System Interfaces (SASSI). Nat Lang Eng 6(3/4):287–303 CrossRef
24.
Nielsen J (1993) Usability engineering. Academic Press, Boston MATH
25.
Alonso-Ríos D, Vázquez-García A, Mosqueira-Rey E, Moret-Bonillo V (2010) Usability: a critical analysis and a taxonomy. Int J Hum-Comput Interact 26(1):53–74 CrossRef
26.
Möller S, Kühnel C, Weiss B (2010) Extending Suppl 24 to P-series towards multimodal systems and services. International Telecommunication Union, Geneva. Source: Deutsche Telekom Laboratories, ITU-T SIG12 Meeting 18–27 May 2010
27.
Möller S (2005) Quality of telephone-based spoken dialogue systems. Springer, New York
28.
Pallett D, Fiscus J, Fisher W, Garofolo J (1993) Benchmark tests for the DARPA spoken language program. In: Proc DARPA human language technology workshop, Princeton, pp 7–18 CrossRef
29.
Bennett CL, Black AW (2006) The blizzard challenge 2006. In: Proc blizzard satellite workshop to interspeech, Pittsburgh
30.
Möller S (2005) Quality of telephone-based spoken dialogue systems. Springer, New York
31.
Hermann F, Niedermann I, Peissner M, Henke K, Naumann A (2007) Users interact differently: towards a usability-oriented taxonomy. In: Jacko J (ed) Interaction design and usability, proc HCII 2007, vol 1. Springer, Heidelberg, pp 812–817 CrossRef
32.
33.
Heinssen RK, Glass CR, Knight LA (1987) Assessing computer anxiety: development and validation of the computer anxiety rating scale. Comput Hum Behav 3(1):49–59 CrossRef
34.
Van Vliet PJA, Kletke MG, Chakraborty G (1994) The measurement of computer literacy—a comparison of self-appraisal and objective tests. Int J Hum-Comput Stud 40(5):835–857 CrossRef
35.
Richter T, Naumann J, Groeben N (2000) Attitudes toward the computer: construct validation of an instrument with scales differentiated by content. Comput Hum Behav 16:473–491 CrossRef
36.
Compeau DR, Higgins CA (1995) Computer self efficacy: development of a measure and initial test. Manag Inf Syst Q June:189–211 CrossRef
37.
Smith B, Caputi P, Rawstorne P (2007) The development of a measure of subjective computer experience. Comput Hum Behav 23(1):127–145 CrossRef
38.
Kalbe E, Kessler J, Calabrese P, Smith R, Passmore AP, Brand M, Bullock R (2004) DemTect: a new, sensitive cognitive screening test to support the diagnosis of mild cognitive impairment and early dementia. Int J Geriatr Psychiatry 19:136–143 CrossRef
39.
Lopez-Cozar Delgado R, Araki M (2005) Spoken, multilingual and multimodal dialogue systems—development and assessment. Wiley, New York CrossRef
40.
Kühnel C, Weiss B, Möller S (2010) Parameters describing multimodal interaction—definitions and three usage scenarios. In: Proc INTERSPEECH, pp 2014–2017
41.
Mostefa D, Garcia M-N, Choukri K (2006) Evaluation of multimodal components within CHIL: the evaluation packages and results. In: Proc LREC, Genoa, pp 915–918
42.
Bernsen NO (2002) Multimodality in language and speech systems—from theory to design support tool. In: Granström B, House D, Karlsson I (eds) Multimodality in language and speech systems. Kluwer Academic, Dordrecht, pp 93–148
43.
Grice H (1975) Logic and conversation. In: Cole P, Morgan JL (eds) Syntax and semantics. Speech acts, vol 3. Academic Press, New York, pp 41–58
44.
Zimbardo PG (1995) Psychologie. Springer, Berlin
45.
Borg G (1982) Psychophysical bases of perceived exertion. Med Sci Sports Exerc 14:377–381
46.
De Waard D (1996) The measurement of drivers’ mental workload. PhD thesis, University of Groningen, Haren
47.
Hart SG, Staveland LE (1988) Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Hancock P, Meshkati N (eds) Human mental workload. North Holland, Amsterdam, pp 139–183 CrossRef
48.
Zijlstra FRH (1993) Efficiency in work behavior. A design approach for modern tools. PhD thesis, Delft University of Technology, Delft University Press, Delft
49.
Cacioppo JT, Tassinary LG, Berntson GG (eds) (2002) Handbook of psychophysiology, 3rd edn. Cambridge University Press, New York
50.
Hassenzahl M, Platz A, Burmester M, Lehner K (2000) Hedonic and ergonomic quality aspects determine a software’s appeal. In: Proc CHI 2000, Den Haag, pp 201–208
51.
Mandryk RL, Inkpen K, Calvert TW (2006) Using psycho-physiological techniques to measure user experience with entertainment technologies. Behav Inf Technol 25(2):141–158 CrossRef
52.
Chateau N, Gros L, Durin V, Macé A (2006) Redrawing the link between customer satisfaction and speech quality. In: Proc 2nd ISCA/DEGA tutorial and research workshop on perceptual quality of systems, Berlin, pp 88–94
53.
Hornbæk K, Law EL (2007) Meta-analysis of correlations among usability measures. In: Proc CHI 2007. ACM Press, New York, pp 617–626
54.
ISO 9241-11 (1999) Ergonomic requirements for office work with visual display terminals (VDTs). Part 11: guidance on usability. International Organization for Standardization, Geneva
55.
ISO DIS 9241-210 (2010) Ergonomics of human system interaction—part 210: human-centred design for interactive systems (formerly known as 13407). International Organization for Standardization (ISO), Geneva
56.
Bevan N (2009) What is the difference between the purpose of usability and user experience evaluation methods. In: Proc UXEM’09 (INTERACT’09), Uppsala
57.
Kirakowski J, Corbett M (1993) SUMI: the software usability measurement inventory. Br J Educ Technol 24(3):210–212 CrossRef
58.
Lewis JR (1995) IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum-Comput Interact 7:57–78 CrossRef
59.
Hassenzahl M (2004) The interplay of beauty, goodness, and usability in interactive products. Hum-Comput Interact 19:319–349 CrossRef
60.
Kelly GA (1955) The psychology of personal constructs. Norton, New York
61.
Burmester M, Mast M, Jäger K, Homans H (2010) Valence method for formative evaluation of user experience. In: Proc DIS ’10. ACM Press, New York, pp 364–367
62.
63.
Hassenzahl M, Wessler R (2000) Capturing design space from a user perspective: the repertory grid technique revisited. Int J Hum-Comput Interact 12(3/4):441–459
64.
Hassenzahl M, Trautmann T (2001) Analysis of web sites with the repertory grid technique. In: Proc CHI 2001. ACM Press, New York, pp 167–168
65.
Hassenzahl M, Diefenbach S, Göritz AS (2010) Needs, affect, and interactive products—facets of user experience. Interact Comput 22(5):353–362 CrossRef
66.
Karapanos E, Martens J-B, Hassenzahl M (2010) On the retrospective assessment of users’ experiences over time: memory or actuality? In: CHI 2010 extended abstracts on human factors in computing systems, Atlanta. ACM Press, New York, pp 4075–4080
67.
Davis F (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. Manag Inf Syst Q 13(3):319–340 CrossRef
68.
Chin J, Diehl V, Norman K (1988) Development of an instrument measuring user satisfaction of the human–computer interface. In: Proceedings of SIGCHI 1988, pp 213–218
69.
Brooke J (1996) SUS: A “quick and dirty” usability scale. In: Jordan P, Thomas B, Weerdmeester B, McClelland I (eds) Usability evaluation in industry. Taylor & Francis, London, pp 189–194
70.
Gediga G, Hamborg K-C, Düntsch I (1999) The IsoMetrics usability inventory: an operationalisation of ISO 9241-10. Behav Inf Technol 18:151–164 CrossRef
71.
Lund AM (2001) Measuring usability with the USE questionnaire. In: Usability and user experience 8, STC usability SIG newsletter.
72.
Ullrich D, Diefenbach S (2010) INTUI. Exploring the facets of intuitive interaction. In: Mensch & computer 2010 interaktive kulturen.
73.
Naumann A, Hurtienne J, Israel JH, Mohs C, Kindsmüller MC, Meyer HA, Husslein S (2007) Intuitive use of user interfaces: defining a vague concept. In: Harris D (ed) Engineering psychology and cognitive ergonomics, vol 13, Proc HCII 2007. LNAI, vol 4562. Springer, Heidelberg, pp 128–136 CrossRef
74.
Polson PG, Lewis C, Rieman J, Wharton C (1992) Cognitive walkthroughs: a method for theory-based evaluation of user interfaces. Int J Man-Mach Stud 36:741–773 CrossRef
75.
John BE, Kieras DE (1996) Using GOMS for user interface design and evaluation: which technique? ACM Trans Comput-Hum Interact 3(4):287–319 CrossRef
76.
Kieras DE, Meyer DE (1997) An overview of the EPIC architecture for cognition and performance with application to human–computer interaction. Hum-Comput Interact 12(4):391–438 CrossRef
77.
Schleicher R, Trösterer S (2009) The ‘joy-of-use’-button: recording pleasant moments while using a PC. In: Human–computer interaction—INTERACT 2009, vol 5727/2009. Springer, Heidelberg
78.
Hekkert P (2006) Design aesthetics: principles of pleasure in product design. Psychol Sci 48:157–172
79.
Isbister K, Nass C (2000) Consistency of personality in interactive characters: verbal cues, non-verbal cues, and user characteristics. Int J Hum-Comput Stud 53(2):251–267 CrossRef
80.
Marcus A (1995) Principles of effective visual communication for graphical user interface design. In: Baecker R, Grudin J, Buxton W, Greenberg S (eds) Human-computer interaction: toward the year 2000. Morgan Kaufmann, San Francisco, pp 425–441
81.
Mairesse F, Walker M, Mehl M, Moore R (2007) Using linguistic cues for the automatic recognition of personality in conversation and text. J Artif Intell Res 30. doi:10.​1613/​jair.​2349
82.
Stelmaszweska H, Fields B, Blandford A (2004) Conceptualising user hedonic experience. In: Reed DJ, Baxter G, Blythe M (eds) Proc ECCE-12, living and working with technology. EACE, York, pp 83–89
83.
Tractinsky N, Katz AS, Ikar D (2000) What is beautiful is usable. Interact Comput 13(2):127–145 CrossRef
84.
Mahlke S, Lindgaard G (2007) Emotional experiences and quality perceptions of interactive products. In: Jacko JA (ed) Proceedings of the 12th international conference on human–computer interaction: interaction design and usabilit. Lecture notes in computer science. Springer, Berlin/Heidelberg, pp 164–173
85.
Hassenzahl M (2008) Aesthetics in interactive products: correlates and consequences of beauty. In: Schifferstein HNJ, Hekkert P (eds) Product experience. Elsevier, San Diego, pp 287–302 CrossRef
86.
Lavie T, Tractinsky N (2004) Assessing dimensions of perceived visual aesthetics of web sites. Int J Hum-Comput Stud 60:269–298 CrossRef
87.
Bradley MM, Lang PJ (1994) Measuring emotion: the self-assessment Manikin and the semantic differentia. J Behav Ther Exp Psychiatry 25:49–59 CrossRef
88.
Huisman G, Van Hout M (2008) The development of a graphical emotion measurement instrument using caricatured expressions: the LEMtool. In: Peter C, Crane E, Fabri M, Agius H, Axelrod L (eds) Emotion in HCI—designing for people: proceedings of the 2008 international workshop. Fraunhofer, Stuttgart, pp 5–7
89.
Desmet PMA (2004) Measuring emotions: development and application of an instrument to measure emotional responses to products. In: Blythe MA, Overbeeke K, Monk AF, Wright PC (eds) Funology: from usability to enjoyment. Kluwer Academic, Norwell, pp 111–123
90.
Bargas-Avila JA, Hornbæk K (2011) Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience. In: Proc CHI 2011, pp 2689–2698
91.
Bevan N (1995) Usability is quality of use. In: Proc HCII 1995. Elsevier, Amsterdam, pp 349–354
92.
Eurescom Project P.807 Deliverable 1 (1998) Jupiter II—usability, performability and interoperability trials in Europe. European Institute for Research and Strategic Studies in Telecommunications, Heidelberg
93.
Davis FD (1993) User acceptance of information technology system characteristics, user perceptions and behavioral impacts. Int J Man-Mach Stud 38(3):475–487 CrossRef
94.
Ajzen I, Fishbein M (1980) Understanding attitudes and predicting social behavior. Prentice Hall, Englewood Cliffs/New York
95.
Möller S, Engelbrecht K-P, Kühnel C, Wechsung I, Weiss B (2009) Evaluation of multimodal interfaces for ambient intelligence. In: Aghajan H, López-Cózar Delgado R, Augusto JC (eds) Human-centric interfaces for ambient intelligence. Elsevier, Amsterdam, pp 347–370
Metadaten
Titel
Measuring the Quality of Service and Quality of Experience of multimodal human–machine interaction
verfasst von
Ina Wechsung
Klaus-Peter Engelbrecht
Christine Kühnel
Sebastian Möller
Benjamin Weiss
Publikationsdatum
01.07.2012
Verlag
Springer-Verlag
Erschienen in
Journal on Multimodal User Interfaces / Ausgabe 1-2/2012
Print ISSN: 1783-7677
Elektronische ISSN: 1783-8738
DOI
https://doi.org/10.1007/s12193-011-0088-y

Weitere Artikel der Ausgabe 1-2/2012

Journal on Multimodal User Interfaces 1-2/2012 Zur Ausgabe

Original Paper

Generating context-sensitive ECA responses to user barge-in interruptions

Original Paper

Studying and enhancing talking condition recognition in stressful and emotional talking environments based on HMMs, CHMM2s and SPHMMs

Original Paper

Visual SceneMaker—a tool for authoring interactive virtual characters

EditorialNotes

Preface

Original Paper

Interactive sonification of expressive hand gestures on a handheld device

Original Paper

Sound/tracks: artistic real-time sonification of train journeys