Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
MTZ-Fachkongress

Antriebe und Energiesysteme von morgen


Austausch von Automobil- und Energiewirtschaft

Am 14./15. Mai geht es in Chemnitz um die Mobilitätswende durch Elektro- und Wasserstofffahrzeuge.
Erweiterte Suche
Anmelden
nach oben
Erschienen in:
Development and Validation of Usability Heuristics for Evaluation of Interfaces in ATMs Kapitel lesen Erstes Kapitel lesen

2019 | OriginalPaper | Buchkapitel

Which Instrument Should I Use? Supporting Decision-Making About the Evaluation of User Experience

verfasst von : Ticianne Darin, Bianca Coelho, Bosco Borges

Erschienen in: Design, User Experience, and Usability. Practice and Case Studies

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

User Experience (UX) has been intensively investigated lately, resulting in the proposal of several evaluation instruments, methods, and techniques. However, the definition of UX and its constructs is still a work in progress, making User Experience a concept open to various interpretations. Consequently, the development of UX evaluation methods and instruments rely on very different assumptions, often making professionals and beginning researchers uncertain about choosing the right methods to evaluate user evaluation aspects. Aiming to help fill in this gap, in this work we present the results of a systematic snowballing procedure conducted to investigate the characteristics of the UX evaluation instruments that have been proposed and used by HCI community in the last years. We compiled information about 116 instruments aiming to assist researchers and practitioners in making informed choices about which instruments can support UX data collection, according to their research goals. In addition to that, the data analysis provided a glance on the directions the research on UX evaluation instruments is taking.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Vorheriges Kapitel The Potential of User Experience (UX) as an Approach of Evaluation in Tangible User Interfaces (TUI)
Nächstes Kapitel Game for PLAYability Heuristic Evaluation (G4H-PLAY): Adapting G4H Gamification to Allow the Use of Playability Heuristics
Fußnoten
1
Although the original research paper reports 96 methods, the online list at allaboutux.com, mentioned by the authors in the same paper, currently only presents 86 evaluation artifacts (among methods, frameworks, models and instruments).
 
Literatur
1.
Obrist, M., Law, E., Väänänen-Vainio-Mattila, K., Roto, V., Vermeeren, A., Kuutti, K.: UX research: what theoretical roots do we build on–if any? In: CHI’11 Extended Abstracts on Human Factors in Computing Systems, pp. 165–168. ACM (2011)
2.
Law, E.L.C., Roto, V., Hassenzahl, M., Vermeeren, A.P., Kort, J.: Understanding, scoping and defining user experience: a survey approach. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 719–728. ACM (2009)
3.
Ardito, C., Buono, P., Caivano, D., Costabile, M.F., Lanzilotti, R.: Investigating and promoting UX practice in industry: an experimental study. Int. J. Hum.-Comput. Stud. 72, 542–551 (2014)CrossRef
4.
Pucillo, F., Cascini, G.: A framework for user experience, needs and affordances. Des. Stud. 35(2), 160–179 (2014)CrossRef
5.
6.
Obrist, M., Roto, V., Väänänen-Vainio-Mattila, K.: User experience evaluation: do you know which method to use? In: CHI 2009 Extended Abstracts on Human Factors in Computing Systems, pp. 2763–2766. ACM (2009)
7.
Lallemand, C., Koenig, V.: How could an intranet be like a friend to me?: why standardized UX scales don’t always fit. In: Proceedings of the European Conference on Cognitive Ergonomics 2017, pp. 9–16. ACM (2017)
8.
Rajeshkumar, S., Omar, R., Mahmud, M.: Taxonomies of user experience (UX) evaluation methods. In: 2013 International Conference on Research and Innovation in Information Systems (ICRIIS), pp. 533–538. IEEE (2013)
9.
Bevan, N.: Classifying and selecting UX and usability measures. In: International Workshop on Meaningful Measures: Valid Useful User Experience Measurement, vol. 11, pp. 13–18 (2010)
10.
Obrist, M., Roto, V., Vermeeren, A., Väänänen-Vainio-Mattila, K., Law, E.L.C., Kuutti, K.: In search of theoretical foundations for UX research and practice. In: CHI 2012 Extended Abstracts on Human Factors in Computing Systems, pp. 1979–1984. ACM (2012)
11.
Hellweger, S., Wang, X.: What is user experience really: towards a UX conceptual framework (2015). arXiv preprint arXiv:​1503.​01850
12.
Rajanen, D., et al.: UX professionals’ definitions of usability and UX – a comparison between Turkey, Finland, Denmark, France and Malaysia. In: Bernhaupt, R., Dalvi, G., Joshi, A., K. Balkrishan, D., O’Neill, J., Winckler, M. (eds.) INTERACT 2017. LNCS, vol. 10516, pp. 218–239. Springer, Cham (2017). https://​doi.​org/​10.​1007/​978-3-319-68059-0_​14CrossRef
13.
14.
Nielsen, J.: Usability Engineering. Elsevier, San Diego (1994)MATH
15.
Bargas-Avila, J.A., Hornbæk, K.: Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience. In: Proceedings of the SIGCHI Conference on Human Factors in Computing System, pp. 2689–2698. ACM (2011)
16.
Schubert, E.: Measuring emotion continuously: validity and reliability of the two-dimensional emotion-space. Aust. J. Psychol. 51(3), 154–165 (1999)CrossRef
17.
18.
Law, E.L.C., Van Schaik, P., Roto, V.: Attitudes towards user experience (UX) measurement. Int. J. Hum. Comput. Stud. 72(6), 526–541 (2014)CrossRef
19.
Vissers, J., De Bot, L., Zaman, B.: MemoLine: evaluating long-term UX with children. In: Proceedings of the 12th International Conference on Interaction Design and Children, pp. 285–288. ACM (2013)
20.
Väänänen-Vainio-Mattila, K., Roto, V., Hassenzahl, M.: Towards practical user experience evaluation methods. In: Meaningful Measures: Valid Useful User Experience Measurement (VUUM), pp. 19–22 (2008)
21.
Law, E.L.C.: The measurability and predictability of user experience. In: Proceedings of the 3rd ACM SIGCHI Symposium on Engineering Interactive Computing Systems, pp. 1–10. ACM (2011)
22.
Wohlin, C.: Guidelines for snowballing in systematic literature studies and a replication in software engineering. In: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, p. 38. ACM (2014)
23.
Bevan, N., Macleod, M.: Usability measurement in context. Behav. Inf. Technol. 13(1–2), 132–145 (1994)CrossRef
24.
Agarwal, A., Andrew M.: Beyond usability: evaluating emotional response as an integral part of the user experience. In: CHI 2009 Extended Abstracts on Human Factors in Computing Systems. ACM (2009)
25.
Roto, V., Marianna O., Väänänen-Vainio-Mattila, K.: User experience evaluation methods in academic and industrial contexts. In: Proceedings of the Workshop UXEM, vol. 9 (2009)
26.
Hassenzahl, M.: The interplay of beauty, goodness, and usability in interactive products. Hum. Comput. Interact. 19(4), 319–349 (2004)CrossRef
27.
Ganglbauer, E., et al.: Applying psychophysiological methods for measuring user experience: possibilities, challenges and feasibility. In: Workshop on User Experience Evaluation Methods in Product Development (2009)
28.
Ng, Y., Khong, C.W., Nathan, R.J.: Evaluating affective user-centered design of video games using qualitative methods. Int. J. Comput. Games Technol. 2018 (2018)
29.
Hung, Y.-H., Parsons, P.: Affective engagement for communicative visualization: quick and easy evaluation using survey instruments. In: Visualization for Communication (VisComm) 2018 (2018, to appear)
30.
Vermeeren, A.P., et al.: User experience evaluation methods: current state and development needs. In: Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, pp. 521–530. ACM (2010)
31.
Poels, K., De Kort, Y. A.W., IJsselsteijn, W.A.: D3. 3: Game experience questionnaire: development of a self-report measure to assess the psychological impact of digital games (2007)
32.
Karapanos, E., Martens, J.B.O.S., Hassenzahl, M.: On the retrospective assessment of users’ experiences over time: memory or actuality? In: CHI 2010 Physiological User Interaction Workshop, Atlanta, GA, 10–15 April 2010, pp. 4075–4080. Association for Computing Machinery, Inc. (2010)
33.
Ryan, R.M.: Control and information in the intrapersonal sphere: an extension of cognitive evaluation theory. J. Pers. Soc. Psychol. 43(3), 450 (1982)CrossRef
34.
Almeida, R., Darin, T., Andrade, R., de Araújo, I.: Towards developing a practical tool to assist UX evaluation in the IoT scenario. In: Anais Estendidos do XXIV Simpósio Brasileiro de Sistemas Multimídia e Web, pp. 91–95. SBC (2018)
35.
Granato, M., Gadia, D., Maggiorini, D., Ripamonti, L.A.: Software and hardware setup for emotion recognition during video game fruition. In: Proceedings of the 4th EAI International Conference on Smart Objects and Technologies for Social Good, pp. 19–24. ACM (2018)
36.
Kahneman, D., Krueger, A.B., Schkade, D.A., Schwarz, N., Stone, A.A.: A survey method for characterizing daily life experience: the day reconstruction method. Science 306(5702), 1776–1780 (2004)CrossRef
37.
Gomez, R.E.: The evolving emotional experience with portable interactive devices. Doctoral dissertation, Queensland University of Technology (2012)
38.
Russell, J.A., Weiss, A., Mendelsohn, G.A.: Affect grid: a single-item scale of pleasure and arousal. J. Pers. Soc. Psychol. 57(3), 493–502 (1989)CrossRef
39.
Laviea, T., Tractinsky, N.: Assessing dimensions of perceived visual aesthetics of web sites. Int. J. Hum Comput Stud. 60(3), 269–298 (2004)CrossRef
40.
Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)CrossRef
41.
Scherer, K.R.: Appraisal considered as a process of multilevel sequential checking. Apprais. Process. Emotion Theory Methods Res. 92(120), 57 (2001)
42.
Schaefer, K.: The perception and measurement of human-robot trust (2013)
43.
Van Damme, K., et al.: 360° Video journalism: experimental study on the effect of immersion on news experience and distant suffering. J. Stud. 1–24 (2018)
44.
Meijman, T., et al.: The measurement of perceived effort. In: Contemporary Ergonomics, pp. 242–246 (1986)
45.
Väänänen-Vainio-Mattila, K., Segerstähl, K.: A tool for evaluating service user experience (ServUX): development of a modular questionnaire. In: User Experience Evaluation Methods, UXEM 2009 Workshop at Interact (2009)
46.
Lazar, J., Feng, J.H., Hochheiser, H.: Research Methods in Human-Computer Interaction. Morgan Kaufmann, Burlington (2017)
47.
Dillman, D.: Mail and Internet Surveys: The Tailored Design Method. Wiley, New York (2000)
48.
Väätäjä, H., Koponen, T., Roto, V.: Developing practical tools for user experience evaluation: a case from mobile news journalism. In: European Conference on Cognitive Ergonomics: Designing Beyond the Product—Understanding Activity and User Experience in Ubiquitous Environments, p. 23 (2009)
49.
Sauro, J.: SUPR-Q: a comprehensive measure of the quality of the website user experience. J. Usability Stud. 10(2), 68–86 (2015)
50.
Sales, A., Reis, L., Araújo, T., Aguiar, Y.: Tutaform: a multimedia form for Brazilian deaf users. In: Proceedings of the 24th Brazilian Symposium on Multimedia and the Web, pp. 269–276. ACM (2018)
51.
Kirakowski, J.: The software usability measurement inventory: background and usage. In: Usability Evaluation in Industry, pp. 169–178 (1996)
52.
Lessiter, J., Freeman, J., Keogh, E., Davidoff, J.: A cross-media presence questionnaire: the ITC-Sense of Presence Inventory. Presence Teleoper. Virtual Environ. 10(3), 282–297 (2001)CrossRef
53.
Bernhaupt, R., Pirker, M.: Methodological challenges of UX evaluation in the living room: developing the IPTV-UX questionnaire. In: PUX 2011 Program Committee, p. 51 (2011)
54.
55.
Saadé, R., Bahli, B.: The impact of cognitive absorption on perceived usefulness and perceived ease of use in online learning: an extension of the technology acceptance model. Inf. Manag. 42(2), 317–327 (2005)CrossRef
56.
Kennedy, R.S., Lane, N.E., Berbaum, K.S., Lilienthal, M.G.: Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int. J. Aviat. Psychol. 3(3), 203–220 (1993)CrossRef
57.
Witmer, B.G., Singer, M.J.: Measuring presence in virtual environments: a presence questionnaire. Presence 7(3), 225–240 (1998)CrossRef
58.
Lewis, J.R., Utesch, B. S., Maher, D.E.: UMUX-LITE: when there’s no time for the SUS. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2099–2102. ACM (2013)
59.
Sim, G., Horton, M.: Investigating children’s opinions of games: fun toolkit vs. this or that. In: Proceedings of the 11th International Conference on Interaction Design and Children, pp. 70–77. ACM (2012)
60.
61.
Zaman, B.: Introduction and validation of a pairwise comparison scale for UX evaluations and benchmarking with preschoolers. In: INTERACT 2009. Springer, Uppsala (2009)
62.
Horn, D., Salvendy, G.: Product creativity: conceptual model, measurement and characteristics. Theor. Issues Ergon. Sci. 7(4), 395–412 (2006)CrossRef
63.
Lasa, G., Justel, D., Gonzalez, I., Iriarte, I., Val, E.: Next generation of tools for industry to evaluate the user emotional perception: the biometric-based multimethod tools. Des. J. 20(sup1), S2771–S2777 (2017)
64.
Stahl, A., Hook, K., Svensson, M., Taylor, A.S., Combetto, M.: Experiencing the affective diary. Pers. Ubiquit. Comput. 13(5), 365–378 (2009)CrossRef
65.
Robinson, R.B.: All the feels: a twitch overlay that displays streamers’ biometrics to spectators. Dissertation, UC Santa Cruz (2018)
66.
Hussain, J., et al.: A multimodal deep log-based user experience (UX) platform for UX evaluation. Sensors 18(5), 1622 (2018)CrossRef
67.
Kujala, S., et al.: UX curve: a method for evaluating long-term user experience. Interact. Comput. 23(5), 473–483 (2011)CrossRef
68.
Broekens, J., Brinkman, W.P.: AffectButton: a method for reliable and valid affective self-report. Int. J. Hum.-Comput. Stud. 71(6), 641–667 (2013)CrossRef
69.
Nagel, F., et al.: EMuJoy: software for continuous measurement of perceived emotions in music. Behav. Res. Methods 39(2), 283–290 (2007)CrossRef
70.
71.
Betella, A., Verschure, P.F.M.J.: The affective slider: a digital self-assessment scale for the measurement of human emotions. PLoS ONE 11, 2 (2016)CrossRef
72.
Kim, J.H., et al.: Tracking real-time user experience (TRUE): a comprehensive instrumentation solution for complex systems. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 443–452. ACM (2008)
73.
Moura, D., El-Nasr, M.S., Shaw, C.D.: Visualizing and understanding players’ behavior in video games: discovering patterns and supporting aggregation and comparison. In: Proceedings of the 2011 ACM SIGGRAPH Symposium on Video Games, pp. 11–15. ACM (2011)
74.
Schütte, S., Almagro, L.M.: Development and application of online tools for kansei engineering evaluations. In: KEER2018, Go Green with Emotion. 7th International Conference on Kansei Engineering & Emotion Research 2018, Kuching, Malaysia, 19–22 Mar 2018, no. 146, pp. 20–28. Linköping University Electronic Press (2018)
75.
Ayzenberg, Y., Hernandez Rivera, J., Picard, R.: FEEL. In: Proceedings of the 2012 ACM Annual Conference Extended Abstracts on Human Factors in Computing Systems Extended Abstracts - CHI EA 2012 (2012)
76.
Mendes, M.S., Furtado, E.S.: UUX-posts. In: Proceedings of the 8th Latin American Conference on Human-Computer Interaction – CLIHC 2017 (2017)
77.
Drachen, A., Canossa, A.: Analyzing spatial user behavior in computer games using geographic information systems. In: Proceedings of the 13th International MindTrek Conference: Everyday Life in the Ubiquitous Era on – MindTrek 2009 (2009)
78.
Cavalcante, E., Rivero, L., Conte, T.: MAX: a method for evaluating the post-use user eXperience through cards and a board (2015)
79.
Hassenzahl, M., Burmester, M., Koller, F.: AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In: Mensch & Computer 2003, pp. 187–196. Vieweg+Teubner Verlag (2003)
80.
Moreira, E.A., dos Reis, J.C., Baranauskas, M.C.C.: TangiSAM. In: Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems - IHC (2017)
81.
Law, E., Roto, V., Vermeeren, A.P., Kort, J., Hassenzahl, M.: Towards a shared definition of user experience. In: CHI 2008 Extended Abstracts on Human Factors in Computing Systems, pp. 2395–2398. ACM (2008)
82.
Lasa, G., Justel, D., Retegi, A.: Eyeface: a new multimethod tool to evaluate the perception of conceptual user experiences. Comput. Hum. Behav. 52, 359–363 (2015)CrossRef
83.
Veeneklaas, J.N.: VisUX: a framework of user experience within data visualizations. MS thesis (2018)
84.
Desmet, P.: Measuring emotion: development and application of an instrument to measure emotional responses to products. In: Blythe, M.A., Overbeeke, K., Monk, A.F., Wright, P.C. (eds.) Funology, pp. 111–123. Springer, Dordrecht (2003)CrossRef
85.
McDuff, D., Karlson, A., Kapoor, A., Roseway, A., Czerwinski, M.: AffectAura: an intelligent system for emotional memory. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 849–858. ACM (2012)
86.
Isbister, K., Höök, K., Laaksolahti, J., Sharp, M.: The sensual evaluation instrument: developing a trans-cultural self-report measure of affect. Int. J. Hum.-Comput. Stud. 65(4), 315–328 (2007)CrossRef
87.
Mehrabian, A.: Framework for a comprehensive description and measurement of emotional states. Genet. Soc. Gen. Psychol. Monogr. (1995)
88.
Watson, D., Clark, L.A., Tellegen, A.: Development and validation of brief measures of positive and negative affect: the PANAS scales. J. Pers. Soc. Psychol. 54(6), 1063 (1998)CrossRef
89.
Law, E.L.C., Van Schaik, P.: Modelling user experience—an agenda for research and practice. Interact. Comput. 22(5), 313–322 (2010)CrossRef
90.
Andersen, E., Liu, Y.E., Apter, E., Boucher-Genesse, F., Popović, Z.: Gameplay analysis through state projection. In: Proceedings of the Fifth International Conference on the Foundations of Digital Games, pp. 1–8. ACM (2010)
Metadaten
Titel
Which Instrument Should I Use? Supporting Decision-Making About the Evaluation of User Experience
verfasst von
Ticianne Darin
Bianca Coelho
Bosco Borges
Copyright-Jahr
2019
Verlag
Springer International Publishing
Buch
Design, User Experience, and Usability. Practice and Case Studies

Print ISBN: 978-3-030-23534-5

Electronic ISBN: 978-3-030-23535-2

Copyright-Jahr: 2019

DOI
https://doi.org/10.1007/978-3-030-23535-2_4

Neuer Inhalt

    Bildnachweise