Skip to main content

2016 | OriginalPaper | Buchkapitel

A Systematic Review About User Experience Evaluation

verfasst von : Camila Loiola Brito Maia, Elizabeth Sucupira Furtado

Erschienen in: Design, User Experience, and Usability: Design Thinking and Methods

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The user experience related to a product can determine its success or failure. As a result, companies have invested in research to understand what the user feels when he or she uses a product, but it is not simple to get this information, and many approaches arose through the years. This article presents a systematic review on the user experience evaluation field, based on 25 studies. We found (1) that psychophysiological measures are not yet widely applied in the evaluation of the user experience; (2) researchers prefer qualitative approaches; (3) the evaluations are mostly manual (not real time, therefore), and using ready-made products; and (4) most studies used single measurement, not considering the use of the product over time. This scenario shows an opportunity for UX evaluation methods which consider the use of the product over time and provide real time results.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Pine, B.J., Gilmore, J.H.: Welcome to experience economy. Harvard Bus. Rev. 76, 97–105 (1998) Pine, B.J., Gilmore, J.H.: Welcome to experience economy. Harvard Bus. Rev. 76, 97–105 (1998)
2.
Zurück zum Zitat Kujala, S., Roto, V., Vaananen-Vainio-Mattila, K., Karapanos, E., Sinnela, A.: UX Curve: a method for evaluating long-term user experience. Interact. Comput. 23(5), 473–483 (2011)CrossRef Kujala, S., Roto, V., Vaananen-Vainio-Mattila, K., Karapanos, E., Sinnela, A.: UX Curve: a method for evaluating long-term user experience. Interact. Comput. 23(5), 473–483 (2011)CrossRef
3.
Zurück zum Zitat Sproll, S., Peissner, M., Sturm, C.: From product concept to user experience- Exploring UX potentials at early product stages. In: Nordic Conference on Human-Computer Interaction, pp. 473–482 (2010) Sproll, S., Peissner, M., Sturm, C.: From product concept to user experience- Exploring UX potentials at early product stages. In: Nordic Conference on Human-Computer Interaction, pp. 473–482 (2010)
4.
Zurück zum Zitat Roto, V., Obrist, M., Vaananen-Vainio-Mattila, K.: User experience evaluation methods in academic and industrial contexts. In: International Conference on Human-Computer Interaction (2009) Roto, V., Obrist, M., Vaananen-Vainio-Mattila, K.: User experience evaluation methods in academic and industrial contexts. In: International Conference on Human-Computer Interaction (2009)
5.
Zurück zum Zitat Makela, S., Bednarik, R., Tukiainen, M.: Evaluating user experience of autistic children through video observation. In: ACM Conference on Human Factors in Computing Systems, Extended Abstracts on Human Factors in Computing Systems, pp. 463–468 (2013) Makela, S., Bednarik, R., Tukiainen, M.: Evaluating user experience of autistic children through video observation. In: ACM Conference on Human Factors in Computing Systems, Extended Abstracts on Human Factors in Computing Systems, pp. 463–468 (2013)
6.
Zurück zum Zitat Keskinen, T., Hakulinen, J., Heimoen, T., Turunen, M., Sharma, S., Miettinen, T., Luhtala, M.: Evaluating the experiential user experience of public display applications in the wild. In: International Conference on Mobile and Ubiquitous Multimedia (2013) Keskinen, T., Hakulinen, J., Heimoen, T., Turunen, M., Sharma, S., Miettinen, T., Luhtala, M.: Evaluating the experiential user experience of public display applications in the wild. In: International Conference on Mobile and Ubiquitous Multimedia (2013)
7.
Zurück zum Zitat Foglia, P., Zanda, M.: Towards relating physiological signals to usability metrics: a case study with a web avatar. WSEAS Trans. Comput. 13, 624–634 (2014) Foglia, P., Zanda, M.: Towards relating physiological signals to usability metrics: a case study with a web avatar. WSEAS Trans. Comput. 13, 624–634 (2014)
8.
Zurück zum Zitat Yao, L., Liu, Y., Li, W., Zhou, L., Ge, Y., Chai, J., Sun, X.: Using physiological measures to evaluate user experience of mobile applications. In: Harris, D. (ed.) EPCE 2014. LNCS, vol. 8532, pp. 301–310. Springer, Heidelberg (2014) Yao, L., Liu, Y., Li, W., Zhou, L., Ge, Y., Chai, J., Sun, X.: Using physiological measures to evaluate user experience of mobile applications. In: Harris, D. (ed.) EPCE 2014. LNCS, vol. 8532, pp. 301–310. Springer, Heidelberg (2014)
9.
Zurück zum Zitat Wiebe, E.N., Lamb, A., Hardy, M., Sharek, D.: Measuring engagement in video game-based environments: investigation of the user engagement scale. Comput. Hum. Behav. 32, 123–132 (2014)CrossRef Wiebe, E.N., Lamb, A., Hardy, M., Sharek, D.: Measuring engagement in video game-based environments: investigation of the user engagement scale. Comput. Hum. Behav. 32, 123–132 (2014)CrossRef
10.
Zurück zum Zitat Read, J. C.: Evaluating artefacts with children: age and technology effects in the reporting of expected and experienced fun. In: International Conference on Multimodal Interaction, pp. 241–248 (2012) Read, J. C.: Evaluating artefacts with children: age and technology effects in the reporting of expected and experienced fun. In: International Conference on Multimodal Interaction, pp. 241–248 (2012)
11.
Zurück zum Zitat Masip, L., Oliva, M., Granollers, T.: OPEN-HEREDEUX: OPEN HEuristic REsource for Designing and Evaluating User eXperience. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds.) INTERACT 2011, Part IV. LNCS, vol. 6949, pp. 418–421. Springer, Heidelberg (2011)CrossRef Masip, L., Oliva, M., Granollers, T.: OPEN-HEREDEUX: OPEN HEuristic REsource for Designing and Evaluating User eXperience. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds.) INTERACT 2011, Part IV. LNCS, vol. 6949, pp. 418–421. Springer, Heidelberg (2011)CrossRef
12.
Zurück zum Zitat Mercun, T.: Evaluation of information visualization techniques – analysing user experience with reaction cards. In: Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization, pp. 103–109 (2014) Mercun, T.: Evaluation of information visualization techniques – analysing user experience with reaction cards. In: Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization, pp. 103–109 (2014)
13.
Zurück zum Zitat Law, E.L., Roto, V., Hassenzahl, M., Vermeeren, A., Kort, J.: Understanding, scoping and defining user experience: a survey approach. In: ACM Conference on Human Factors in Computing Systems, pp. 719–728 (2009) Law, E.L., Roto, V., Hassenzahl, M., Vermeeren, A., Kort, J.: Understanding, scoping and defining user experience: a survey approach. In: ACM Conference on Human Factors in Computing Systems, pp. 719–728 (2009)
14.
Zurück zum Zitat Hassenzahl, M.: The interplay of beauty, goodness, and usability in interactive products. Hum.-Comput. Interac. 19(4), 319–349 (2004)CrossRef Hassenzahl, M.: The interplay of beauty, goodness, and usability in interactive products. Hum.-Comput. Interac. 19(4), 319–349 (2004)CrossRef
15.
Zurück zum Zitat Mahlke, S., Thuring, M.: Studying antecedents of emotional experiences in interactive contexts. In: ACM Conference on Human Factors in Computing Systems, pp. 915–918 (2007) Mahlke, S., Thuring, M.: Studying antecedents of emotional experiences in interactive contexts. In: ACM Conference on Human Factors in Computing Systems, pp. 915–918 (2007)
16.
Zurück zum Zitat Roto, V., Rantavuo, H., Vaananen-Vainio-Mattila, K.: Evaluating user experience of early product concepts. In: International Conference on Designing Pleasurable Products and Interfaces, pp. 199–208 (2009) Roto, V., Rantavuo, H., Vaananen-Vainio-Mattila, K.: Evaluating user experience of early product concepts. In: International Conference on Designing Pleasurable Products and Interfaces, pp. 199–208 (2009)
17.
Zurück zum Zitat Law, E., Schaik, P., Roto, V.: Attitudes towards user experience (UX) measurement. Int. J. Hum.-Comput. Stud. 72(6), 526–541 (2014)CrossRef Law, E., Schaik, P., Roto, V.: Attitudes towards user experience (UX) measurement. Int. J. Hum.-Comput. Stud. 72(6), 526–541 (2014)CrossRef
18.
Zurück zum Zitat Robert, J., Lesage, A.: Designing and evaluating user experience. In: Handbook of Human-Machine Interaction (2011) Robert, J., Lesage, A.: Designing and evaluating user experience. In: Handbook of Human-Machine Interaction (2011)
19.
Zurück zum Zitat Mashapa, J., van Greunen, D.: User experience evaluation metrics for usable accounting tools. In: Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists, pp. 170–181 (2010) Mashapa, J., van Greunen, D.: User experience evaluation metrics for usable accounting tools. In: Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists, pp. 170–181 (2010)
20.
Zurück zum Zitat Kujala, S., Roto, V., Vaananen-Vainio-Mattila, K., Sinnela, A.: Identifying hedonic factors in long-term user experience. In: Conference on Designing Pleasurable Products and Interfaces (2011) Kujala, S., Roto, V., Vaananen-Vainio-Mattila, K., Sinnela, A.: Identifying hedonic factors in long-term user experience. In: Conference on Designing Pleasurable Products and Interfaces (2011)
21.
Zurück zum Zitat Nawaz, A., Halbostad, J.L., Chiari, L., Chesani, F., Cattelani, L.: User Experience (UX) of the Fall Risk Assessment Tool (FRAT-up). In: International Symposium on Computer-Based Medical Systems, pp. 19–22 (2015) Nawaz, A., Halbostad, J.L., Chiari, L., Chesani, F., Cattelani, L.: User Experience (UX) of the Fall Risk Assessment Tool (FRAT-up). In: International Symposium on Computer-Based Medical Systems, pp. 19–22 (2015)
22.
Zurück zum Zitat Carofiglio, V., Ricci, G., Abbattista, F.: User brain-driven evaluation of an educational 3D virtual environment. In: Iberian Conference on Information Systems and Technologies, pp. 1–7 (2015) Carofiglio, V., Ricci, G., Abbattista, F.: User brain-driven evaluation of an educational 3D virtual environment. In: Iberian Conference on Information Systems and Technologies, pp. 1–7 (2015)
23.
Zurück zum Zitat Lasa, G., Justel, D., Retegi, A.: Eyeface: A new multimethod tool to evaluate the perception of conceptual user experiences. Comput. Hum. Behav. 52, 359–363 (2015)CrossRef Lasa, G., Justel, D., Retegi, A.: Eyeface: A new multimethod tool to evaluate the perception of conceptual user experiences. Comput. Hum. Behav. 52, 359–363 (2015)CrossRef
24.
Zurück zum Zitat Walsh, T., Varsaluoma, J., Kujala, S., Nurkka, P., Petrie, H., Power, C.: Axe UX: Exploring long-term user experience with iScale and AttrakDiff. In: International Academic Mindtrek Conference: Media Business, Management, Content & Services, pp. 32–39 (2014) Walsh, T., Varsaluoma, J., Kujala, S., Nurkka, P., Petrie, H., Power, C.: Axe UX: Exploring long-term user experience with iScale and AttrakDiff. In: International Academic Mindtrek Conference: Media Business, Management, Content & Services, pp. 32–39 (2014)
25.
Zurück zum Zitat Colombo, L., Landoni, M.: A diary study of children’s user experience with EBooks using flow theory as framework. In: Conference on Interaction Design and Children, pp. 135–144 (2014) Colombo, L., Landoni, M.: A diary study of children’s user experience with EBooks using flow theory as framework. In: Conference on Interaction Design and Children, pp. 135–144 (2014)
26.
Zurück zum Zitat Santoso, H.B., Isal, R., Yugo K., Basaruddin, T., Sadira, L., Schrepp, M.: Research-in-progress: user experience evaluation of Student Centered E-Learning Environment for computer science program. In: International Conference on User Science and Engineering, pp. 52–55 (2014) Santoso, H.B., Isal, R., Yugo K., Basaruddin, T., Sadira, L., Schrepp, M.: Research-in-progress: user experience evaluation of Student Centered E-Learning Environment for computer science program. In: International Conference on User Science and Engineering, pp. 52–55 (2014)
27.
Zurück zum Zitat Keskinen, T., Hakulinen, J., Heimoen, T., Turunen, M., Sharma, S., Miettinen, T., Luhtala, M.: Evaluating the experiential user experience of public display applications in the wild. In: International Conference on Mobile and Ubiquitous Multimedia (2013) Keskinen, T., Hakulinen, J., Heimoen, T., Turunen, M., Sharma, S., Miettinen, T., Luhtala, M.: Evaluating the experiential user experience of public display applications in the wild. In: International Conference on Mobile and Ubiquitous Multimedia (2013)
28.
Zurück zum Zitat Saarinen, P., Partala, T., Vaananen-Vainio-Mattila, K: Little backpackers: studying children’s psychological needs in an interactive exhibition context. In: International Conference on Interaction Design and Children, pp. 415–418 (2013) Saarinen, P., Partala, T., Vaananen-Vainio-Mattila, K: Little backpackers: studying children’s psychological needs in an interactive exhibition context. In: International Conference on Interaction Design and Children, pp. 415–418 (2013)
29.
Zurück zum Zitat Sim, G., Cassidy, B.: Investigating the fidelity effect when evaluating game prototypes with children. In: International BCS Human Computer Interaction Conference (2013) Sim, G., Cassidy, B.: Investigating the fidelity effect when evaluating game prototypes with children. In: International BCS Human Computer Interaction Conference (2013)
30.
Zurück zum Zitat Changyuan, G., Shiying, W., Chongran, Z.: Research on user experience evaluation system of information platform based on web environment. In: International Conference on Measurement, Information and Control, vol. 1, pp. 558–562 (2013) Changyuan, G., Shiying, W., Chongran, Z.: Research on user experience evaluation system of information platform based on web environment. In: International Conference on Measurement, Information and Control, vol. 1, pp. 558–562 (2013)
31.
Zurück zum Zitat Sim, G., Horton, M., Danino, N.: Evaluating game preference using the fun toolkit across cultures. In: Interaction Specialist Group Conference on People and Computers, pp. 386–391 (2012) Sim, G., Horton, M., Danino, N.: Evaluating game preference using the fun toolkit across cultures. In: Interaction Specialist Group Conference on People and Computers, pp. 386–391 (2012)
32.
Zurück zum Zitat Sim, G., Horton, M.: Investigating children’s opinions of games: Fun Toolkit vs. This or That. In: International Conference on Interaction Design and Children, pp. 70–77 (2012) Sim, G., Horton, M.: Investigating children’s opinions of games: Fun Toolkit vs. This or That. In: International Conference on Interaction Design and Children, pp. 70–77 (2012)
33.
Zurück zum Zitat Staiano, J., Menéndez, M., Battocchi, A., de Angeli, A., Sebe, N.: UX_Mate: from facial expressions to UX evaluation. In: Designing Interactive Systems Conference, pp. 741–750 (2012) Staiano, J., Menéndez, M., Battocchi, A., de Angeli, A., Sebe, N.: UX_Mate: from facial expressions to UX evaluation. In: Designing Interactive Systems Conference, pp. 741–750 (2012)
34.
Zurück zum Zitat Kronbauer, A. H., Santos, C., Vieira, V.: Um Estudo Experimental de Avaliação da Experiência dos Usuários de Aplicativos Móveis a partir da Captura Automática dos Dados Contextuais e de Interação. In: Brazilian Symposium on Human Factors in Computing Systems, pp. 305–314 (2012) Kronbauer, A. H., Santos, C., Vieira, V.: Um Estudo Experimental de Avaliação da Experiência dos Usuários de Aplicativos Móveis a partir da Captura Automática dos Dados Contextuais e de Interação. In: Brazilian Symposium on Human Factors in Computing Systems, pp. 305–314 (2012)
35.
Zurück zum Zitat Rauschenberger, M., Olachner, S., Cota, M.P., Schrepp, M., Thomasschewski, J.: Measurement of user experience: a Spanish language version of the user experience questionnaire (UEQ). In: Iberian Conference on Information Systems and Technologies (2012) Rauschenberger, M., Olachner, S., Cota, M.P., Schrepp, M., Thomasschewski, J.: Measurement of user experience: a Spanish language version of the user experience questionnaire (UEQ). In: Iberian Conference on Information Systems and Technologies (2012)
36.
Zurück zum Zitat Bach, C., Gauducheau, N., Salembier, P.: Combining interviews and scales in the multidimensional evaluation of user experience: a case study in 3D games. In: Annual European Conference on Cognitive Ergonomics, pp. 157–160 (2011) Bach, C., Gauducheau, N., Salembier, P.: Combining interviews and scales in the multidimensional evaluation of user experience: a case study in 3D games. In: Annual European Conference on Cognitive Ergonomics, pp. 157–160 (2011)
37.
Zurück zum Zitat Schulze, K., Kromker, H.: A framework to measure user experience of interactive online products. In: International Conference on Methods and Techniques in Behavioral Research (2010) Schulze, K., Kromker, H.: A framework to measure user experience of interactive online products. In: International Conference on Methods and Techniques in Behavioral Research (2010)
38.
Zurück zum Zitat Meschtscherjakov, A., Reitberger, W., Tscheligi, M.: MAESTRO: orchestrating user behavior driven and context triggered experience sampling. In: International Conference on Methods and Techniques in Behavioral Research, p. 29 (2010) Meschtscherjakov, A., Reitberger, W., Tscheligi, M.: MAESTRO: orchestrating user behavior driven and context triggered experience sampling. In: International Conference on Methods and Techniques in Behavioral Research, p. 29 (2010)
39.
Zurück zum Zitat Korhonen, H., Arrasvuori, J., Vaananen-Vainio-Mattila, K.: Let users tell the story: evaluating user experience with experience reports. In: ACM Conference on Human Factors in Computing Systems, Extended Abstracts, pp. 4051–4056 (2010) Korhonen, H., Arrasvuori, J., Vaananen-Vainio-Mattila, K.: Let users tell the story: evaluating user experience with experience reports. In: ACM Conference on Human Factors in Computing Systems, Extended Abstracts, pp. 4051–4056 (2010)
40.
Zurück zum Zitat Agarwal, A., Meyer, A.: Beyond usability: evaluating emotional response as an integral part of the user experience. In: ACM Conference on Human Factors in Computing Systems, Extended Abstracts, pp. 2919–2930 (2009) Agarwal, A., Meyer, A.: Beyond usability: evaluating emotional response as an integral part of the user experience. In: ACM Conference on Human Factors in Computing Systems, Extended Abstracts, pp. 2919–2930 (2009)
Metadaten
Titel
A Systematic Review About User Experience Evaluation
verfasst von
Camila Loiola Brito Maia
Elizabeth Sucupira Furtado
Copyright-Jahr
2016
DOI
https://doi.org/10.1007/978-3-319-40409-7_42

Neuer Inhalt