skip to main content
10.1145/1996461.1996485acmconferencesArticle/Chapter ViewAbstractPublication PageseicsConference Proceedingsconference-collections
keynote

The measurability and predictability of user experience

Published:13 June 2011Publication History

ABSTRACT

User experience is an emerging research area with a range of issues to be resolved. Among them, the measurability of UX remains contentious. The key argument hinges on the meaningfulness, validity and usefulness of reducing fuzzy experiential qualities such as fun, challenge and trust to numbers. UX people seem ambivalent towards UX measures. In UX empirical studies, qualitative approaches are predominant, though the popular use of questionnaires in these studies suggests that some form of numeric measures is deemed useful or even necessary. The tension between the two camps (i.e. qualitative design-based and quantitative model-based) stimulates scientific discussions to bring the field forward. As measures may enable us to predict, the concomitant issue of UX predictability is explored. Besides, we look into theoretical frameworks that potentially contribute to a deeper understanding of UX. Of particular interest is theory of memory.

References

  1. Blythe, M., Wright, P., McCarthy, J. & Bertelsen, O.W. (2006). Theory and method for experience centered design. In Proc. CHI EA '06. ACM, NY, USA, 1691--1694. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Blythe, M., McCarthy, J., Light, A., Bardzell, S., Wright, P., Bardzell, J., & Blackwell, A (2010). Critical dialogue: interaction, experience and cultural theory. In Proc. CHI EA '10. ACM, NY, USA, 4521--4524. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Bulmer, M. (2001). Social measurement: What stands in its way? Social Research, 62(2).Google ScholarGoogle Scholar
  4. Bargas-Avila, J.A., & Hornbæk, K. (2011). Old wine in new bottles or novel challenges? A critical analysis of empirical studies of user experience. In Proc. CHI'11. Vancouver, Canada. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Boehner, K., DePaula, R., Dourish, P., & Senger, P. (2007). How emotion is made and measured? Journal International Journal of Human-Computer Studies, 65(4). Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Bødker, S. (1990). Through the interface: a human activity approach to user interface design. Hillsdale, NJ: Lawrence Erlbaum Associates. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Cannon, W.B. (1927). The James-Lange Theory of Emotions: A Critical Examination and an Alternative Theory. The American Journal of Psychology. 39 (1/4).Google ScholarGoogle Scholar
  8. Damasio, A. (1999). The Feeling of What Happens. Body, Emotion and the Making of Consciousness. Heinemann: London.Google ScholarGoogle Scholar
  9. Desmet, P. (2004). Measuring Emotion. In M.A. Blythe, A.F. Monk, K. Overbeeke, & P.C. Wright (Eds.), Funology: from usability to enjoyment. Kluwer.Google ScholarGoogle Scholar
  10. Følstad, A., Law, E. L-C., Hornbæk, K. (2010). Analysis in usability evaluations: an exploratory study. In Proc. NordiCHI 2010 (pp. 647--650). Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Forlizzi, J., & Battarbee, K. (2004). Understanding experience in interactive systems. In Proc. DIS '04, 261--268. ACM, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. French, T., Liu, K., & Springett, M. (2007). A card-sorting probe of e-banking trust perceptions. BCS HCI (1), 45--5. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Furniss, D., Blandford, A., Curzon, P. (2008). Usability Work in Professional Website Design: Insights from Practitioners' Perspectives. In Law, E., Hvannberg, E., Cockton, G. (Eds.). Maturing Usability: Quality in Software, Interaction and Value (pp. 144--167). London: Springer London.Google ScholarGoogle Scholar
  14. Hartmann, J., De Angeli, A. & Sutcliffe, A. (2008). Framing the user experience: information biases on website quality judgement. In Proc. CHI '08, 855--864. ACM, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Hassenzahl, M. (2010). Experience Design: Technology for all the right reasons. Synthesis Lectures in Human-Centered Informatics (no. 8). San Rafael, CA: Morgan & Claypool Publishers. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Hassenzahl, M., Diefenbach, S., & Göritz, A. (2010). Needs, affect, and interactive products -- facets of user experience. Interacting with Computers, 22:353--362. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Hassenzahl, M., & Ullrich, D. (2007). To do or not to do: Differences in user experience and retrospective judgments depending on the presence or absence of instrumental goals. Interacting with Computers 19(4), 429--437. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Hornbæk, K., & Law, E. L-C. (2007). Meta-analysis of correlations among usability measures. In Proc. CHI 2007, San Jose, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Isomursu, M., Tähti, M., Väinämö, S. & Kuutti, K. (2007). Experimental evaluation of five methods for collecting emotions in field settings with mobile applications. International Journal of Human-Computer Studies, 65(4):404--418. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Izard, C. E. Libero, D. Z., Putnam, P., Haynes, O. M. (1993). Stability of emotion experiences and their relations to traits of personality. Journal of Personality and Social Psychology, 64(5), 847--860.Google ScholarGoogle ScholarCross RefCross Ref
  21. James, W. (1884). What is emotion? Mind, ix, 189.Google ScholarGoogle Scholar
  22. Karapanos, E., Zimmerman, J. Forlizzi, J., & Martens, J. (2009). User experience over time: an initial framework. In Proc. CHI '09, 729--738. ACM, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Karapanos, E., Zimmerman, J. Forlizzi, J., & Martens, J. (2010). Measuring the Dynamics of Remembered Experience Over Time. Interacting with Computers, 22(5):328--335. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Kaptelinin, V. (1996). Computer-mediated activity: Functional organs in social and developmental contexts. In B. Nardi (Ed.), Context and consciousness: Activity theory and human-computer interaction (45--68). Cambridge, MA: MI. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Kaptelinin, V., & Nardi, B.A. (2006). Acting with technology: Activity theory and interaction design. Cambridge, MA: MIT press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Kuutti, K. (2010). Where are the Ionians of user experience research?. In Proc. NordiCHI '10. ACM, NY, USA, 715--718. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Kindlund, E. & Sauro, J. (2005), "A method to standardize usability metrics into a single score", In Proceedings of CHI 2005 ACM Conference on Human Factors in Computing Systems, 401--409. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Lang, P. J. (1994). The varieties of emotional experience: A meditation on James-Lange theory. Psychological Review, 101(2), 211--221.Google ScholarGoogle ScholarCross RefCross Ref
  29. Lange, C. (1887). Ueber Gemuthsbewgungen, 3, 8.Google ScholarGoogle Scholar
  30. Lavie, T., & Tractinsky, N. (2004). Assessing dimensions of perceived visual aesthetics of web sites. International Journal of Human-Computer Studies, 60(3):269--298. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Luojus, S. (2010). From a momentary experience to a lasting one - The concept of and research on expanded user experience of mobile devices. Doctoral Dissertation, Acta Universitatis Ouluensis Scientiae Rerum Naturalium A 559, Oulu.Google ScholarGoogle Scholar
  32. Law, E. L-C, Roto, V., Hassenzahl, M., Vermeeren, A., & Kort, J. (2009). Understanding, scoping and defining user experience: a survey approach. In Proc. CHI '09, 719--728. ACM, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Law, E. L-C., & van Schaik. P. (2010). Modelling user experience -- an agenda for research and practice. Interacting with Computers, 22(5):313--322. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Law, E. L-C., & Sun, X. (under review). Evaluating User Experience of Adaptive Digital Educational Games with Activity Theory. International Journal of Human Computer Studies. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. McGee, M. (2004), "Master usability scaling: magnitude estimation and master scaling applied to usability measurement", In Proc. of CHI, 335--342. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Müller, D., Law, E., & Strohmeier, S. (2010). Analysis of the Persuasiveness of User Experience Feedback on a Virtual Learning Environment. In Proc. of Workshop on Interplay between User Experience Evaluation and System Development (I-UxSED'10) in conjunction with NordiCHI 2010, Iceland.Google ScholarGoogle Scholar
  37. Nardi, B.A. (1996) 'Activity theory and human omputer interaction', In B.A. Nardi (Ed), Context and Consciousness: Activity Theory and Human-Computer Interaction (pp.7--16). Cambridge, MA: MIT Pres. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Nørgaard, M., Høegh, R.T. (2008). Evaluating usability: using models of argumentation to improve persuasiveness of usability feedback. In Proc. DIS 2008 (pp. 212--221). Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Norman, D. A. (1990): The Design of Everyday Things. New York, Doubleday.Google ScholarGoogle Scholar
  40. Obrist, M., Law, E.L-C., Väänänen-Vainio-Mattila, K., Roto, V., Vermeeren, A., & Kuutti, K. (2011). UX research- which theoretical roots do we build on -- if any. In Extended Abstract CHI'11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Roussou, M., Oliver, M., and Slater, M.: Exploring activity theory as a tool for evaluating interactivity and learning in virtual environments for children. Cognition, Technology and Work, 10, (2008), 141--15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Roto, V., Law, E., Vermeeren, A., & Hoonholt, J. (2011). User Experience White Paper. Retrieved from http://www.allaboutux.org.Google ScholarGoogle Scholar
  43. Tullis, T., & Albert, W. (2008). Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics (Interactive Technologies). Morgan Kaufman. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Vermeeren, A. P.O.S. Law, E. L-C., Roto, V., Obrist, M., Hoonhout, J., Väänänen-Vainio-Mattila, K. (2010). User experience evaluation methods: current state and development needs. In Proc NordiCHI 2010 (pp. 521--530). Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Winograd, T., & Flores, F. (1986). Understanding computers and cognition: A new foundation for design. Ablex. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Wright, P., & McCarthy, J. (2008). Empathy and experience in HCI. In Proc. CHI'08. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Wright, P., & McCarthy J. (2010). Experience-Centered Design: Designers, Users, and Communities in Dialogue. Synthesis Lectures in Human-Centered Informatics, ed. John Carroll, no. 8, San Rafael, CA: Morgan & Claypool Publishers. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. The measurability and predictability of user experience

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      EICS '11: Proceedings of the 3rd ACM SIGCHI symposium on Engineering interactive computing systems
      June 2011
      356 pages
      ISBN:9781450306706
      DOI:10.1145/1996461

      Copyright © 2011 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 13 June 2011

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • keynote

      Acceptance Rates

      Overall Acceptance Rate73of299submissions,24%

      Upcoming Conference

      EICS '24

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader