Top

Published in:

2017 | OriginalPaper | Chapter

Subjective Usability, Mental Workload Assessments and Their Impact on Objective Human Performance

Author : Luca Longo

Published in: Human-Computer Interaction - INTERACT 2017

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Self-reporting procedures and inspection methods have been largely employed in the fields of interaction and web-design for assessing the usability of interfaces. However, there seems to be a propensity to ignore features related to end-users or the context of application during the usability assessment procedure. This research proposes the adoption of the construct of mental workload as an additional aid to inform interaction and web-design. A user-study has been performed in the context of human-web interaction. The main objective was to explore the relationship between the perception of usability of the interfaces of three popular web-sites and the mental workload imposed on end-users by a set of typical tasks executed over them. Usability scores computed employing the System Usability Scale were compared and related to the mental workload scores obtained employing the NASA Task Load Index and the Workload Profile self-reporting assessment procedures. Findings advise that perception of usability and subjective assessment of mental workload are two independent, not fully overlapping constructs. They measure two different aspects of the human-system interaction. This distinction enabled the demonstration of how these two constructs cab be jointly employed to better explain objective performance of end-users, a dimension of user experience, and informing interaction and web-design.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Night Mode, Dark Thoughts: Background Color Influences the Perceived Sentiment of Chat Messages

next chapter What is User’s Perception of Naturalness? An Exploration of Natural User Experience

Addie, J., Niels, T.: Processing resources and attention. In: Handbook of Human Factors in Web Design, pp. 3424–3439. Lawrence Erlbaum Associates (2005)

Albers, M.: Tapping as a measure of cognitive load and website usability. In: Proceedings of the 29th ACM International Conference on Design of Communication, pp. 25–32 (2011)

Alonso-Ríos, D., Vázquez-García, A., Mosqueira-Rey, E., Moret-Bonillo, V.: A context-of-use taxonomy for usability studies. Int. J. Hum.-Comput. Interact. 26(10), 941–970 (2010)CrossRef

Annett, J.: Subjective rating scales in ergonomics: a reply. Ergonomics 45(14), 1042–1046 (2002)CrossRef

Annett, J.: Subjective rating scales: science or art? Ergonomics 45(14), 966–987 (2002)CrossRef

Baber, C.: Subjective evaluation of usability. Ergonomics 45(14), 1021–1025 (2002)CrossRef

Bangor, A., Kortum, P.T., Miller, J.T.: An empirical evaluation of the system usability scale. Int. J. Hum.-Comput. Interact. 24(6), 574–594 (2008)CrossRef

Bennett, K.P., Campbell, C.: Support vector machines: hype or hallelujah? SIGKDD Explor. Newsl. 2(2), 1–13 (2000)CrossRef

Brooke, J.: SUS: a quick and dirty usability scale. In: Jordan, P.W., Weerdmeester, B., Thomas, A., Mclelland, I.L. (eds.) Usability Evaluation in Industry. Taylor and Francis, London (1996)

10.

Cain, B.: A review of the mental workload literature. Technical report, Defence Research and Development Canada, Human System Integration (2007)

11.

Cleveland, W.S.: Robust locally weighted regression and smoothing scatterplots. Am. Stat. Assoc. 74, 829–836 (1979)MathSciNetCrossRefMATH

12.

Cohen, J.: Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates, Mahwah (1988)MATH

13.

Cooper, G.E., Harper, R.P.: The use of pilot ratings in the evaluation of aircraft handling qualities. Technical report AD689722, 567, Advisory Group for Aerospace Research and Development, April 1969

14.

De Waard, D.: The measurement of drivers’ mental workload. University of Groningen, The Traffic Research Centre VSC (1996)

15.

Edwards, A., Kelly, D., Azzopardi, L.: The impact of query interface design on stress, workload and performance. In: Hanbury, A., Kazai, G., Rauber, A., Fuhr, N. (eds.) ECIR 2015. LNCS, vol. 9022, pp. 691–702. Springer, Cham (2015). doi:10.1007/978-3-319-16354-3_76

16.

Fischer, G.: User modeling in human-computer interaction. User Model. User-Adapt. Interact. 11(1–2), 65–86 (2001)CrossRefMATH

17.

Gwizdka, J.: Assessing cognitive load on web search tasks. Ergon. Open J. 2(1), 114–123 (2009)CrossRef

18.

Gwizdka, J.: Distribution of cognitive load in web search. J. Am. Soc. Inf. Sci. Technol. 61(11), 2167–2187 (2010)CrossRef

19.

Harper, B.D., Norman, K.L.: Improving user satisfaction: the questionnaire for user interaction satisfaction version 5.5. In: 1st Annual Mid-Atlantic Human Factors Conference, pp. 224–228 (1993)

20.

Hart, S.G.: Nasa-task load index (NASA-TLX); 20 years later. In: Human Factors and Ergonomics Society Annual Meeting, vol. 50. SAGE Journals (2006)

21.

Hornbaek, K.: Current practice in measuring usability: challenges to usability studies and research. Int. J. Hum.-Comput. Stud. 64(2), 79–102 (2006)CrossRef

22.

Huey, B.M., Wickens, C.D.: Workload Transition: Implication for Individual and Team Performance. National Academy Press, Washington, D.C. (1993)

23.

Karatzoglou, A., Meyer, D.: Support vector machines in R. J. Stat. Softw. 15(9), 1–32 (2006)CrossRef

24.

Lehmann, J., Lalmas, M., Yom-Tov, E., Dupret, G.: Models of user engagement. In: Masthoff, J., Mobasher, B., Desmarais, M.C., Nkambou, R. (eds.) UMAP 2012. LNCS, vol. 7379, pp. 164–175. Springer, Heidelberg (2012). doi:10.1007/978-3-642-31454-4_14 CrossRef

25.

Lewis, J.R.: IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum.-Comput. Interact. 7, 57–78 (1995)CrossRef

26.

Longo, L., Kane, B.: A novel methodology for evaluating user interfaces in health care. In: 2011 24th International Symposium on Computer-Based Medical Systems (CBMS), pp. 1–6, June 2011

27.

Longo, L.: Human-computer interaction and human mental workload: assessing cognitive engagement in the world wide web. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds.) INTERACT 2011. LNCS, vol. 6949, pp. 402–405. Springer, Heidelberg (2011). doi:10.1007/978-3-642-23768-3_43 CrossRef

28.

Longo, L.: Formalising human mental workload as non-monotonic concept for adaptive and personalised web-design. In: Masthoff, J., Mobasher, B., Desmarais, M.C., Nkambou, R. (eds.) UMAP 2012. LNCS, vol. 7379, pp. 369–373. Springer, Heidelberg (2012). doi:10.1007/978-3-642-31454-4_38 CrossRef

29.

Longo, L.: Formalising human mental workload as a defeasible computational concept. Ph.D. thesis, Trinity College Dublin (2014)

30.

Longo, L.: A defeasible reasoning framework for human mental workload representation and assessment. Behav. Inf. Technol. 34(8), 758–786 (2015)CrossRef

31.

Longo, L.: Designing medical interactive systems via assessment of human mental workload. In: International Symposium on Computer-Based Medical Systems, pp. 364–365 (2015)

32.

Longo, L.: Mental workload in medicine: foundations, applications, open problems, challenges and future perspectives. In: 2016 IEEE 29th International Symposium on Computer-Based Medical Systems (CBMS), pp. 106–111, June 2016

33.

Longo, L., Barrett, S.: A computational analysis of cognitive effort. In: Nguyen, N.T., Le, M.T., Świątek, J. (eds.) ACIIDS 2010, Part II. LNCS, vol. 5991, pp. 65–74. Springer, Heidelberg (2010). doi:10.1007/978-3-642-12101-2_8 CrossRef

34.

Longo, L., Dondio, P.: On the relationship between perception of usability and subjective mental workload of web interfaces. In: IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2015, Singapore, 6–9 December, vol. I, pp. 345–352 (2015)

35.

Longo, L., Rusconi, F., Noce, L., Barrett, S.: The importance of human mental workload in web-design. In: 8th International Conference on Web Information Systems and Technologies, pp. 403–409. SciTePress, April 2012

36.

Macleod, M.: Usability in context: improving quality of use. In: Proceedings of the International Ergonomics Association 4th International Symposium on Human Factors in Organizational Design and Management. Elsevier (1994)

37.

Moustafa, K., Luz, S., Longo, L.: Assessment of mental workload: a comparison of machine learning methods and subjective assessment techniques. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2017. CCIS, vol. 726, pp. 30–50. Springer, Cham (2017). doi:10.1007/978-3-319-61061-0_3 CrossRef

38.

Nielsen, J.: Heuristic evaluation. In: Nielsen, J., Mack, R.L.E. (eds.) Usability Inspection Methods. Wiley, New York (1994)CrossRef

39.

Nielsen, J.: Usability inspection methods. In: Conference Companion on Human Factors in Computing Systems, CHI 1995, pp. 377–378. ACM, New York (1995)

40.

O’Donnel, R.D., Eggemeier, T.F.: Workload assessment methodology. In: Boff, K., Kaufman, L., Thomas, J. (eds.) Handbook of Perception and Human Performance, pp. 42/1–42/49. Wiley-Interscience, New York (1986)

41.

O’Brien, H.L., Toms, E.G.: What is user engagement? A conceptual framework for defining user engagement with technology. J. Am. Soc. Inf. Sci. Technol. 59(6), 938–955 (2008). doi:10.1002/asi.20801 CrossRef

42.

Reid, G.B., Nygren, T.E.: The subjective workload assessment technique: a scaling procedure for measuring mental workload. In: Hancock, P.A., Meshkati, N. (eds.) Human Mental Workload, Advances in Psychology, vol. 52, chap. 8, pp. 185–218. North-Holland, Amsterdam (1988)

43.

Rizzo, L., Dondio, P., Delany, S.J., Longo, L.: Modeling mental workload via rule-based expert system: a comparison with NASA-TLX and workload profile. In: Iliadis, L., Maglogiannis, I. (eds.) AIAI 2016. IAICT, vol. 475, pp. 215–229. Springer, Cham (2016). doi:10.1007/978-3-319-44944-9_19 CrossRef

44.

Roscoe, A.H., Ellis, G.A.: A subjective rating scale for assessing pilot workload in flight: a decade of practical use. Technical report 90019, Royal Aerospace Establishment, Farnborough (UK), March 1990

45.

Rubio, S., Diaz, E., Martin, J., Puente, J.M.: Evaluation of subjective mental workload: a comparison of SWAT, NASA-TLX, and workload profile methods. Appl. Psychol. 53(1), 61–86 (2004)CrossRef

46.

Saket, B., Endert, A., Stasko, J.: Beyond usability and performance: a review of user experience-focused evaluations in visualization. In: Proceedings of the Sixth Workshop on Beyond Time and Errors on Novel Evaluation Methods for Visualization, BELIV 2016, pp. 133–142. ACM, New York (2016). http://doi.acm.org/10.1145/2993901.2993903

47.

Schmutz, P., Heinz, S., Métrailler, Y., Opwis, K.: Cognitive load in ecommerce applications: measurement and effects on user satisfaction. Adv. Hum.-Comput. Interact. 2009, 3/1–3/9 (2009)CrossRef

48.

Shackel, B.: Usability - context, framework, definition, design and evaluation. Interact. Comput. 21(5–6), 339–346 (2009)CrossRef

49.

Slaughter, L.A., Harper, B.D., Norman, K.L.: Assessing the equivalence of paper and on-line versions of the QUIS 5.5. In: 2nd Annual Mid-Atlantic Human Factors Conference, pp. 87–91 (1994)

50.

Tracy, J.P., Albers, M.J.: Measuring cognitive load to test the usability of web sites. Usability and Information Design, pp. 256–260 (2006)

51.

Tsang, P.S.: Mental workload. In: Karwowski, W. (ed.) International Encyclopedia of Ergonomics and Human Factors, 2nd edn., vol. 1, chap. 166. Taylor & Francis, Abingdon (2006)

52.

Tsang, P.S., Velazquez, V.L.: Diagnosticity and multidimensional subjective workload ratings. Ergonomics 39(3), 358–381 (1996)CrossRef

53.

Tsang, P.S., Vidulich, M.A.: Mental workload and situation awareness. In: Salvendy, G. (ed.) Handbook of Human Factors and Ergonomics, pp. 243–268. Wiley, Hoboken (2006)CrossRef

54.

Tullis, T.S., Stetson, J.N.: A comparison of questionnaires for assessing website usability. In: Annual Meeting of the Usability Professionals Association (2004)

55.

Vidulich, M.A., Ward Frederic, G., Schueren, J.: Using the subjective workload dominance (SWORD) technique for projective workload assessment. Hum. Factors Soc. 33(6), 677–691 (1991)CrossRef

56.

Wickens, C.D.: Multiple resources and mental workload. Hum. Factors 50(2), 449–454 (2008)CrossRef

57.

Wickens, C.D., Hollands, J.G.: Engineering Psychology and Human Performance, 3rd edn. Prentice Hall, Upper Saddle River (1999)

58.

Wilcoxon, F.: Individual comparisons by ranking methods. Biom. Bull. 1(6), 80–83 (1945). doi:10.2307/3001968 CrossRef

59.

Wilson, G.F., Eggemeier, T.F.: Mental workload measurement. In: Karwowski, W. (ed.) International Encyclopedia of Ergonomics and Human Factors, vol. 1, 2nd edn., chap. 167. Taylor & Francis, Abingdon (2006)

60.

Xie, B., Salvendy, G.: Review and reappraisal of modelling and predicting mental workload in single and multi-task environments. Work Stress 14(1), 74–99 (2000)CrossRef

61.

Young, M.S., Stanton, N.A.: Mental workload. In: Stanton, N.A., Hedge, A., Brookhuis, K., Salas, E., Hendrick, H.W. (eds.) Handbook of Human Factors and Ergonomics Methods, chap. 39, pp. 1–9. CRC Press, Boca Raton (2004)

62.

Young, M.S., Stanton, N.A.: Mental workload: theory, measurement, and application. In: Karwowski, W. (ed.) International Encyclopedia of Ergonomics and Human Factors, vol. 1, 2nd edn., pp. 818–821. Taylor & Francis, Abingdon (2006)

63.

Zhu, H., Hou, M.: Restrain mental workload with roles in HCI. In: Proceedings of Science and Technology for Humanity, pp. 387–392 (2009)

64.

Zijlstra, F.R.H.: Efficiency in work behaviour. Doctoral thesis, Delft University, The Netherlands (1993)

Title: Subjective Usability, Mental Workload Assessments and Their Impact on Objective Human Performance
Author: Luca Longo
Publisher: Springer International Publishing
Book: Human-Computer Interaction - INTERACT 2017
Print ISBN: 978-3-319-67683-8

Electronic ISBN: 978-3-319-67684-5

Copyright Year: 2017
DOI: https://doi.org/10.1007/978-3-319-67684-5_13

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"