Top

Published in:

2021 | OriginalPaper | Chapter

Remote Evaluation in Universal Design Using Video Conferencing Systems During the COVID-19 Pandemic

Authors : Joschua Thomas Simon-Liedtke, Way Kiat Bong, Trenton Schulz, Kristin Skeide Fuglerud

Published in: Universal Access in Human-Computer Interaction. Design Methods and User Experience

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Usability and accessibility evaluations with diverse users are an essential part of an iterative universal design process for digital solutions. The COVID-19 pandemic has made it difficult to run traditional local evaluations due to social distancing restrictions to reduce infections. Remote synchronous and asynchronous evaluation methods may be a solution if it can be used by various user groups, including people with impairments. Incorporating video conferencing systems into traditional remote evaluations can be a valuable supplement to—or, given the current situation, alternative to—traditional usability and accessibility evaluations. We present a protocol for remote formative usability evaluations designed with accessibility in mind. The protocol explains how to prepare for an evaluation and its technical setup, how to conduct synchronous evaluations using video conferencing systems, and how to debrief and analyze the collected data. We tested this protocol in a pilot study for an ongoing project where we ran both synchronous and asynchronous remote evaluations. In our pilot, the synchronous evaluation using video conferencing systems provided opportunities for richer qualitative data than the asynchronous evaluation as we ran it. The findings from the pilot study indicate that the protocol is feasible and can be used when having participants with diverse abilities or impairments. We also provide suggestions for others that wish to adapt the protocol.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Methods of Usability Testing for Users with Cognitive Impairments

next chapter A Multidisciplinary User-Centered Approach to Designing an Information Platform for Accessible Tourism: Understanding User Needs and Motivations

Andreasen, M.S., Nielsen, H.V., Schrder, S.O., Stage, J.: What happened to remote usability testing? An empirical study of three methods. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2007, pp. 1405–1414. Association for Computing Machinery, San Jose, April 2007. https://doi.org/10.1145/1240624.1240838

Begnum, M.E.N.: Universal design of ICT: a historical journey from specialized adaptations towards designing for diversity. In: Antona, M., Stephanidis, C. (eds.) HCII 2020. LNCS, vol. 12188, pp. 3–18. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49282-3_1CrossRef

Bonacin, R., Dos Reis, J.C., Baranauskas, M.C.C.: Universal participatory design: achievements and challenges. SBC J. Interact. Syst. 10(1), 2–16 (2019)

Boren, T., Ramey, J.: Thinking aloud: reconciling theory and practice. IEEE Trans. Prof. Commun. 43(3), 261–278 (2000). https://doi.org/10.1109/47.867942CrossRef

Brooke, J.: SUS - a quick and dirty usability scale. In: Jordan, P.W., Thomas, B., Werdmeester, B.A., McClelland, I.L. (eds.) Usability Evaluation in Industry, 1st edn., pp. 189–194. Tayler & Francis, London (1996)

Brush, A.B., Ames, M., Davis, J.: A comparison of synchronous remote and local usability studies for an expert interface. In: CHI 2004 Extended Abstracts on Human Factors in Computing Systems, CHI EA 2004, pp. 1179–1182. Association for Computing Machinery, Vienna, April 2004. https://doi.org/10.1145/985921.986018

Bruun, A., Gull, P., Hofmeister, L., Stage, J.: Let your users do the testing: a comparison of three remote asynchronous usability testing methods. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2009, pp. 1619–1628. Association for Computing Machinery, Boston, April 2009. https://doi.org/10.1145/1518701.1518948

Castillo, J.C., Hartson, H.R., Hix, D.: Remote usability evaluation: can users report their own critical incidents? In: CHI 1998 Conference Summary on Human Factors in Computing Systems, CHI 1998, pp. 253–254. Association for Computing Machinery, Los Angeles, April 1998. https://doi.org/10.1145/286498.286736

World Leaders in Research-Based User Experience: Formative vs. Summative Evaluations, July 2019. https://www.nngroup.com/articles/formative-vs-summative-evaluations/. Accessed 2 Feb 2021

10.

Fischer, B., Peine, A., stlund, B.: The importance of user involvement: a systematic review of involving older users in technology design. Gerontologist 60(7), e513–e523 (2020). https://doi.org/10.1093/geront/gnz163. https://academic.oup.com/gerontologist/article/60/7/e513/5644100

11.

Fuglerud, K.S.: Inclusive design of ICT: the challenge of diversity. Ph.D. thesis, University of Oslo, Faculty of humanities (2014). https://doi.org/10.13140/2.1.4471.5844. http://publications.nr.no/1418159224/Fuglerud_2014_PhD-Thesis_Inclusive-design-of-ICT.pdf

12.

Fuglerud, K.S., Halbach, T., Tjøstheim, I.: Cost-benefit analysis of universal design, January 2015

13.

Giannoumis, G.A., Stein, M.A.: Conceptualizing universal design for the information society through a universal human rights lens. Int. Hum. Rights Law Rev. 8(1), 38–66 (2019). https://doi.org/10.1163/22131035-00801006. https://brill.com/view/journals/hrlr/8/1/article-p38_38.xml

14.

Halbach, T., Fuglerud, K.S.: On assessing the costs and benefits of universal design of ICT. Stud. Health Technol. Inf. 229, 662–672 (2016)

15.

Hartson, H.R., Castillo, J.C., Kelso, J., Neale, W.C.: Remote evaluation: the network as an extension of the usability laboratory. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 228–235 (1996)

16.

Hertzum, M., Jacobsen, N.E.: The evaluator effect: a chilling fact about usability evaluation methods. Int. J. Hum.-Comput. Interact. 13(4), 421–443 (2001). https://doi.org/10.1207/S15327590IJHC1304_05CrossRef

17.

Janson, A.L., Moen, A., Fuglerud, K.S.: Design of the CAPABLE prototype: preliminary results of citizen expectations. In: Norwegian Centre for E-health Research og EHiN Research (ed.) Linkping Electronic Conference Proceedings, p. 181. Linkping Electronic Conference Proceedings, No (2019). http://www.ep.liu.se/ecp/161/ecp19161.pdf

18.

Janson, A.L., Moen, A., Fuglerud, K.S.: Design of the capable health empowerment tool: citizens’ needs and expectations. Stud. Health Technol. Inf. 270, 926–930 (2020)

19.

McFadden, E., Hager, D.R., Elie, C.J., Blackwell, J.M.: Remote usability evaluation: overview and case studies. Int. J. Hum.-comput. Interact. 14(3–4), 489–502 (2002)CrossRef

20.

McLaughlin, A.C., et al.: Evaluating medical devices remotely: current methods and potential innovations. Hum. Factors 62(7), 1041–1060 (2020)CrossRef

21.

Miao, M., Pham, H.A., Friebe, J., Weber, G.: Contrasting usability evaluation methods with blind users. Univ. Access Inf. Soci. 15(1), 63–76 (2016). https://doi.org/10.1007/s10209-014-0378-8CrossRef

22.

Microsoft: Accessibility overview of Microsoft Teams (2020). https://support.microsoft.com/en-us/office/accessibility-overview-of-microsoft-teams-2d4009e7-1300-4766-87e8-7a217496c3d5

23.

Miljverndepartementet: T-1468 B/E Universell utforming, November 2007. https://www.regjeringen.no/no/dokumenter/t-1468-universell-utforming/id493083/

24.

Nielsen, J.: Demonstrate thinking aloud by showing users a video (2014)

25.

Øyvann, S.: Vipps har vunnet Innovasjonsprisen for universell utforming—Computerworld, January 2021. https://www.cw.no/artikkel/utvikling/vipps-har-vunnet-innovasjonsprisen-universell-utforming

26.

Persson, H., Ohlsson, K., Petersén, S., Jonsäll, A.: Unexploited resources in interaction design for universal access: people with impairments as a resource for interaction designers. In: Stephanidis, C. (ed.) UAHCI 2009. LNCS, vol. 5614, pp. 145–153. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02707-9_16CrossRef

27.

Petrie, H., Hamilton, F., King, N., Pavan, P.: Remote usability evaluations with disabled people. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2006, pp. 1133–1141. Association for Computing Machinery, Montréal, April 2006. https://doi.org/10.1145/1124772.1124942

28.

Power, C., Petrie, H.: Working with participants. In: Yesilada, Y., Harper, S. (eds.) Web Accessibility. HIS, pp. 153–168. Springer, London (2019). https://doi.org/10.1007/978-1-4471-7440-0_9CrossRef

29.

Richter, A.: Locked-down digital work. Int. J. Inf. Manage. 55 (2020). https://doi.org/10.1016/j.ijinfomgt.2020.102157. http://www.sciencedirect.com/science/article/pii/S0268401220308422

30.

Rickly, J., Halpern, N., Hansen, M., McCabe, S., Fellenor, J.: Covid-19: the effects of isolation and social distancing on people with vision impairment, September 2020. https://doi.org/10.17639/nott.7074. https://rdmc.nottingham.ac.uk/handle/internal/8608. Accessed 28 Jan 2021

31.

Røssvoll, T.H., Fuglerud, K.S.: Best practice for efficient development of inclusive ICT. In: Stephanidis, C., Antona, M. (eds.) UAHCI 2013. LNCS, vol. 8009, pp. 97–106. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39188-0_11CrossRef

32.

Sauro, J.: A practical guide to measuring usability: 72 answers to the most common questions about quantifying the usability of websites and software. Measuring Usability LCC, Denver (2010)

33.

Schmutz, S., Sonderegger, A., Sauer, J.: Implementing recommendations from web accessibility guidelines: would they also provide benefits to nondisabled users. Hum. Factors 58(4), 611–629 (2016). https://doi.org/10.1177/0018720816640962CrossRef

34.

Schmutz, S., Sonderegger, A., Sauer, J.: Implementing recommendations from web accessibility guidelines: a comparative study of nondisabled users and users with visual impairments. Hum. Factors 59(6), 956–972 (2017). https://doi.org/10.1177/0018720817708397CrossRef

35.

Schnepp, J., Shiver, B.: Improving deaf accessibility in remote usability testing. In: The Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2011, pp. 255–256. Association for Computing Machinery, Dundee, October 2011. https://doi.org/10.1145/2049536.2049594

36.

Sova, D.H., Nielsen, J.: 234 Tips and tricks for recruiting users as participants in usability studies. Technical report, Nilsen Norman Group, January 2003. https://www.nngroup.com/reports/how-to-recruit-participants-usability-studies/

37.

Venable, J., Pries-Heje, J., Baskerville, R.: FEDS: a framework for evaluation in design science research. Eur. J. Inf. Syst. 25(1), 77–89 (2016). https://doi.org/10.1057/ejis.2014.36CrossRef

38.

World Wide Web Consortium (W3C): Web content accessibility guidelines (WCAG) 2.0, December 2008. https://www.w3.org/TR/WCAG20/. Accessed 17 Sept 2020

39.

World Wide Web Consortium (W3C): Web content accessibility guidelines (WCAG) 2.1, June 2018. https://www.w3.org/TR/WCAG21/. Accessed 17 Sept 2020

40.

Zoom Video Communications: Meetings for Everyone (2019). https://zoom.us/accessibility

Title: Remote Evaluation in Universal Design Using Video Conferencing Systems During the COVID-19 Pandemic
Authors: Joschua Thomas Simon-Liedtke
Way Kiat Bong
Trenton Schulz
Kristin Skeide Fuglerud
Publisher: Springer International Publishing
Book: Universal Access in Human-Computer Interaction. Design Methods and User Experience
Print ISBN: 978-3-030-78091-3

Electronic ISBN: 978-3-030-78092-0

Copyright Year: 2021
DOI: https://doi.org/10.1007/978-3-030-78092-0_8

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"