Skip to main content

2015 | OriginalPaper | Buchkapitel

Comparing Concurrent and Retrospective Verbal Protocols for Blind and Sighted Users

verfasst von : Andreas Savva, Helen Petrie, Christopher Power

Erschienen in: Human-Computer Interaction – INTERACT 2015

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Verbal protocols are widely used in user studies for evaluating websites. This study investigated the effectiveness and efficiency of concurrent and retrospective verbal protocols (CVP and RVP) for both blind and sighted participants, as well as participant workload and attitudes towards these methods. Eight blind and eight sighted participants undertook both protocols in a website evaluation. RVP was more effective as measured by problems encountered for both groups, although it was no more efficient than CVP. The severity of problems identified by both protocols was equivalent. As measured on the NASA TLX, participants found RVP found more demanding than CVP. Sighted participants found rating problems during CVP more disruptive than blind participants. These results show that RVP is a more useful protocol for practitioners and researchers even though it takes more time and is more demanding for participants. It is equally applicable for both blind and sighted participants.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Boren, T., Ramey, J.: Thinking aloud: reconciling theory and practice. IEEE Trans. Prof. Commun. 43, 261–278 (2000)CrossRef Boren, T., Ramey, J.: Thinking aloud: reconciling theory and practice. IEEE Trans. Prof. Commun. 43, 261–278 (2000)CrossRef
2.
Zurück zum Zitat Borsci, S., Federici, S.: The partial concurrent thinking aloud: a new usability evaluation technique for blind users. Assistive technology from adapted equipment to inclusive environments—AAATE, vol. 25, pp. 421–425 (2009) Borsci, S., Federici, S.: The partial concurrent thinking aloud: a new usability evaluation technique for blind users. Assistive technology from adapted equipment to inclusive environments—AAATE, vol. 25, pp. 421–425 (2009)
3.
Zurück zum Zitat Bowers, V.A., Snyder, H.L.: Concurrent versus retrospective verbal protocol verbal for comparing window usability. In: Human Factors and Ergonomics Society Annual Meeting, pp. 1270–1274 (1990) Bowers, V.A., Snyder, H.L.: Concurrent versus retrospective verbal protocol verbal for comparing window usability. In: Human Factors and Ergonomics Society Annual Meeting, pp. 1270–1274 (1990)
4.
Zurück zum Zitat Chandrashekar, S., Stockman, T., Fels, D., Benedyk, R.: Using think aloud protocol with blind users: a case for inclusive usability evaluation methods. In: 8th international ACM SIGACCESS Conference on Computers and Accessibility, pp. 251–252. ACM, New York (2006) Chandrashekar, S., Stockman, T., Fels, D., Benedyk, R.: Using think aloud protocol with blind users: a case for inclusive usability evaluation methods. In: 8th international ACM SIGACCESS Conference on Computers and Accessibility, pp. 251–252. ACM, New York (2006)
5.
Zurück zum Zitat Disability Rights Commission.: The Web: Access and inclusion for disabled people. The Stationery Office, London (2004) Disability Rights Commission.: The Web: Access and inclusion for disabled people. The Stationery Office, London (2004)
6.
Zurück zum Zitat Dumas, J.F., Redish, J.C.: A practical Guide to Usability Testing. Greenwood Publishing Group Inc., Westpost, CT, USA (1993) Dumas, J.F., Redish, J.C.: A practical Guide to Usability Testing. Greenwood Publishing Group Inc., Westpost, CT, USA (1993)
7.
Zurück zum Zitat Ericsson, K.A., Simon, H.A.: Protocol Analysis. MIT-press, Cambridge (1984) Ericsson, K.A., Simon, H.A.: Protocol Analysis. MIT-press, Cambridge (1984)
8.
Zurück zum Zitat Harrison, C., Petrie, H.: Severity of usability and accessibility problems in eCommerce and eGovernment websites. In: BryanKinns, N., Blandfor, A., Curzon, P., Nigay, L. (Eds.), People and Computers XX - Engage. pp. 255–262, Godalming: Springer-Verlag London Ltd (2007) Harrison, C., Petrie, H.: Severity of usability and accessibility problems in eCommerce and eGovernment websites. In: BryanKinns, N., Blandfor, A., Curzon, P., Nigay, L. (Eds.), People and Computers XX - Engage. pp. 255–262, Godalming: Springer-Verlag London Ltd (2007)
9.
Zurück zum Zitat Krahmer, E., Ummelen, N.: Thinking about thinking aloud: a comparison of two verbal protocols for usability testing. IEEE Trans. Prof. Commun. 47, 105–117 (2004)CrossRefMATH Krahmer, E., Ummelen, N.: Thinking about thinking aloud: a comparison of two verbal protocols for usability testing. IEEE Trans. Prof. Commun. 47, 105–117 (2004)CrossRefMATH
10.
Zurück zum Zitat Kuusela, H., Paul, P.: A comparison of concurrent and retrospective verbal protocol analysis. Am. J. Psychol. 113(3), 387–404 (2000)CrossRefMATH Kuusela, H., Paul, P.: A comparison of concurrent and retrospective verbal protocol analysis. Am. J. Psychol. 113(3), 387–404 (2000)CrossRefMATH
11.
Zurück zum Zitat Lavery, D., Cockton, G., Atkinson, M.P.: Comparison of evaluation methods using structured usability problem reports. Behav. Inf. Technol. 16, 246–266 (1997)CrossRefMATH Lavery, D., Cockton, G., Atkinson, M.P.: Comparison of evaluation methods using structured usability problem reports. Behav. Inf. Technol. 16, 246–266 (1997)CrossRefMATH
12.
Zurück zum Zitat Lazar, J., Olalere, A., Wentz, B.: Investigating the accessibility and usability of job application web sites for blind users. J. Usability Stud. 7, 68–87 (2012) Lazar, J., Olalere, A., Wentz, B.: Investigating the accessibility and usability of job application web sites for blind users. J. Usability Stud. 7, 68–87 (2012)
13.
Zurück zum Zitat Lewis: Using the Thinking Aloud Method in Cognitive Interface Design, Technical report. IBM Research Center (1982) Lewis: Using the Thinking Aloud Method in Cognitive Interface Design, Technical report. IBM Research Center (1982)
15.
Zurück zum Zitat Nielsen, J.: Usability Engineering. Elsevier (1994) Nielsen, J.: Usability Engineering. Elsevier (1994)
16.
Zurück zum Zitat Nørgaard, M., Hornbæk, K.: What do usability evaluators do in practice?: an explorative study of think-aloud testing. In: 6th Design Interactive systems conference, pp. 209–218, ACM, New York (2006) Nørgaard, M., Hornbæk, K.: What do usability evaluators do in practice?: an explorative study of think-aloud testing. In: 6th Design Interactive systems conference, pp. 209–218, ACM, New York (2006)
17.
Zurück zum Zitat Ohnemus, K.R., Biers, D.W.: Retrospective versus concurrent thinking-out-loud in usability testing. In: Human Factors and Ergonomics Society Annual Meeting, pp. 1127–1131 (1993) Ohnemus, K.R., Biers, D.W.: Retrospective versus concurrent thinking-out-loud in usability testing. In: Human Factors and Ergonomics Society Annual Meeting, pp. 1127–1131 (1993)
18.
Zurück zum Zitat Olmsted-Hawala, E.L., Murphy, E.D., Hawala, S., Ashenfelter, K.T.: Think-aloud protocols: a comparison of three think-aloud protocols for use in testing data-dissemination web sites for usability. In: SIGCHI Conference on Human Factors in Computing Systems, pp. 2381–2390. ACM, New York (2010) Olmsted-Hawala, E.L., Murphy, E.D., Hawala, S., Ashenfelter, K.T.: Think-aloud protocols: a comparison of three think-aloud protocols for use in testing data-dissemination web sites for usability. In: SIGCHI Conference on Human Factors in Computing Systems, pp. 2381–2390. ACM, New York (2010)
19.
Zurück zum Zitat Petrie, H., Kheir, O.: The relationship between accessibility and usability of websites. In: SIGHI conference on Human Factors in Computing Systems, pp. 397–406. ACM, New York (2007) Petrie, H., Kheir, O.: The relationship between accessibility and usability of websites. In: SIGHI conference on Human Factors in Computing Systems, pp. 397–406. ACM, New York (2007)
20.
Zurück zum Zitat Power, C., Freire, A., Petrie, H., Swallow, D.: Guidelines are only half the story: accessibility problems encountered by blind users on the web. In: SIGHI Conference on Human Factors in Computing Systems, pp. 433–442, ACM, New York (2012) Power, C., Freire, A., Petrie, H., Swallow, D.: Guidelines are only half the story: accessibility problems encountered by blind users on the web. In: SIGHI Conference on Human Factors in Computing Systems, pp. 433–442, ACM, New York (2012)
21.
Zurück zum Zitat Rogers, Y., Sharp, H., Preece, J.: Interaction Design: Beyond Human-Computer Interaction. John Wiley & Sons, Hoboken (2011) Rogers, Y., Sharp, H., Preece, J.: Interaction Design: Beyond Human-Computer Interaction. John Wiley & Sons, Hoboken (2011)
22.
Zurück zum Zitat Rømen, D., Svanæs, D.: Validating WCAG versions 1.0 and 2.0 through usability testing with disabled users. Univ. Access Inf. Soc. 11, 375–385 (2012)CrossRef Rømen, D., Svanæs, D.: Validating WCAG versions 1.0 and 2.0 through usability testing with disabled users. Univ. Access Inf. Soc. 11, 375–385 (2012)CrossRef
23.
Zurück zum Zitat Shi, Q: A field study of the relationship and communication between Chinese evaluators and users in thinking aloud usability tests. In: 5th Nordic conference on Human-Computer Interaction: Building Bridges, pp. 344–352. ACM, New York (2008) Shi, Q: A field study of the relationship and communication between Chinese evaluators and users in thinking aloud usability tests. In: 5th Nordic conference on Human-Computer Interaction: Building Bridges, pp. 344–352. ACM, New York (2008)
24.
Zurück zum Zitat Strain, P., Shaikh, A.D., Boardman, R.: Thinking but not seeing: think-aloud for non-sighted users. In: CHI 2007 Extended Abstracts on Human Factors in Computing Systems, pp. 1851–1856. ACM, New York (2007) Strain, P., Shaikh, A.D., Boardman, R.: Thinking but not seeing: think-aloud for non-sighted users. In: CHI 2007 Extended Abstracts on Human Factors in Computing Systems, pp. 1851–1856. ACM, New York (2007)
25.
Zurück zum Zitat van den Haak, M.J., De Jong, M.D., Schellens, P.J.: Employing think-aloud protocols and constructive interaction to test the usability of online library catalogues: a methodological comparison. Gov. Inf. Q. 16, 1153–1170 (2004) van den Haak, M.J., De Jong, M.D., Schellens, P.J.: Employing think-aloud protocols and constructive interaction to test the usability of online library catalogues: a methodological comparison. Gov. Inf. Q. 16, 1153–1170 (2004)
26.
Zurück zum Zitat van den Haak, M.J., De Jong, M.D., Schellens, P.J.: Evaluating municipal websites: A methodological comparison of three think-aloud variants. Gov. Inf. Q. 26, 193–202 (2009)CrossRef van den Haak, M.J., De Jong, M.D., Schellens, P.J.: Evaluating municipal websites: A methodological comparison of three think-aloud variants. Gov. Inf. Q. 26, 193–202 (2009)CrossRef
27.
Zurück zum Zitat van den Haak, M.J., De Jong, M.D., Schellens, P.J.: Evaluation of an informational web site: three variants of the think-aloud method compared. Tech. Commun. 54, 58–71 (2007)MATH van den Haak, M.J., De Jong, M.D., Schellens, P.J.: Evaluation of an informational web site: three variants of the think-aloud method compared. Tech. Commun. 54, 58–71 (2007)MATH
28.
Zurück zum Zitat van den Haak, M.J., De Jong, M.D., Schellens, P.J.: Retrospective vs concurrent think-aloud protocols: testing the usability of an online library catalogue. Behav. Inf. Technol. 22, 339–351 (2003)CrossRef van den Haak, M.J., De Jong, M.D., Schellens, P.J.: Retrospective vs concurrent think-aloud protocols: testing the usability of an online library catalogue. Behav. Inf. Technol. 22, 339–351 (2003)CrossRef
29.
Zurück zum Zitat van den Haak, M.J., De Jong, M.D.: Exploring two methods of usability testing: concurrent versus retrospective think-aloud protocols. In: IEEE International Professional Communication Conference. pp. 285–287 (2003) van den Haak, M.J., De Jong, M.D.: Exploring two methods of usability testing: concurrent versus retrospective think-aloud protocols. In: IEEE International Professional Communication Conference. pp. 285–287 (2003)
30.
Zurück zum Zitat Wright, R.B., Converse, S.A.: Method bias and concurrent verbal protocol in software usability testing. In: Human Factors and Ergonomics Society Annual Meeting, pp. 1220–1224 (1992) Wright, R.B., Converse, S.A.: Method bias and concurrent verbal protocol in software usability testing. In: Human Factors and Ergonomics Society Annual Meeting, pp. 1220–1224 (1992)
Metadaten
Titel
Comparing Concurrent and Retrospective Verbal Protocols for Blind and Sighted Users
verfasst von
Andreas Savva
Helen Petrie
Christopher Power
Copyright-Jahr
2015
DOI
https://doi.org/10.1007/978-3-319-22701-6_5

Neuer Inhalt