Top

Published in:

2022 | OriginalPaper | Chapter

Understanding the Design Effects of Progress Indicators on Online Surveys

Authors : Wonyoung Park, Jaemyung Lee, Sangsu Lee

Published in: [ ] With Design: Reinventing Design Modes

Publisher: Springer Nature Singapore

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

A progress indicator (PI) is often used to inform respondents of the task completion status of online surveys. When researchers conduct online surveys, reducing the dropout rate and the participants’ cognitive loads is important for improving the surveys’ efficiency. Although many studies have investigated the effect of using PIs with online surveys, it is unclear which PI design should be used to reduce the dropout rate and the participants’ perceived task load. Moreover, even in the Web content accessibility guidelines (WCAG), which should be followed in the building of websites to improve accessibility, the design guidelines for PIs are vague compared to those for other graphic elements. We noted that PIs have various designs and then created 25 types of PIs by combining design factors, labels, and bar graphs. We conducted an online survey through mTurk and collected 1,948 participants’ data. The online survey study results showed that the PI’s label design had a significant effect on the number of items the participants answered. In addition, we found that the respondents’ mental load changed significantly according to the combination of bar graph and label designs of the PI. This study provides strong support for designing the way a PI communicates the task process without reducing the speed of progress, which encourages respondents to answer more items and decreases their mental loads in online survey environments.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter A Study of STEAM Education and Related Learning Environments in Elementary Schools: A Case Study of Elementary Schools of Fukuoka City

next chapter Hand-in-O: Exploring Possibilities of Sensing and Constraining the Gestures with its Frame to Provide Light and Sound Feedback

Becker, L.J.: Joint effect of feedback and goal setting on performance: A field study of residential energy conservation. J. Appl. Psychol. 63(4), 428 (1978)CrossRef

Buhrmester, M., Kwang, T., Gosling, S.D.: Amazon’s mechanical turk: a new source of inexpensive, yet high-quality data? (2016)

Chang, J.C., Amershi, S., Kamar, E.: Revolt: collaborative crowdsourcing for labeling machine learning datasets. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. CHI 2017, pp. 2334–2346. Association for Computing Machinery, Denver, Colorado, USA (2017). https://doi.org/10.1145/3025453.3026044, isbn: 9781450346559

Clark, J.M., Paivio, A.: Dual coding theory and education. Educ. Psychol. Rev. 3(3), 149–210 (1991). https://doi.org/10.1007/BF01320076CrossRef

Conrad, F., et al.: Impact of progress feedback on task completion: first impressions matter. In: CHI 2005 Extended Abstracts on Human Factors in Computing Systems. CHI EA 2005, pp. 1921–1924. Association for Computing Machinery, Portland, OR, USA (2005). https://doi.org/10.1145/1056808.1057057, isbn: 1595930027

Conrad, F.G., et al.: The impact of progress indicators on task completion. Interact. Comput. 22(5), 417–427 (2010)CrossRef

Couper, M.P., Traugott, M.W., Lamias, M.J.: Web survey design and administration*. Public Opin. Q. 65(2), 230–253 (2001). https://doi.org/10.1086/322199. eprint: https://academic.oup.com/poq/article-pdf/65/2/230/5307018/650230.pdf. issn: 0033–362X

Couper, M.P.: Discussion: what can CAI learn from HCI. In: Proceedings of the Seminar on New Directions in Statistical Methodology, pp. 363–377 (1994)

Crawford, S.D., Couper, M.P., Mark, Lamias, J.: Web surveys: perceptions of Burden. Soc. Sci. Comput. Rev. 19(2), 146–162 (2001). https://doi.org/10.1177/089443930101900202

Dillman, D.A., Smyth, J.D., Christian, L.M.: Internet, Phone, Mail, and Mixed-Mode Surveys: the Tailored Design Method. Wiley, Hoboken (2014)

Dillman, D.A., Tortora, R.D., Bowker, D.: Principles for constructing web surveys. J. Meetings Am. Stat. Assoc. 64, 1–16 (1998)

Drutsa, A., et al.: Crowdsourcing practice for efficient data labeling: aggregation, incremental relabeling, and pricing. In: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. SIGMOD 2020, pp. 2623–2627. Association for Computing Machinery, Portland, OR, USA (2020). https://doi.org/10.1145/3318464.3383127, isbn: 9781450367356

Egeln, L.S., Joseph, J.A.: Shopping cart abandonment in online shopping. Atlantic Mark. J. 1(1), 1 (2012)

Galesic, M.: Dropouts on the web: effects of interest and burden experienced during an online survey. J. Official Stat. 22(2), 313 (2006)

Gleicher, M., et al.: Perception of average value in multiclass scatterplots. IEEE Trans. Vis. Comput. Graph. 19(12), 2316–2325 (2013)CrossRef

Google. Google Form. https://www.google.com/forms/about/. Accessed 2020[a]

. Google material design guideline for progress indicators. https://material.io/components/progress-indicators. Accessed 2020[b]

Hart, S.G.: Nasa-task load index (NASA-TLX); 20 years later. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 50, no. 9, pp. 904–908 (2006). https://doi.org/10.1177/154193120605000909

Healey, B., Macpherson, T., Kuijten, B.: An empirical evaluation of three web survey design principles. Mark. Bull. 16 (2005)

Heerwegh, D.: Using progress indicators in web surveys. In: 59^th AAPOR conference, Phoenix, Arizona (2004)

Heerwegh, D., Loosveldt, G.: An experimental study on the effects of personalization, survey length statements, progress indicators, and survey sponsor logos in web surveys. J. Official Stat. 22(2), 191 (2006)

Kaczmirek, L.: Human-survey interaction : usability and nonresponse in online surveys. English. an enhanced book version of this work is published by Herbert von Halem Verlag. Ph.D. thesis (2008). http://www.kaczmirek.de/book2008, https://madoc.bib.uni-mannheim.de/2150/

Soomin, K., Lee, J., Gweon, G.: Comparing data from Chatbot and web surveys: effects of platform and conversational style on survey response quality. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. CHI 2019, pp. 1–12. Association for Computing Machinery, Glasgow, Scotland, UK (2019). https://doi.org/10.1145/3290605.3300316, isbn: 9781450359702

Komatsu, T., Yamada, S.: Exploring auditory information to change users’ perception of time passing as shorter. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. CHI 2020, pp. 1–12. Association for Computing Machinery, Honolulu, HI, USA (2020). https://doi.org/10.1145/3313831.3376157, isbn: 9781450367080

Kukar-Kinney, M., Close, A.G.: The determinants of consumers’ online shopping cart abandonment. J. Acad. Mark. Sci. 38(2), 240–250 (2010). https://doi.org/10.1007/s11747-009-0141-5CrossRef

Liu, M., Wronski, L.: Examining completion rates in web surveys via over 25,000 real-world surveys. Soc. Sci. Comput. Rev. 36(1), 116–124 (2018). https://doi.org/10.1177/0894439317695581CrossRef

Locke, E.A.: The motivation to work: what we know. In: Advances in motivation and achievement, vol. 10, no. 2, pp. 375–412 (1997)

Locke, E.A., Latham, G.P.: Work motivation and satisfaction: light at the end of the tunnel. Psychol. Sci. 1(4), 240–246 (1990). https://doi.org/10.1111/j.1467-9280.1990.tb00207.xCrossRef

Matejka, J., et al.: The effect of visual appearance on the performance of continuous sliders and visual analogue scales. In: CHI 2016, pp. 5421–5432. Association for Computing Machinery, San Jose, California, USA (2016). https://doi.org/10.1145/2858036.2858063, isbn: 9781450333627

Matzat, U., Snijders, C., van der Horst, W.: Effects of different types of progress indicators on dropout rates in web surveys. Soc. Psychol. 40(1), 43–52 (2009)CrossRef

Monkey, Survey. Survey Monkey question examples (2020). https://www.surveymonkey.com/mp/survey-question-examples/. Accessed 2020

Müller, H., Sedley, A.: Designing surveys for HCI research. In: CHI EA 2015, pp. 2485–2486. Association for Computing Machinery, Seoul, Republic of Korea (2015). https://doi.org/10.1145/2702613.2706683

Müller, H., Sedley, A., Ferrall-Nunge, E.: Survey research in HCI. In: Olson, J.S., Kellogg, W.A. (eds.) Ways of Knowing in HCI, pp. 229–266. Springer, New York (2014). https://doi.org/10.1007/978-1-4939-0378-8_10CrossRef

Myers, B.A.: The importance of percent-done progress indicators for computer-human interfaces. SIGCHI Bull. 16(4), 11–17 (1985). https://doi.org/10.1145/1165385.317459CrossRef

Nielsen, J.: Usability Engineering. Morgan Kaufmann, Burlington (1994)

Nielsen, J., Rolf, M.: Heuristic evaluation of user inter-faces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI 1990, pp. 249–256. Association for Computing Machinery Seattle, Washington, USA (1990).https://doi.org/10.1145/97243.97281, isbn: 0201509326

Oviatt, S.: Human-centered design meets cognitive load theory: designing interfaces that help people think. In: Proceedings of the 14thACM International Conference on Multimedia. MM 2006, pp. 871–880. Association for Computing Machinery, Santa Barbara, CA, USA (2006). https://doi.org/10.1145/1180639.1180831, isbn: 1595934472

Paolacci, G., Chandler, J., Ipeirotis, P.G.: Running experiments on amazon mechanical Turk. Judgment Decis. Making 5(5), 411–419 (2010)CrossRef

Ross, J., et al.: Who are the Crowdworkers? Shifting demographics in mechanical turk. In: CHI 2010 Extended Abstracts on Human Factors in Computing Systems. CHI EA 2010, pp. 2863–2872. Association for Computing Machinery, Atlanta, Georgia, USA (2010). https://doi.org/10.1145/1753846.1753873, isbn: 9781605589305

Sarraf, S., Tukibayeva, M.: Survey page length and progress indicators: what are their relationships to item nonresponse? New Dir. Inst. Res. 161, 83–97 (2014)

Smyth, J.D., et al.: Effects of using visual design principles to group response options in web surveys (2006)

Strang, H.R., Lawrence, E.C., Fowler, P.C.: Effects of assigned goal level and knowledge of results on arithmetic computation: a laboratory study. J. Appl. Psychol. 63(4), 446 (1978)CrossRef

Streeter, S.B., et al.: Patient and plan characteristics affecting abandonment of oral oncolytic prescriptions. J. Oncol. Pract. 7(3S), 46s–51s (2011). https://doi.org/10.1200/JOP.2011.000316, PMID: 21886519

Tijdens, K.: Dropout rates and response times of an occupation search tree in a web survey. J. Official Stat. 30(1), 23–43 (2014). https://doi.org/10.2478/jos-2014-0002, https://content.sciendo.com/view/journals/jos/30/1/article-p23.xml

Tourangeau, R., Conrad, F.G., Couper, M.P.: The Science of Web Surveys. Oxford University Press, Oxford (2013)CrossRef

Tukibayeva, M., Sarraf, S.: The relationships between survey page length, progress indicators, and item completion rates. Am. Educ. Res. Assoc. Annu. Meeting (2012)

Villar, A., Callegaro, M., Yang, Y.: Where am I? a meta-analysis of experiments on the effects of progress indicators for web surveys. Soc. Sci. Comput. Rev. 31(6), 744–762 (2013). https://doi.org/10.1177/0894439313497468

W3C. Web Content Accessibility Guidelines. https://www.w3.org/WAI/standards-guidelines/wcag/. Accessed 2020

Yentes, R.D., et al.: Effects of survey progress bars on data quality and enjoyment. In: 27th annual meeting of the Society for Industrial and Organizational Psychology (2012)

Yentes, R.D., et al.: Attention and data quality in online surveys: the role of survey length, progress bars, and time disclosure (2015)

Zhang, J., Wu, X., Sheng, V.S.: Learning from crowdsourced labeled data: a survey. Artif. Intell. Rev. 46(4), 543–576 (2016). https://doi.org/10.1007/s10462-016-9491-9CrossRef

Title: Understanding the Design Effects of Progress Indicators on Online Surveys
Authors: Wonyoung Park
Jaemyung Lee
Sangsu Lee
Publisher: Springer Nature Singapore
Book: [ ] With Design: Reinventing Design Modes
Print ISBN: 978-981-19-4471-0

Electronic ISBN: 978-981-19-4472-7

Copyright Year: 2022
DOI: https://doi.org/10.1007/978-981-19-4472-7_92

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"