Aliaksei Miniukovich
Michele Scaltritti
ABSTRACT
Effortless reading remains an issue for many Web users, despite a large number of readability guidelines available to designers. This paper presents a study of manual and automatic use of 39 readability guidelines in webpage evaluation. The study collected the ground-truth readability for a set of 50 webpages using eye-tracking with average and dyslexic readers (n = 79). It then matched the ground truth against human-based (n = 35) and automatic evaluations. The results validated 22 guidelines as being connected to readability. The comparison between human-based and automatic results also revealed a complex framework: algorithms were better or as good as human experts at evaluating webpages on specific guidelines - particularly those about low-level features of webpage legibility and text formatting. However, multiple guidelines still required a human judgment related to understanding and interpreting webpage content. These results contribute a guideline categorization laying the ground for future design evaluation methods.
- Arditi, A. and Cho, J. Letter case and text legibility in normal and low vision. Vision research, 47, 19 (2007), 2499--2505.Google Scholar
- Aziz, F. A. and Husni, H. Interaction Design for Dyslexic Children Reading Application: A Guideline. In Knowledge Management International Conference (KMICe) ( 2012), 682--686.Google Scholar
- Bates, D., Maechler, M., Bolker, B. & Walker, S. Fitting linear mixed-e ects models using lme4. Journal of Statistical Software, 67 (2014), 1--48.Google Scholar
- Brown, C. M. Human-computer interface design guidelines. Intellect Books, 1999.Google ScholarDigital Library
- Cassim, R., Talcott, J. B., and Moores, E. Adults with dyslexia demonstrate large effects of crowding and detrimental effects of distractors in a visual tilt discrimination task. PloS one, 9, 9 (2014).Google Scholar
- Chorianopoulos, K. User interface design principles for interactive television applications. Intl. Journal of Human--Computer Interaction, 24, 6 (2008), 556573.Google ScholarCross Ref
- Dingli, A., & Cassar, S. An intelligent framework for website usability. Advances in Human-Computer Interaction, 5 (2014), 1--13. Google ScholarDigital Library
- Dumas, J. S. and Salzman, M. C. Usability assessment methods. Reviews of human factors and ergonomics, 2, 1 (2006), 109--140.Google Scholar
- Flesch, R. A new readability yardstick. Journal of applied psychology, 32, 3, 1948.Google Scholar
- Friedman, M. G. and Bryen, D. N. Web accessibility design recommendations for people with cognitive disabilities. Technology and Disability, 19, 4 (2007), 205--212.Google Scholar
- Graesser, A. C., McNamara, D. S., Louwerse, M. M., and Cai, Z. Coh-Metrix: Analysis of text on cohesion and language. Behavior research methods, instruments, & computers, 36, 2 (2004), 193--202.Google Scholar
- Grigera, J., Garrido, A., Rivero, J. M., & Rossi, G. Automatic detection of usability smells in web applications. International Journal of Human-Computer Studies, 97 (2017), 129--148.Google ScholarCross Ref
- Henninger, S. A methodology and tools for applying context-specific usability guidelines to interface design. Interacting with computers, 12, 3 (2000), 225--243.Google Scholar
- Henry, S. L. Developing text customisation functionality requirements of PDF reader and other user agents. In International Conference on Computers for Handicapped Persons ( 2012), Springer, 602--609. Google ScholarDigital Library
- Höök, K. and Löwgren, J. Strong concepts: Intermediate-level knowledge in interaction design research. ACM Transactions on Computer-Human Interaction (TOCHI), 19, 3 (2012), 1--18. Google ScholarDigital Library
- Ivory, M. Y. and Hearst, M. A. Improving web site design. Internet Computing, 6, 2 (2002), 56--63. Google ScholarDigital Library
- Jeffries, R. and Desurvire, H. Usability testing vs. heuristic evaluation: was there a contest? ACM SIGCHI Bulleti, 24, 4 (1992), 39--41. Google ScholarDigital Library
- Ji, Y. G., Park, J. H., Lee, C., & Yun, M. H. A usability checklist for the usability evaluation of mobile phone user interface. International journal of humancomputer interaction, 20, 3 (2006), 207--231.Google ScholarCross Ref
- Jo, J., Kim, B., & Seo, J. EyeBookmark: Assisting recovery from interruption during reading. In the 33rd Annual ACM Conference on Human Factors in Computing Systems ( 2015), ACM, 2963--2966. Google ScholarDigital Library
- Johannessen, G. H. J., & Hornbæk, K. Must evaluation methods be about usability? Devising and assessing the utility inspection method. Behaviour & Information Technology, 33, 2 (2014), 195--206. Google ScholarDigital Library
- Kember, P. and Varley, D. The legibility and readability of a visual display unit at threshold. Ergonomics, 30, 6 (1987), 925--931.Google ScholarCross Ref
- Kirkwood, T. B., Bond, J., May, C., McKeith, I., and Teh, M. M. Foresight mental capital and wellbeing project. The Government Office for Science, 2008.Google Scholar
- Kurniawan, S. and Zaphiris, P. Research-Derived Web Design Guidelines for Older People. In 7th ACM Conference on Computers and Assessibility ( 2005), ACM, 129--135. Google ScholarDigital Library
- Lanzilotti, R., Ardito, C., Costabile, M. F., and De Angeli, A. Do patterns help novice evaluators? A comparative study. International journal of humancomputer studies, 69, 1 (2011), 52--69. Google ScholarDigital Library
- Large, A., Beheshti, J., Nesset, V., & Bowler, L. Web portal design guidelines as identified by children through the processes of design and evaluation. Proceedings of the American Society for information Science and Technology, 43, 1 (2006), 1--23.Google ScholarCross Ref
- Mi, N., Cavuoto, L. A., Benson, K., Smith-Jackson, T., & Nussbaum, M. A. A heuristic checklist for an accessible smartphone interface design. Universal access in the information society, 13, 4 (2014), 351--365. Google ScholarDigital Library
- Miniukovich, A., Sulpizio, S., & De Angeli, A. Visual complexity of graphical user interfaces. In the 2018 International Conference on Advanced Visual Interfaces ( 2018), ACM, 1--9. Google ScholarDigital Library
- Miniukovich, A. and De Angeli, A. Pick Me! Getting Noticed on Google Play. In CHI'16 ( 2016), ACM. Google ScholarDigital Library
- Miniukovich, A. and De Angeli, A. Quantification of Interface Visual Complexity. In the 2014 International Working Conference on Advanced Visual Interfaces (Como 2014a), ACM, 153--160. Google ScholarDigital Library
- Miniukovich, A., De Angeli, A., Sulpizio, S., and Venuti, P. Design Guidelines for Web Readability. In the 2017 Conference on Designing Interactive Systems ( 2017), ACM, 285--296. Google ScholarDigital Library
- Montero, F., Vanderdonckt, J., & Lozano, M. Quality models for automated evaluation of web sites usability and accessibility. In International COST294 workshop on User Interface Quality Models ( 2005).Google Scholar
- Morrison, R.E. and Inhoff, A.-W. Visual factors and eye movements in reading. Visible Language, 15, 2 (1981), 129--146.Google Scholar
- Nebeling, M., Speicher, M., & Norrie, M. W3touch: metrics-based web page adaptation for touch. In the SIGCHI Conference on Human Factors in Computing Systems ( 2013), ACM, 2311--2320. Google ScholarDigital Library
- Petrie, H. and Kheir, O. The relationship between accessibility and usability of websites. In SIGCHI conference on Human factors in computing systems ( 2007), ACM, 397--406. Google ScholarDigital Library
- Rayner, K. Eye movements and the perceptual span in beginning and skilled readers. Journal of Experimental Child Psychology, 41, 2, 211--236.Google Scholar
- Reinecke, K., Yeh, T., Miratrix, L., Mardiko, R., Zhao, Y., and Liu, J., Gajos, K. Z. Predicting users' first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness. In CHI (Paris 2013), ACM, 2049--2058. Google ScholarDigital Library
- Rello, L. and Baeza-Yates, R. Evaluation of Dyswebxia: A reading app designed for people with dyslexia. In the 11th Web for All Conference ( 2014), ACM. Google ScholarDigital Library
- Rello, L., Kanvinde, G., and Baeza-Yates, R. Layout guidelines for web text and a web service to improve accessibility for dyslexics. In International crossdisciplinary conference on web accessibility ( 2012), ACM. Google ScholarDigital Library
- Rello, L., Pielot, M., and Marcos, M. C. Make it big! The effect of font size and line spacing on online readability. In CHI'16 ( 2016), ACM, 3637--3648. Google ScholarDigital Library
- Rosenbaum, S. The future of usability evaluation: increasing impact on value. In Law, E. et al., eds., Maturing usability. Springer, 2008.Google ScholarCross Ref
- Rosenholtz, R., Li, Y., and Nakano, L. Measuring visual clutter. Journal of vision, 7, 2 (2007), 1--22.Google Scholar
- Santana, V. F., de Oliveira, R., Almeida, L. D. A., and Baranauskas, M. C. C. Web accessibility and people with dyslexia: a survey on techniques and guidelines. In International Cross-Disciplinary Conference on Web Accessibility ( 2012), ACM. Google ScholarDigital Library
- Santana, V. F., Oliveira, R., Almeida, L. D. A., and Ito, M. Firefixia: An accessibility web browser customization toolbar for people with dyslexia. In 10th International Cross-Disciplinary Conference on Web Accessibility ( 2013), ACM. Google ScholarDigital Library
- Schiefele, U., & Krapp, A. Topic interest and free recall of expository text. Learning and individual differences, 8, 2 (1996), 141--160.Google Scholar
- Slattery, T. J. Eye movements: From psycholinguistics to font design. In Dyson, M. C. and Yuen, C. Y., eds., Digital Fonts and Reading. World Scientific, 2016.Google ScholarCross Ref
- Sperling, A. J., Lu, Z. L., Manis, F. R., and Seidenberg, M. S. Motion-perception deficits and reading impairment: it's the noise, not the motion. Psychological Science, 17, 12 (2006), 1047--1053.Google ScholarCross Ref
- Sutcliffe, A. G. and Carroll, J. M. Designing claims for reuse in interactive systems design. International Journal of Human-Computer Studies, 50, 3 (1999), 213--241. Google ScholarDigital Library
- Tetzlaff, L., & Schwartz, D. R. The use of guidelines in interface design. In SIGCHI Conference on Human Factors in Computing Systems ( 1991), ACM, 329333. Google ScholarDigital Library
- Theofanos, M. F. and Redish, J. Guidelines for accessible and usable web sites: Observing users who work with screen readers. Interactions, 10, 6 (2003), 3851.Google Scholar
- Tuch, A. N., Presslaber, E. E., Stocklin, M., Opwis, K., and Bargas-Avila, J. A. The role of visual complexity and prototypicality regarding first impression of websites: Working towards understanding aesthetic judgments. International Journal of Human-Computer Studies, 70 (2012), 794--811. Google ScholarDigital Library
- Vigo, M., Brown, J., & Conway, V. Benchmarking web acceassibility evaluation tools: measuring the harm of sole reliance on automated tests. In the 10th International Cross-Disciplinary Conference on Web Accessibility ( 2013), ACM. Google ScholarDigital Library
- Yáñez Gómez, R., Cascado Caballero, D., & Sevillano, J. L. Heuristic evaluation on mobile interfaces: A new checklist. The Scientific World Journal (2014).Google Scholar
- Yu, C. H. and Miller, R. C. Enhancing web page readability for non-native readers. In the sIGCHI conference on human factors in computing systems ( 2010), 2523--2532. Google ScholarDigital Library
- Yu, H. and Winkler, S. Image complexity and spatial information. In Quality of Multimedia Experience (QoMEX) ( 2013), IEEE, 12--17.Google ScholarCross Ref
- Zajicek, M. Successful and available: interface design exemplars for older users. Interacting with computers, 16, 3 (2994), 411--430.Google Scholar
- Zorzi, M., Barbiero, C., Facoetti, A. et al. Extra-large letter spacing improves reading in dyslexia. Proceedings of the National Academy of Sciences, 109, 28 (2012), 11455--11459.Google ScholarCross Ref
Index Terms
- Guideline-Based Evaluation of Web Readability
Recommendations
Design Guidelines for Web Readability
DIS '17: Proceedings of the 2017 Conference on Designing Interactive SystemsReading is fundamental to interactive-system use, but around 800 million of people might struggle with it due to literacy difficulties. Few websites are designed for high readability, as readability remains an underinvestigated facet of User Experience. ...
Designing for an Aging Population: Toward Universal Design
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing SystemsThis course describes age-related factors that affect ability to use the Web, and presents Web design guidelines that reflect the capabilities, usage patterns, and preferences of older Web users. The course also explains the value of testing websites on ...
Designing Websites for Adults 55+: Toward Universal Design
CHI EA '15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing SystemsThis course describes age-related factors that affect ability to use the Web, and presents Web design guidelines that reflect the capabilities, usage patterns, and preferences of older Web users. The course also explains the value of testing websites on ...
Comments