skip to main content
10.1145/3447535.3462485acmconferencesArticle/Chapter ViewAbstractPublication PageswebsciConference Proceedingsconference-collections
research-article

Measuring Digital Literacy with Eye Tracking: An examination of skills and performance based on user gaze

Published:22 June 2021Publication History

ABSTRACT

Digital inequality has been intensively studied in recent decades, due to the considerable social significance of this phenomenon. Research has struggled with finding quality and profound ways to measure digital literacy of people from different social groups, due to the dynamic character of digital technology, which results in ever-changing, nuanced types of digital inequality. This study proposes an innovative method for examining how users approach and accomplish digital tasks by introducing eye tracking for measuring user scan patterns, gaze and attention during completion of tasks. Eye tracking as a measurement of attention and focus reflects the processes that occur while users are performing required tasks, and therefore may be a useful tool to comprehend digital literacy. We apply this innovative methodology in a repeated measures observation study of digital skills of low-skilled participants in a computer introductory course. 19 participants were requested to perform several online tasks before and after completing the course. The paper describes the results, which demonstrate that although participants’ skills have improved, the improvement is manifest in basic, trivial uses, while advance uses, such as understanding of efficient searching, or using the Internet in sophisticated ways as an environmental resource supporting one's situational awareness, are only slightly improved, as data based on tracking user gaze and mouse movements reveals.

Skip Supplemental Material Section

Supplemental Material

PS1.2_NiliSteinfeld_MeasuringDigitalLiteracy_with_EyeTracking_16_6_2021.mp4

mp4

161.7 MB

References

  1. Bhanu Bhakta Acharya. 2017. Conceptual evolution of the digital divide: A systematic review of the literature over a period of five years (2010 – 2015). Journal of Russian Media & Journalism Studies, 41-74.Google ScholarGoogle Scholar
  2. Monica Anderson and John B. Horrigan. 2016. Smartphones help those without broadband get online, but don't necessarily bridge the digital divide.  PEW Internet Research Center.Google ScholarGoogle Scholar
  3. Selami Aydin. 2012. A review of research on Facebook as an educational environment. Educational Technology Research and Development, 60 6 , (June 2012), 1093-1106.Google ScholarGoogle Scholar
  4. Laurence Baker, Todd H. Wagner, Sara Singer and Bundorf, M.K. 2003. Use of the Internet and e-mail for health care information. Journal of the American Medical Association, 289, 2400–2406.Google ScholarGoogle ScholarCross RefCross Ref
  5. Yochai Benkler. 2006. The wealth of networks: How social production transforms markets and freedom. Yale University Press. New Haven, CT.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Manuel Castells. 2002. The Internet galaxy: Reflections on the Internet, business, and society. Oxford University Press. Oxford, UK.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Stephen Coleman and Jay G. Blumler. 2009. The Internet and democratic citizenship: Theory, practice and policy. Cambridge University Press. Cambridge, UK.Google ScholarGoogle Scholar
  8. Daniela Di Gianatonio. 2018. The journey to digital equality: Siteimprove and the UN's sustainable development Goals. Retrieved: https://siteimprove.com/en-gb/blog/the-journey-to-digital-equality-siteimprove-and-the-un-s-sustainable-development-goals/Google ScholarGoogle Scholar
  9. Paul DiMaggio, Eszter Hargittai, Coral Celeste and Steven Shafer. 2004. Digital inequality: From unequal access to differentiated use. In: Kathryn Neckerman, (Ed.), Social inequality (pp. 355-400). New York, NY: Russell Sage Foundation.Google ScholarGoogle Scholar
  10. Paul DiMaggio and Bart Bonikowsky. 2008. Make money surfing the web? The impact of Internet use on the earnings of US worker. American Sociological Review, 73, 2 (April 2008), 227–250.Google ScholarGoogle ScholarCross RefCross Ref
  11. Soussan Djamasbi, Maria Siegel and Tom Tullis. 2010. Generation Y, web design, and eye tracking. International Journal of Human Computer Studies, 68, 5 (May 2010), 307-323.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Matias Dodel and Gustavo Mesch. 2018. Inequality in digital skills and the adoption of online safety behaviors. Information, Communication & Society 21, 5 (February 2018), 712-728.Google ScholarGoogle ScholarCross RefCross Ref
  13. Matthew S. Eastin and Robert LaRose. 2000. Internet self‐efficacy and the psychology of the digital divide. Journal of Computer‐Mediated Communication, 6, 1 (September 2000).Google ScholarGoogle Scholar
  14. Claudia Ehmke and Stephanie Wilson. 2007. Identifying web usability problems fromeye-tracking data. In Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI... but not as we know it-Volume 1, 119-128.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Mica R. Endsley and Debra G. Jones .2016. Designing for situation awareness: An approach to user-centered design. CRC press.‏ Boca Raton, USA.Google ScholarGoogle Scholar
  16. Yoram Eshet-Alkalai and Yair Amichai-Hamburger. 2004. Experiments in digital literacy. CyberPsychology & Behavior, 7, 4 (September 2004), 421–429.Google ScholarGoogle Scholar
  17. Gunther Eysenbach. 2003. The impact of the Internet on cancer outcomes. Ca: A Cancer Journal for Clinicians 53, 6 (November 2003), 356– 371.Google ScholarGoogle ScholarCross RefCross Ref
  18. Steven Fraiberg. 2017. Start-up nation: Studying transnational entrepreneurial practices in Israel's start-up ecosystem. Journal of Business and Technical Communication 31, 3 (April 2017), 350-388.Google ScholarGoogle ScholarCross RefCross Ref
  19. Yael Fisher and Orit Bandes-Jacob. 2003. Tsimtsum Hapaar Hadigitali BeIsrael- Proyect Lehava Doch mesacem. Henrietta Szold Institute. Jerusalem, Israel.Google ScholarGoogle Scholar
  20. Asmaa Ganayem. 2018. Hainternet Bahevra Haaravit BeIsrael. Igud Hainternet HaIsraeli, available at: https://cdn.the7eye.org.il/uploads/2018/10/internet-arab-society.pdfGoogle ScholarGoogle Scholar
  21. Eszter Hargittai. 2002. Beyond logs and surveys: In‐depth measures of people's web use skills. Journal of the American Society for Information Science and technology, 53, 14 (December 2002), 1239-1244.Google ScholarGoogle Scholar
  22. Eszter Hargittai. 2003. The digital divide and what to do about it. In: D. C. Jones, (Ed.), New economy handbook (pp. 822-841). San Diego, CA: Academic Press.Google ScholarGoogle Scholar
  23. Martin Hilbert. 2011. The end justifies the definition: The manifold outlooks on the digital divide and their practical usefulness for policy-making. Telecommunications Policy, 35, 8 (September 2011), 715-736.Google ScholarGoogle Scholar
  24. Khan, M. Laeeq, Howard T. Welser, Claudia Cisneros, Gaone Manatong and Ika Karlina Idris. 2020. Digital inequality in the Appalachian Ohio: Understanding how demographics, internet access, and skills can shape vital information use (VIU). Telematics and Informatics, 50 (July 2020).Google ScholarGoogle Scholar
  25. Lee Kahner, Gilad Malach and Maya Hoshen. 2017. Shnaton Hahevra Hacharedit Beisrael, Israel Democracy Institute. Jerusalem, Israel.Google ScholarGoogle Scholar
  26. Seok Kang and Sherice Gearhart. 2010. E-government and civic engagement: How is citizens' use of city web sites related with civic involvement and political behaviors? Journal of Broadcasting and Electronic Media, 54, 3(August 2010), 443-462.Google ScholarGoogle ScholarCross RefCross Ref
  27. Andreas M. Kaplan and Michael Haenlein. 2010. Users of the world, unite! The challenges and opportunities of social media. Business Horizon, 53, 1(February 2010), 59-68.Google ScholarGoogle Scholar
  28. Christos Katsanos, Nikolaos Tselios and Nikolaos Avouris. 2010. Evaluating website navigability: validation of a tool-based approach through two eye-tracking user studies. New review of Hypermedia and Multimedia, 16, 1-2 (March 2010), 195-214.Google ScholarGoogle Scholar
  29. Sabina Lissitsa and Azi Lev-On. 2014. Gaps close, gaps open: A repeated cross-sectional study of the scope and determinants of the ethnic digital divide. International Journal of Electronic Governance, 7, 1, 56–71.Google ScholarGoogle ScholarCross RefCross Ref
  30. Sabina Lissitsa. 2015. Patterns of digital uses among Israeli Arabs: Between citizenship in modern society and traditional cultural roots. Asian Journal of Communication, 25, 5, 447-464.Google ScholarGoogle ScholarCross RefCross Ref
  31. Eden Litt. 2013. Measuring users’ internet skills: A review of past assessments and a look toward the future. New Media & Society, 15, 4 (May 2013), 612-630.‏Google ScholarGoogle Scholar
  32. Han-Chin Liu, Meng-lung Lai and Hsueh-Hua Chuang. 2011. Using eye-tracking technology to investigate the redundant effect of multimedia web pages on viewers' cognitive processes. Computers in Human Behavior, 27, 6 (November 2011), 2410-2417.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Sonia Livingstone. 2008. Internet literacy: Young people's negotiation of new online opportunities. In: McPherson, T. (Ed.), The John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning, Digital Youth, Innovation, and the Unexpected (pp. 101-122). Cambridge, MA: The MIT Press.Google ScholarGoogle Scholar
  34. Sonia Livingstone and Ellen Helsper. 2007. Gradations in digital inclusion: Children, young people and the digital divide. New Media & Society, 9, 4 (August 2007) 671–696.Google ScholarGoogle ScholarCross RefCross Ref
  35. Mary Madden and Lee Rainie. 2003. America's online pursuits: The changing picture of who's online and what they do. PEW Internet and American Life Project.‏Google ScholarGoogle Scholar
  36. Gustavo Mesch, Ilan Talmud and Tanya Kolobov. 2013. 14 Explaining digital inequalities in israel. The Digital Divide (2013): 222.Google ScholarGoogle Scholar
  37. Gustavo Mesch, Rita Mano and Judith Tsamir. 2012. Minority status and health information search: A test of the diversification hypothesis. Social Science & Medicine, 75, 5(September 2012), 854-858.Google ScholarGoogle Scholar
  38. Michal Mičík and Hana Kunešová. 2020. Using an eye tracker to optimise career websites as a communication channel with Generation Y. Economic Research-Ekonomska Istraživanja, 1-24.Google ScholarGoogle Scholar
  39. Efrat Neter and Esther Brainin. 2012. eHealth literacy: extending the digital divide to the realm of health information. Journal of medical Internet research, 14, 1(January 2012).Google ScholarGoogle Scholar
  40. Pippa Norris. 2001. Digital divide: Civic engagement, information poverty, and the Internet worldwide. Cambridge University Press. Cambridge, UK.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Hun M. Park and James L. Perry. 2008. Does Internet use really facilitate civic engagement? Empirical evidences from the American National Election Studies. In: E. Bergrud, & K. Yang (Eds.), Civic engagement in a networked society (pp. 237-270). Charlotte, NC: Information Age Publishing.Google ScholarGoogle Scholar
  42. Yong Jin Park. 2015. My whole world's in my palm! The second-level divide of teenagers’ mobile use and skill. New media & society, 17, 6, 977-995.Google ScholarGoogle Scholar
  43. Katy E. Pearce and Ronald E. Rice. 2013. Digital divides from access to activities: Comparing mobile and personal computer Internet users. Journal of Communication, 63, 4, 721-744.Google ScholarGoogle Scholar
  44. Katy E. Pearce and Ronald E. Rice. 2017. Somewhat separate and unequal: digital divides, social networking sites, and capital-enhancing activities. Social Media+ Society, 3, 2 (June 2017).Google ScholarGoogle Scholar
  45. Alex Poole and Linden J. Ball. 2006. Eye tracking in HCI and usability research. Encyclopedia of Human Computer Interaction, 1, 211-219.Google ScholarGoogle ScholarCross RefCross Ref
  46. Simon J. Purdy. 2017. Internet use and civic engagement: A structural equation approach. Computers in Human Behavior, 71 (June 2017), 318-326.‏Google ScholarGoogle Scholar
  47. Keith Rayner. 2009. Eye movements and attention in reading, scene perception, and visual search. The quarterly journal of experimental psychology, 62, 8 (August 2009), 1457-1506. doi:10.1080/17470210902816461Google ScholarGoogle Scholar
  48. Dennis Rosenberg. 2020. Immigrants’ disadvantage online: understanding the effects of immigration status, gender, and country of origin on the e-government use in Israel. Digital Policy, Regulation and GovernanceGoogle ScholarGoogle Scholar
  49. Aaron W. Smith, Kay L. Schlozman, Sidney Verba and Henry Brady. 2009. The Internet and Civic Engagement. Pew Internet & American Life Project. Washington, DC.Google ScholarGoogle Scholar
  50. Stuart Soroka, Patrick Fournier, Lilach Nir and John Hibbing. 2018. Psychophysiology in the Study of Political Communication: An Expository Study of Individual-Level Variation in Negativity Biases. Political Communication, 36, 2, 288-302.Google ScholarGoogle Scholar
  51. Brian H. Spitzberg. 2006. Preliminary development of a model and measure of computer‐mediated communication (CMC) competence. Journal of Computer‐Mediated Communication, 11, 2 (January 2006), 629-666.‏Google ScholarGoogle Scholar
  52. Nili Steinfeld. 2016. “I agree to the terms and conditions”: (How) do users read privacy policies online? An eye-tracking experiment. Computers in human behavior, 55, 2 (February 2016), 992-1000.‏Google ScholarGoogle Scholar
  53. Nili Steinfeld and Ohad Shaked. (in print). Looking my (Enemy?) in the Eyes: An eye tracking study of Simulated Virtual Intergroup Contact. Media, War and Conflict.Google ScholarGoogle Scholar
  54. Nili Steinfeld, Tal Samuel-Azran and Azi Lev-On. 2016. User comments and opinion formation: Findings from an eye-tracking experiment. Computers in Human Behavior, 61 (August 2016), 63-72.Google ScholarGoogle Scholar
  55. Jan Van Dijk. 2005. The network society: Social aspects of the new media. Thousand Oaks, CA: Sage Publications.Google ScholarGoogle Scholar
  56. Jan van Dijk. 2012. The evolution of the digital divide: The digital divide turns to inequality of skills and usage. In: J. Bus, M. Crompton, M. Hildebrandt, & G. Metakides, (Eds.), Digital Enlightenment Yearbook 2012 (pp. 57-78).Google ScholarGoogle Scholar
  57. Jan van Dijk and Alexander van Deursen. 2010. Traditional media skills and digital media skills: Much of a difference. In: 60th Annual International Communication Association (ICA) Conference, Singapore.Google ScholarGoogle Scholar
  58. Sara Vieweg, Amanda L. Hughes, Kate Starbird and Leysia Palen. 2010. Microblogging during two natural hazards events: what twitter may contribute to situational awareness. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1079-1088). New York: ACM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Gila Zelka. 2012. Tsimtsum Hapaar Hadigitali Bekerev Uchlusiot Muchlashot BeIsrael, Teaching and learning in the Internet age, 14, 101-13.Google ScholarGoogle Scholar
  60. Sheila Zimic. 2009. Not so ‘techno-savvy’: Challenging the stereotypical images of the ‘Net generation’. Digital Culture & Education, 1, 2, 129-144.‏Google ScholarGoogle Scholar

Recommendations

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Sign in
  • Published in

    cover image ACM Conferences
    WebSci '21: Proceedings of the 13th ACM Web Science Conference 2021
    June 2021
    328 pages
    ISBN:9781450383301
    DOI:10.1145/3447535

    Copyright © 2021 ACM

    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    • Published: 22 June 2021

    Permissions

    Request permissions about this article.

    Request Permissions

    Check for updates

    Qualifiers

    • research-article
    • Research
    • Refereed limited

    Acceptance Rates

    Overall Acceptance Rate218of875submissions,25%

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format .

View HTML Format