Top

Journal of Computing in Higher Education

Published in:

14-06-2018

The design, development, and implementation of student-facing learning analytics dashboards

Authors: Robert Bodily, Tarah K. Ikahihifo, Benjamin Mackley, Charles R. Graham

Published in: Journal of Computing in Higher Education | Issue 3/2018

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

We have designed, developed, and implemented a student-facing learning analytics dashboard in order to support students as they learn in online environments. There are two separate dashboards in our system: a content recommender dashboard and a skills recommender dashboard. The content recommender helps students identify gaps in their content knowledge; the skills recommender helps students improve their metacognitive strategies. We discuss the technical requirements needed to develop a real-time student dashboard as well as report our inquiry into the functionality students want in a dashboard. The dashboards were evaluated with focus groups and a perceptions survey. Students were positive in their perceptions of the dashboards and 79% of the students that used the dashboards found them user-friendly, engaging, useful, and informative. One challenge encountered was low student use of the dashboard. Only 25% of students used the dashboard multiple times, despite favorable student perceptions of the dashboard. Additional research should examine how to motivate and support students to engage with dashboard feedback in online environments.

previous article Using technology to promote classroom instruction: assessing incidences of on-task and off-task multitasking and learning

next article The importance of students’ motivational dispositions for designing learning analytics

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Allen, I. E., & Seaman, J. (2014). Tracking online education in the United States, 1–45. Retrieved from http://www.onlinelearningsurvey.com/reports/gradechange.pdf. Accessed 18 Nov 2016.

Anaya, A. R., Luque, M., & Peinado, M. (2016). A visual recommender tool in a collaborative learning experience. Expert Systems with Applications, 45, 248–259.CrossRef

Arnold, K. E. (2010). Signals: Applying academic analytics. Educause Quarterly, 33(1), n1.

Arnold, K. E., & Pistilli, M. D. (2012, April). Course signals at Purdue: Using learning analytics to increase student success. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 267–270). ACM.

Baker, R. S., Corbett, A. T., Koedinger, K. R., & Wagner, A. Z. (2004, April). Off-task behavior in the cognitive tutor classroom: When students game the system. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 383–390). ACM.

Bangert-Drowns, R. L., Kulik, C. L. C., Kulik, J. A., & Morgan, M. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61(2), 213–238.CrossRef

Bekele, T. A. (2010). Motivation and satisfaction in internet-supported learning environments: A review. Educational Technology and Society, 13(2), 116–127.

Bodily, R., & Verbert, K. (2017, March). Trends and issues in student-facing learning analytics reporting systems research. In Proceedings of the seventh international learning analytics & knowledge conference (pp. 309–318). ACM.

Charleer, S., Klerkx, J., Odriozola, S., Luis, J., & Duval, E. (2013, December). Improving awareness and reflection through collaborative, interactive visualizations of badges. In ARTEL13: Proceedings of the 3rd workshop on awareness and reflection in technology-enhanced learning (vol. 1103, pp. 69–81). CEUR-WS.

Corrin, L., & de Barba, P. (2015, March). How do students interpret feedback delivered via dashboards? In Proceedings of the fifth international conference on learning analytics and knowledge (pp. 430–431). ACM.

Danado, J., Davies, M., Ricca, P., & Fensel, A. (2010, September). An authoring tool for user generated mobile services. In Future internet symposium (pp. 118–127). Berlin: Springer.CrossRef

Doorn, D. J., Janssen, S., & O’Brien, M. (2010). Student attitudes and approaches to online homework. International Journal for the Scholarship of Teaching and Learning, 4(1), 5.CrossRef

Elias, T. (2011). Learning analytics. Learning Analytics: Definitions, Processes and Potential. Retrieved June 2015, from http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotential.pdf.

Feild, J. (2015, June). Improving student performance using nudge analytics. In Proceedings of the 8th international conference on educational data mining (pp. 464–467).

Ferguson, R., & Shum, S. B. (2012, April). Social learning analytics: Five approaches. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 23–33). ACM.

Few, S. (2006). Information dashboard design: The effective visual communication of data. Sebastopol, CA: O’Reilly Media.

Few, S. (2013). Information dashboard design: Displaying data for at-a-glance monitoring. Burlingame, CA: Analytics Press.

Garrison, D. R. (2003). Cognitive presence for effective asynchronous online learning: The role of reflective inquiry, self-direction and metacognition. Elements of Quality Online Education: Practice and Direction, 4(1), 47–58.

Govaerts, S., Verbert, K., Duval, E., & Pardo, A. (2012, May). The student activity meter for awareness and self-reflection. In CHI’12 extended abstracts on human factors in computing systems (pp. 869–884). ACM.

Govaerts, S., Verbert, K., Klerkx, J., & Duval, E. (2010, December). Visualizing activities for self-reflection and awareness. In International conference on web-based learning (pp. 91–100). Berlin: Springer.CrossRef

Grann, J., & Bushway, D. (2014, March). Competency map: Visualizing student learning to promote student success. In Proceedings of the fourth international conference on learning analytics and knowledge (pp. 168–172). ACM.

Greller, W., & Drachsler, H. (2012). Translating learning into numbers: A generic framework for learning analytics. Educational Technology and Society, 15(3), 42–57.

Hacker, D. J., Dunlosky, J., & Graesser, A. C. (Eds.). (1998). Metacognition in educational theory and practice. New York: Routledge.

Harrer, A. (2015, July). A design proposal for learner-centered visualisations of learning analytics in collaborative scenarios. In 2015 IEEE 15th international conference on advanced learning technologies (ICALT) (pp. 208–210). IEEE.

Harrer, A., & Göhnert, T. (2015, March). Integrated representations and small data: Towards contextualized and embedded analytics tools for learners. In Proceedings of the fifth international conference on learning analytics and knowledge (pp. 406–407). ACM.

Hatziapostolou, T., & Paraskakis, I. (2010). Enhancing the impact of formative feedback on student learning through an online feedback system. Electronic Journal of e-Learning, 8(2), 111–122.

Holman, C., Aguilar, S., & Fishman, B. (2013, April). GradeCraft: What can we learn from a game-inspired learning management system? In Proceedings of the third international conference on learning analytics and knowledge (pp. 260–264). ACM.

Huizing, A., & Cavanagh, M. (2011). Planting contemporary practice theory in the garden of information science. Information Research Journal, 16(4), 497.

Iandoli, L., Quinto, I., De Liddo, A., & Shum, S. B. (2014). Socially augmented argumentation tools: Rationale, design and evaluation of a debate dashboard. International Journal of Human-Computer Studies, 72(3), 298–319.CrossRef

Jeon, J. H., Yeon, J., Lee, S. G., & Seo, J. (2014, January). Exploratory visualization of smartphone-based life-logging data using Smart Reality Testbed. In 2014 International conference on big data and smart computing (BIGCOMP) (pp. 29–33). IEEE.

Jones, A., & Issroff, K. (2007). Learning technologies: Affective and social issues. In G. Conole & M. Oliver (Eds.), Contemporary perspectives in e-learning research: Themes, methods and impact on practice (pp. 190–202). London: Routledge.

Jugo, I., Kovačić, B., & Slavuj, V. (2014, May). Using data mining for learning path recommendation and visualization in an intelligent tutoring system. In 2014 37th international convention on information and communication technology, electronics and microelectronics (MIPRO) (pp. 924–928). IEEE.

Kerly, A., Ellis, R., & Bull, S. (2008). CALMsystem: A conversational agent for learner modelling. Knowledge-Based Systems, 21(3), 238–246. https://doi.org/10.1016/j.knosys.2007.11.015.CrossRef

Kim, J., Jo, I. H., & Park, Y. (2016). Effects of learning analytics dashboard: Analyzing the relations among dashboard utilization, satisfaction, and learning achievement. Asia Pacific Education Review, 17(1), 13–24.CrossRef

Kortemeyer, G. (2015). An empirical study of the effect of granting multiple tries for online homework. American Journal of Physics, 83(7), 646–653.CrossRef

Kuosa, K., Distante, D., Tervakari, A., Cerulo, L., Fernández, A., Koro, J., et al. (2016). Interactive visualization tools to improve learning and teaching in online learning environments. International Journal of Distance Education Technologies (IJDET), 14(1), 1–21.CrossRef

Laffey, J. M., Amelung, C., & Goggins, S. (2014). Using analytics for activity awareness in learning systems. International Journal of Designs for Learning, 5(2), 101.CrossRef

Lent, R. W., Brown, S. D., & Larkin, K. C. (1984). Relation of self-efficacy expectations to academic achievement and persistence. Journal of Counseling Psychology, 31(3), 356.CrossRef

Manso-Vázquez, M., & Llamas-Nistal, M. (2015). Proposal of a learning organization tool with support for metacognition. IEEE Revista Iberoamericana de Tecnologias del Aprendizaje, 10(2), 35–42.CrossRef

McAuley, J., O’Connor, A., & Lewis, D. (2012, April). Exploring reflection in online communities. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 102–110). ACM.

Melero, J., Hernández-Leo, D., Sun, J., Santos, P., & Blat, J. (2015). How was the activity? A visualization support for a case of location-based learning design. British Journal of Educational Technology, 46(2), 317–329.CrossRef

Millman, J. (1989). If at first you don’t succeed setting passing scores when more than one attempt is permitted. Educational Researcher, 18(6), 5–9.

Muldner, K., Wixon, M., Rai, D., Burleson, W., Woolf, B., & Arroyo, I. (2015). Exploring the impact of a learning dashboard on student affect. In C. Conati, N. Heffernan, A. Mitrovic, M. F. Verdejo (Eds.), International conference on artificial intelligence in education (pp. 307–317). Berlin: Springer.CrossRef

Odriozola, S., Luis, J., Verbert, K., Govaerts, S., & Duval, E. (2011, August). Visualizing PLE usage. In Proceedings of EFEPLE11 1st workshop on exploring the fitness and evolvability of personal learning environments (Vol. 773, pp. 34–38). CEUR WS.

Olmos, M. M., & Corrin, L. (2012). Learning analytics: A case study of the process of design of visualizations. Journal of Asynchronous Learning Networks, 16(3), 39–49.

Ott, C., Robins, A., Haden, P., & Shephard, K. (2015). Illustrating performance indicators and course characteristics to support students’ self-regulated learning in CS1. Computer Science Education, 25(2), 174–198.CrossRef

Park, Y., & Jo, I. H. (2015). Development of the learning analytics dashboard to support students’ learning performance. Journal of Universal Computer Science, 21(1), 110–133.

Resnick, P., & Varian, H. R. (1997). Recommender systems. Communications of the ACM, 40(3), 56–58.CrossRef

Ruipérez-Valiente, J. A., Muñoz-Merino, P. J., & Kloos, C. D. (2013, November). An architecture for extending the learning analytics support in the Khan Academy framework. In Proceedings of the first international conference on technological ecosystem for enhancing multiculturality (pp. 277–284). ACM.

Ruipérez-Valiente, J. A., Muñoz-Merino, P. J., Leony, D., & Kloos, C. D. (2015). ALAS-KA: A learning analytics extension for better understanding the learning process in the Khan Academy platform. Computers in Human Behavior, 47, 139–148.CrossRef

Rydberg, E. (2011). Visualizing the software development process by analyzing software engineering data. Stockholm: Royal Institute of Technology.

Santos, J. L., Govaerts, S., Verbert, K., & Duval, E. (2012, April). Goal-oriented visualizations of activity tracking: A case study with engineering students. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 143–152). ACM.

Santos, J. L., Verbert, K., & Duval, E. (2012b). Empowering students to reflect on their activity with StepUp!: Two case studies with engineering students (p. 73). Editorial: Awareness and Reflection in Technology Enhanced Learning.

Santos, J. L., Verbert, K., Govaerts, S., & Duval, E. (2013, April). Addressing learner issues with StepUp!: An evaluation. In Proceedings of the third international conference on learning analytics and knowledge (pp. 14–22). ACM.

Santos, J., Verbert, K., Klerkx, J., Duval, E., Charleer, S., & Ternier, S. (2015). Tracking data in open learning environments. Journal of Universal Computer Science, 21(7), 976–996.

Schmitz, H. C., Scheffel, M., Friedrich, M., Jahn, M., Niemann, K., & Wolpers, M. (2009, September). CAMera for PLE. In European conference on technology enhanced learning (pp. 507–520). Berlin: Springer.

Schwendimann, B. A., Rodriguez-Triana, M. J., Vozniuk, A., Prieto, L. P., Boroujeni, M. S., Holzer, A., et al. (2017). Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies, 10(1), 30–41.CrossRef

Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.CrossRef

Siemens, G. (2010, July 22). 1st International conference on learning analytics and knowledge 2011. Retrieved March 30, 2016, from https://tekri.athabascau.ca/analytics/.

Silius, K., Miilumäki, T., Huhtamäki, J., Tebest, T., Meriläinen, J., & Pohjolainen, S. (2010). Students’ motivations for social media enhanced studying and learning. Knowledge Management and E-Learning: An International Journal, 2(1), 51–67.

Silius, K., Tervakari, A. M., & Kailanto, M. (2013, March). Visualizations of user data in a social media enhanced web-based environment in higher education. In 2013 IEEE global engineering education conference (EDUCON) (pp. 893–899). IEEE.

Stallings, J. (1980). Allocated academic learning time revisited, or beyond time on task. Educational researcher, 9(11), 11–16.CrossRef

Steel, P. (2007). The nature of procrastination: A meta-analytic and theoretical review of quintessential self-regulatory failure. Psychological Bulletin, 133(1), 65–94. https://doi.org/10.1037/0033-2909.133.1.65.CrossRef

Tervakari, A. M., Silius, K., Koro, J., Paukkeri, J., & Pirttila, O. (2014, April). Usefulness of information visualizations based on educational data. In 2014 IEEE global engineering education conference (EDUCON) (pp. 142–151). IEEE.

Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57(10), 1–10. https://doi.org/10.1177/0002764213479363.CrossRef

Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Van Assche, F., Parra, G., et al. (2014). Learning dashboards: An overview and future research opportunities. Personal and Ubiquitous Computing, 18(6), 1499–1514. https://doi.org/10.1007/s00779-013-0751-2.CrossRef

Wilson, B. G. (2013). A practice-centered approach to instructional design. In Learning, problem solving, and mind tools: Essays in honor of David H. Jonassen, (pp. 35–54).

Yoo, Y., Lee, H., Jo, I., & Park, Y. (2015). Educational dashboards for smart learning: Review of case studies. Emerging Issues in Smart Learning. https://doi.org/10.1007/978-3-662-44188-6.CrossRef

Title: The design, development, and implementation of student-facing learning analytics dashboards
Authors: Robert Bodily
Tarah K. Ikahihifo
Benjamin Mackley
Charles R. Graham
Publication date: 14-06-2018
Publisher: Springer US
Published in: Journal of Computing in Higher Education / Issue 3/2018
Print ISSN: 1042-1726
Electronic ISSN: 1867-1233
DOI: https://doi.org/10.1007/s12528-018-9186-0

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 3/2018

Using technology to promote classroom instruction: assessing incidences of on-task and off-task multitasking and learning

Acceptance and usage of mobile assisted language learning by higher education students

How ready are our students for technology-enhanced learning? Students at a university of technology respond

The role of behavioral expectation in technology acceptance: a CEGEP case study

Barriers to technology implementation in community college mathematics classrooms

A DEMATEL method in identifying design requirements for mobile environments: students’ perspectives

Premium Partner