Skip to main content
Top

2017 | OriginalPaper | Chapter

Evaluating the Effect of Uncertainty Visualisation in Open Learner Models on Students’ Metacognitive Skills

Authors : Lamiya Al-Shanfari, Carrie Demmans Epp, Chris Baber

Published in: Artificial Intelligence in Education

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Self-assessment is widely used in open learner models (OLMs) as a metacognitive process to enhance students’ self-regulated learning. Yet little research has investigated the impact of the visualisation when the OLM shows the conflict (i.e., uncertainty) between the system’s beliefs about student knowledge and students’ confidence in the correctness of their answers. We deployed such an OLM and studied its use. The impact of the uncertainty visualisation on student learning, confidence gains and actions was determined by comparing these measures across two treatment conditions and a control condition. Those who accessed the OLM performed significantly better on the post-test, and those in the treatment group who could see both sets of beliefs separately showed greater confidence gains and used the system more.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Boud, D.: The role of self-assessment in student grading. Assess. Eval. High. Educ. 14(1), 20–30 (1989)CrossRef Boud, D.: The role of self-assessment in student grading. Assess. Eval. High. Educ. 14(1), 20–30 (1989)CrossRef
2.
go back to reference Jackson, S.A., Kleitman, S.: Metacognit. Learn. 9(1), 25–49 (2014). SpringerCrossRef Jackson, S.A., Kleitman, S.: Metacognit. Learn. 9(1), 25–49 (2014). SpringerCrossRef
3.
go back to reference Butler, D.L., Winne, P.H.: Feedback and self-regulated learning: a theoretical synthesis. Rev. Educ. Res. 65(3), 245–281 (1995)CrossRef Butler, D.L., Winne, P.H.: Feedback and self-regulated learning: a theoretical synthesis. Rev. Educ. Res. 65(3), 245–281 (1995)CrossRef
4.
go back to reference Kulhavy, R.W., Stock, W.A.: Feedback in written instruction: the place of response certitude. Educ. Psychol. Rev. 1(4), 279–308 (1989)CrossRef Kulhavy, R.W., Stock, W.A.: Feedback in written instruction: the place of response certitude. Educ. Psychol. Rev. 1(4), 279–308 (1989)CrossRef
5.
go back to reference Bull, S., Kay, J.: Open learner models as drivers for metacognitive processes. In: Azevedo, R., Aleven, V. (eds.) International Handbook of Metacognition and Learning Technologies. SIHE, vol. 28, pp. 349–365. Springer, New York (2013). doi:10.1007/978-1-4419-5546-3_23 CrossRef Bull, S., Kay, J.: Open learner models as drivers for metacognitive processes. In: Azevedo, R., Aleven, V. (eds.) International Handbook of Metacognition and Learning Technologies. SIHE, vol. 28, pp. 349–365. Springer, New York (2013). doi:10.​1007/​978-1-4419-5546-3_​23 CrossRef
6.
go back to reference Woolf, B.: Building Intelligent Interactive Tutors: Student-Centered Strategies for Revolutionizing E-learning, pp. 49–94. Morgan Kaufmann, Burlington (2009) Woolf, B.: Building Intelligent Interactive Tutors: Student-Centered Strategies for Revolutionizing E-learning, pp. 49–94. Morgan Kaufmann, Burlington (2009)
7.
go back to reference Demmans Epp, C., Bull, S.: Uncertainty representation in visualizations of learning analytics for learners: current approaches and opportunities. IEEE Trans. Learn. Technol. 8(3), 242–260 (2015)CrossRef Demmans Epp, C., Bull, S.: Uncertainty representation in visualizations of learning analytics for learners: current approaches and opportunities. IEEE Trans. Learn. Technol. 8(3), 242–260 (2015)CrossRef
8.
go back to reference Zwick, R., Zapata-Rivera, D., Hegarty, M.: Comparing graphical and verbal representations of measurement error in test score reports. Educ. Assess. 19(2), 116–138 (2014). RoutledgeCrossRef Zwick, R., Zapata-Rivera, D., Hegarty, M.: Comparing graphical and verbal representations of measurement error in test score reports. Educ. Assess. 19(2), 116–138 (2014). RoutledgeCrossRef
9.
go back to reference Zapata-Rivera, J.D., Greer, J.E.: Interacting with inspectable bayesian models. Int. J. Artif. Intell. Educ. 14, 127–163 (2004) Zapata-Rivera, J.D., Greer, J.E.: Interacting with inspectable bayesian models. Int. J. Artif. Intell. Educ. 14, 127–163 (2004)
10.
go back to reference Mohanarajah, S., Kemp, R.H., Kemp, E.: Opening a fuzzy learner model. In: Proceedings of Workshop on Learner Modelling for Reflection, International Conference on Artificial Intelligence in Education, pp. 62–71 (2005) Mohanarajah, S., Kemp, R.H., Kemp, E.: Opening a fuzzy learner model. In: Proceedings of Workshop on Learner Modelling for Reflection, International Conference on Artificial Intelligence in Education, pp. 62–71 (2005)
11.
go back to reference Aleven, V., Popescu, O., Ogan, A., Koedinger, K.R.: A formative classroom evaluation of a tutorial dialogue system that supports self-explanation. In: Aleven, V., Hoppe, U., Kay, J., Mizoguchi, R., Pain, H., Verdejo, F., Yacef, K. (eds.) Supplemental Proceedings of the 11th International Conference on Artificial Intelligence in Education, AIED 2003, vol. VI, pp. 345–355. School of Information Technologies, University of Sydney (2003) Aleven, V., Popescu, O., Ogan, A., Koedinger, K.R.: A formative classroom evaluation of a tutorial dialogue system that supports self-explanation. In: Aleven, V., Hoppe, U., Kay, J., Mizoguchi, R., Pain, H., Verdejo, F., Yacef, K. (eds.) Supplemental Proceedings of the 11th International Conference on Artificial Intelligence in Education, AIED 2003, vol. VI, pp. 345–355. School of Information Technologies, University of Sydney (2003)
12.
go back to reference Bull, S., Kay, J.: SMILI☺: a framework for interfaces to learning data in open learner models, learning analytics and related fields. Int. J. Artif. Intell. Educ. 26(1), 293–331 (2016)CrossRef Bull, S., Kay, J.: SMILI☺: a framework for interfaces to learning data in open learner models, learning analytics and related fields. Int. J. Artif. Intell. Educ. 26(1), 293–331 (2016)CrossRef
13.
go back to reference Mitrovic, A., Martin, B.: Evaluating the effect of open student models on self-assessment. Int. J. Artif. Intell. Educ. 17(2), 121–144 (2007) Mitrovic, A., Martin, B.: Evaluating the effect of open student models on self-assessment. Int. J. Artif. Intell. Educ. 17(2), 121–144 (2007)
14.
go back to reference Kerly, A., Bull, S.: Children’s interactions with inspectable and negotiated learner models. In: Woolf, Beverley P., Aïmeur, E., Nkambou, R., Lajoie, S. (eds.) ITS 2008. LNCS, vol. 5091, pp. 132–141. Springer, Heidelberg (2008). doi:10.1007/978-3-540-69132-7_18 CrossRef Kerly, A., Bull, S.: Children’s interactions with inspectable and negotiated learner models. In: Woolf, Beverley P., Aïmeur, E., Nkambou, R., Lajoie, S. (eds.) ITS 2008. LNCS, vol. 5091, pp. 132–141. Springer, Heidelberg (2008). doi:10.​1007/​978-3-540-69132-7_​18 CrossRef
15.
go back to reference Brusilovsky, P., Somyürek, S., Guerra, J., Hosseini, R., Zadorozhny, V.: The value of social: comparing open student modeling and open social student modeling. In: Ricci, F., Bontcheva, K., Conlan, O., Lawless, S. (eds.) UMAP 2015. LNCS, vol. 9146, pp. 44–55. Springer, Cham (2015). doi:10.1007/978-3-319-20267-9_4 CrossRef Brusilovsky, P., Somyürek, S., Guerra, J., Hosseini, R., Zadorozhny, V.: The value of social: comparing open student modeling and open social student modeling. In: Ricci, F., Bontcheva, K., Conlan, O., Lawless, S. (eds.) UMAP 2015. LNCS, vol. 9146, pp. 44–55. Springer, Cham (2015). doi:10.​1007/​978-3-319-20267-9_​4 CrossRef
16.
go back to reference Long, Y., Aleven, V.: Supporting students’ self-regulated learning with an open learner model in a linear equation tutor. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 219–228. Springer, Heidelberg (2013). doi:10.1007/978-3-642-39112-5_23 CrossRef Long, Y., Aleven, V.: Supporting students’ self-regulated learning with an open learner model in a linear equation tutor. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 219–228. Springer, Heidelberg (2013). doi:10.​1007/​978-3-642-39112-5_​23 CrossRef
17.
go back to reference Bull, S., Pain, H.: “Did i say what i think i said, and do you agree with me?” Inspecting and questioning the student model. In: Greer, J. (ed.) Proceedings of World Conference on Artificial Intelligence and Education. Association for the Advancement of Computing in Education, VA, USA, pp. 501–508 (1995) Bull, S., Pain, H.: “Did i say what i think i said, and do you agree with me?” Inspecting and questioning the student model. In: Greer, J. (ed.) Proceedings of World Conference on Artificial Intelligence and Education. Association for the Advancement of Computing in Education, VA, USA, pp. 501–508 (1995)
18.
go back to reference Kinkeldey, C., MacEachren, A.M., Schiewe, J.: How to assess visual communication of uncertainty? A systematic review of geospatial uncertainty visualisation user studies. Cartographic J. 51(4), 372–386 (2014). Taylor & FrancisCrossRef Kinkeldey, C., MacEachren, A.M., Schiewe, J.: How to assess visual communication of uncertainty? A systematic review of geospatial uncertainty visualisation user studies. Cartographic J. 51(4), 372–386 (2014). Taylor & FrancisCrossRef
19.
go back to reference Al-Shanfari, L., Demmans Epp, C., Bull, S.: Uncertainty in open learner models: visualising inconsistencies in the underlying data. In: Bull, S., Ginon, B., Kickmeier-Rust, M., Kay, J., Johnson, M.D. (eds.) Workshop on Learning Analytics for Learners (LAK 2016), CEUR, pp. 23–30 (2016) Al-Shanfari, L., Demmans Epp, C., Bull, S.: Uncertainty in open learner models: visualising inconsistencies in the underlying data. In: Bull, S., Ginon, B., Kickmeier-Rust, M., Kay, J., Johnson, M.D. (eds.) Workshop on Learning Analytics for Learners (LAK 2016), CEUR, pp. 23–30 (2016)
20.
go back to reference Bull, S., Jackson, T., Lancaster, M.: Students’ interest in their misconceptions in first year electrical circuits and mathematics courses. Int. J. Electr. Eng. Educ. 47(3), 307–318 (2010)CrossRef Bull, S., Jackson, T., Lancaster, M.: Students’ interest in their misconceptions in first year electrical circuits and mathematics courses. Int. J. Electr. Eng. Educ. 47(3), 307–318 (2010)CrossRef
21.
go back to reference Schön, D.A.: Educating the Reflective Practitioner. Jossey-Bass Publishers, San Francisco (1987) Schön, D.A.: Educating the Reflective Practitioner. Jossey-Bass Publishers, San Francisco (1987)
Metadata
Title
Evaluating the Effect of Uncertainty Visualisation in Open Learner Models on Students’ Metacognitive Skills
Authors
Lamiya Al-Shanfari
Carrie Demmans Epp
Chris Baber
Copyright Year
2017
DOI
https://doi.org/10.1007/978-3-319-61425-0_2

Premium Partner