Skip to main content

2015 | OriginalPaper | Buchkapitel

A Comparative Framework to Evaluate Recommender Systems in Technology Enhanced Learning: a Case Study

verfasst von : Matteo Lombardi, Alessandro Marani

Erschienen in: Advances in Artificial Intelligence and Its Applications

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

When proposing a novel recommender system, one difficult part is its evaluation. Especially in Technology Enhanced Learning (TEL), this phase is critical because those systems influence students or educators in educational tasks. Our research aims to propose a framework for conducting comparative experiments of different recommender systems in a same educational context. The framework is expected to provide the accuracy of subject systems within a single experiment, depicting the benefits of a novel system against others. We also present an application of such framework for a comparative experiment of popular systems in TEL like Google, Slideshare, Youtube, MERLOT, Connexions and ARIADNE. Our results show that the proposed framework has been effective in comparing the accuracy of those systems, with a clear picture of their performance compared one another. Moreover, the results of the experiment can be used as a benchmark when evaluating novel recommender systems in TEL.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Anand, D., Mampilli, B.S.: Folksonomy-based fuzzy user profiling for improved recommendations. Expert Syst. Appl. 41(5), 2424–2436 (2014)CrossRef Anand, D., Mampilli, B.S.: Folksonomy-based fuzzy user profiling for improved recommendations. Expert Syst. Appl. 41(5), 2424–2436 (2014)CrossRef
2.
Zurück zum Zitat Bobadilla, J., Ortega, F., Hernando, A., Alcalá, J.: Improving collaborative filtering recommender system results and performance using genetic algorithms. Knowl.-Based Syst. 24(8), 1310–1316 (2011)CrossRef Bobadilla, J., Ortega, F., Hernando, A., Alcalá, J.: Improving collaborative filtering recommender system results and performance using genetic algorithms. Knowl.-Based Syst. 24(8), 1310–1316 (2011)CrossRef
3.
Zurück zum Zitat Bozo, J., Alarcón, R., Iribarra, S.: Recommending learning objects according to a teachers’ contex model. In: Wolpers, M., Kirschner, P.A., Scheffel, M., Lindstaedt, S., Dimitrova, V. (eds.) EC-TEL 2010. LNCS, vol. 6383, pp. 470–475. Springer, Heidelberg (2010) CrossRef Bozo, J., Alarcón, R., Iribarra, S.: Recommending learning objects according to a teachers’ contex model. In: Wolpers, M., Kirschner, P.A., Scheffel, M., Lindstaedt, S., Dimitrova, V. (eds.) EC-TEL 2010. LNCS, vol. 6383, pp. 470–475. Springer, Heidelberg (2010) CrossRef
4.
Zurück zum Zitat Brent, I., Gibbs, G.R., Gruszczynska, A.K.: Obstacles to creating and finding open educational resources: the case of research methods in the social sciences. J. Interact. Media Educ. 2012(1):Art-5 (2012) Brent, I., Gibbs, G.R., Gruszczynska, A.K.: Obstacles to creating and finding open educational resources: the case of research methods in the social sciences. J. Interact. Media Educ. 2012(1):Art-5 (2012)
5.
Zurück zum Zitat Casali, A., Gerling, V., Deco, C., Bender, C.: A recommender system for learning objects personalized retrieval. In: Educational Recommender Systems and Technologies: Practices and Challenges, pp. 182–210. PA, Hershey, Information Science Reference (2012) Casali, A., Gerling, V., Deco, C., Bender, C.: A recommender system for learning objects personalized retrieval. In: Educational Recommender Systems and Technologies: Practices and Challenges, pp. 182–210. PA, Hershey, Information Science Reference (2012)
6.
Zurück zum Zitat Cechinel, C., Ochoa, X.: A brief overview of quality inside learning object repositories. In: Proceedings of the XV International Conference on Human Computer Interaction, p. 83. ACM (2014) Cechinel, C., Ochoa, X.: A brief overview of quality inside learning object repositories. In: Proceedings of the XV International Conference on Human Computer Interaction, p. 83. ACM (2014)
7.
Zurück zum Zitat Drachsler, H., Verbert, K., Santos, O., Manouselis, N.: Panorama of recommender systems to support learning. status: accepted (2015) Drachsler, H., Verbert, K., Santos, O., Manouselis, N.: Panorama of recommender systems to support learning. status: accepted (2015)
8.
Zurück zum Zitat Herlocker, J.L., Konstan, J.A., Terveen, L.G., Riedl, J.T.: Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. (TOIS) 22(1), 5–53 (2004)CrossRef Herlocker, J.L., Konstan, J.A., Terveen, L.G., Riedl, J.T.: Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. (TOIS) 22(1), 5–53 (2004)CrossRef
9.
Zurück zum Zitat Karampiperis, P., Koukourikos, A., Stoitsis, G.: Collaborative filtering recommendation of educational content in social environments utilizing sentiment analysis techniques. In: Manouselis, N., Drachsler, H., Verbert, K., Santos, O.C. (eds.) Recommender Systems for Technology Enhanced Learning, pp. 3–23. Springer, Heidelberg (2014) CrossRef Karampiperis, P., Koukourikos, A., Stoitsis, G.: Collaborative filtering recommendation of educational content in social environments utilizing sentiment analysis techniques. In: Manouselis, N., Drachsler, H., Verbert, K., Santos, O.C. (eds.) Recommender Systems for Technology Enhanced Learning, pp. 3–23. Springer, Heidelberg (2014) CrossRef
10.
Zurück zum Zitat Klerkx, J., Vandeputte, B., Parra, G., Santos, J.L., Van Assche, F., Duval, E.: How to share and reuse learning resources: the ARIADNE experience. In: Wolpers, M., Kirschner, P.A., Scheffel, M., Lindstaedt, S., Dimitrova, V. (eds.) EC-TEL 2010. LNCS, vol. 6383, pp. 183–196. Springer, Heidelberg (2010) CrossRef Klerkx, J., Vandeputte, B., Parra, G., Santos, J.L., Van Assche, F., Duval, E.: How to share and reuse learning resources: the ARIADNE experience. In: Wolpers, M., Kirschner, P.A., Scheffel, M., Lindstaedt, S., Dimitrova, V. (eds.) EC-TEL 2010. LNCS, vol. 6383, pp. 183–196. Springer, Heidelberg (2010) CrossRef
11.
Zurück zum Zitat Liang, C.: User profile for personalized web search. In: 2011 Eighth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), vol. 3, pp. 1847–1850. IEEE (2011) Liang, C.: User profile for personalized web search. In: 2011 Eighth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), vol. 3, pp. 1847–1850. IEEE (2011)
12.
Zurück zum Zitat Limongelli, C., Lombardi, M., Marani, A., Sciarrone, F.: A teacher model to speed up the process of building courses. In: Kurosu, M. (ed.) HCII/HCI 2013, Part II. LNCS, vol. 8005, pp. 434–443. Springer, Heidelberg (2013) Limongelli, C., Lombardi, M., Marani, A., Sciarrone, F.: A teacher model to speed up the process of building courses. In: Kurosu, M. (ed.) HCII/HCI 2013, Part II. LNCS, vol. 8005, pp. 434–443. Springer, Heidelberg (2013)
13.
Zurück zum Zitat Limongelli, C., Lombardi, M., Marani, A., Sciarrone, F.: A teaching-style based social network for didactic building and sharing. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 774–777. Springer, Heidelberg (2013) CrossRef Limongelli, C., Lombardi, M., Marani, A., Sciarrone, F.: A teaching-style based social network for didactic building and sharing. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 774–777. Springer, Heidelberg (2013) CrossRef
14.
Zurück zum Zitat Limongelli, C., Miola, A., Sciarrone, F., Temperini, M.: Supporting teachers to retrieve and select learning objects for personalized courses in the moodle_ls environment. In: 2012 IEEE 12th International Conference on Advanced Learning Technologies (ICALT), pp. 518–520. IEEE (2012) Limongelli, C., Miola, A., Sciarrone, F., Temperini, M.: Supporting teachers to retrieve and select learning objects for personalized courses in the moodle_ls environment. In: 2012 IEEE 12th International Conference on Advanced Learning Technologies (ICALT), pp. 518–520. IEEE (2012)
15.
Zurück zum Zitat Maloney, S., Moss, A., Keating, J., Kotsanas, G., Morgan, P.: Sharing teaching and learning resources: perceptions of a university’s faculty members. Med. Educ. 47(8), 811–819 (2013)CrossRef Maloney, S., Moss, A., Keating, J., Kotsanas, G., Morgan, P.: Sharing teaching and learning resources: perceptions of a university’s faculty members. Med. Educ. 47(8), 811–819 (2013)CrossRef
16.
Zurück zum Zitat Mangina, E., Kilbride, J.: Evaluation of keyphrase extraction algorithm and tiling process for a document/resource recommender within e-learning environments. Comput. Educ. 50(3), 807–820 (2008)CrossRef Mangina, E., Kilbride, J.: Evaluation of keyphrase extraction algorithm and tiling process for a document/resource recommender within e-learning environments. Comput. Educ. 50(3), 807–820 (2008)CrossRef
17.
Zurück zum Zitat Manouselis, N., Drachsler, H., Vuorikari, R., Hummel, H., Koper, R.: Recommender systems handbook. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems in Technology Enhanced Learning, pp. 387–415. Springer, Heidelberg (2011) Manouselis, N., Drachsler, H., Vuorikari, R., Hummel, H., Koper, R.: Recommender systems handbook. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems in Technology Enhanced Learning, pp. 387–415. Springer, Heidelberg (2011)
18.
Zurück zum Zitat Manouselis, N., Vuorikari, R., Assche, F.V.: Simulated analysis of maut collaborative filtering for learning object recommendation. In: Proceedings of the 1st Workshop on Social Information Retrieval for Technology Enhanced Learning, pp. 27–35 (2007) Manouselis, N., Vuorikari, R., Assche, F.V.: Simulated analysis of maut collaborative filtering for learning object recommendation. In: Proceedings of the 1st Workshop on Social Information Retrieval for Technology Enhanced Learning, pp. 27–35 (2007)
19.
Zurück zum Zitat Ochoa, X., Klerkx, J., Vandeputte, B., Duval, E.: On the use of learning object metadata: the globe experience. Towards Ubiquitous Learning, pp. 271–284. Springer, Heidelberg (2011) CrossRef Ochoa, X., Klerkx, J., Vandeputte, B., Duval, E.: On the use of learning object metadata: the globe experience. Towards Ubiquitous Learning, pp. 271–284. Springer, Heidelberg (2011) CrossRef
20.
Zurück zum Zitat Palavitsinis, N., Manouselis, N., Sanchez-Alonso, S.: Metadata quality in learning object repositories: a case study. Electron. Libr. 32(1), 62–82 (2014)CrossRef Palavitsinis, N., Manouselis, N., Sanchez-Alonso, S.: Metadata quality in learning object repositories: a case study. Electron. Libr. 32(1), 62–82 (2014)CrossRef
21.
Zurück zum Zitat Salehi, M.: Application of implicit and explicit attribute based collaborative filtering and bide for learning resource recommendation. Data Knowl. Eng. 87, 130–145 (2013)CrossRef Salehi, M.: Application of implicit and explicit attribute based collaborative filtering and bide for learning resource recommendation. Data Knowl. Eng. 87, 130–145 (2013)CrossRef
22.
Zurück zum Zitat Sergis, S., Sampson, D.: Learning object recommendations for teachers based on elicited ICT competence profiles. IEEE Trans. Learn. Technol. (2015) Sergis, S., Sampson, D.: Learning object recommendations for teachers based on elicited ICT competence profiles. IEEE Trans. Learn. Technol. (2015)
23.
Zurück zum Zitat Shani, G., Gunawardana, A.: Evaluating recommendation systems. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 257–297. Springer, Heidelberg (2011) CrossRef Shani, G., Gunawardana, A.: Evaluating recommendation systems. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 257–297. Springer, Heidelberg (2011) CrossRef
24.
Zurück zum Zitat Shulman, L.S.: Those who understand: knowledge growth in teaching. Educ. Res. 15(2), 4–14 (1986)CrossRef Shulman, L.S.: Those who understand: knowledge growth in teaching. Educ. Res. 15(2), 4–14 (1986)CrossRef
25.
Zurück zum Zitat Sicilia, M.Á., García-Barriocanal, E., Sánchez-Alonso, S., Cechinel, C.: Exploring user-based recommender results in large learning object repositories: the case of merlot. Procedia Comput. Sci. 1(2), 2859–2864 (2010)CrossRef Sicilia, M.Á., García-Barriocanal, E., Sánchez-Alonso, S., Cechinel, C.: Exploring user-based recommender results in large learning object repositories: the case of merlot. Procedia Comput. Sci. 1(2), 2859–2864 (2010)CrossRef
26.
Zurück zum Zitat Sosteric, M., Hesemeier, S.: When is a learning object not an object: a first step towards a theory of learning objects. Int. Rev. Res. Open Distance Learn. 3(2) (2002) Sosteric, M., Hesemeier, S.: When is a learning object not an object: a first step towards a theory of learning objects. Int. Rev. Res. Open Distance Learn. 3(2) (2002)
27.
Zurück zum Zitat Spink, V., Jansen, B.J.: Searching multimedia federated content web collections. Online Inf. Rev. 30(5), 485–495 (2006)CrossRef Spink, V., Jansen, B.J.: Searching multimedia federated content web collections. Online Inf. Rev. 30(5), 485–495 (2006)CrossRef
28.
Zurück zum Zitat Tumer, D., Shah, M.A., Bitirim, Y.: An empirical evaluation on semantic search performance of keyword-based and semantic search engines: Google, yahoo, msn and hakia. In: Fourth International Conference on Internet Monitoring and Protection, ICIMP 2009, pp. 51–55. IEEE (2009) Tumer, D., Shah, M.A., Bitirim, Y.: An empirical evaluation on semantic search performance of keyword-based and semantic search engines: Google, yahoo, msn and hakia. In: Fourth International Conference on Internet Monitoring and Protection, ICIMP 2009, pp. 51–55. IEEE (2009)
29.
Zurück zum Zitat Verbert, K., Manouselis, N., Ochoa, X., Wolpers, M., Drachsler, H., Bosnic, I., Duval, E.: Context-aware recommender systems for learning: a survey and future challenges. IEEE Trans. Learn. Technol. 5(4), 318–335 (2012)CrossRef Verbert, K., Manouselis, N., Ochoa, X., Wolpers, M., Drachsler, H., Bosnic, I., Duval, E.: Context-aware recommender systems for learning: a survey and future challenges. IEEE Trans. Learn. Technol. 5(4), 318–335 (2012)CrossRef
30.
Zurück zum Zitat Wan, X., Okamoto, T.: Utilizing learning process to improve recommender system for group learning support. Neural Comput. Appl. 20(5), 611–621 (2011)CrossRef Wan, X., Okamoto, T.: Utilizing learning process to improve recommender system for group learning support. Neural Comput. Appl. 20(5), 611–621 (2011)CrossRef
31.
Zurück zum Zitat Zapata, A., Menéndez, V.H., Prieto, M.E., Romero, C.: A framework for recommendation in learning object repositories: an example of application in civil engineering. Adv. Eng. Softw. 56, 1–14 (2013)CrossRef Zapata, A., Menéndez, V.H., Prieto, M.E., Romero, C.: A framework for recommendation in learning object repositories: an example of application in civil engineering. Adv. Eng. Softw. 56, 1–14 (2013)CrossRef
Metadaten
Titel
A Comparative Framework to Evaluate Recommender Systems in Technology Enhanced Learning: a Case Study
verfasst von
Matteo Lombardi
Alessandro Marani
Copyright-Jahr
2015
DOI
https://doi.org/10.1007/978-3-319-27101-9_11