Top

Published in:

2021 | OriginalPaper | Chapter

Towards the Automatic Generation of Pedagogical Conversational Agents from Lecture Slides

Author : Matthias Wölfel

Published in: Multimedia Technology and Enhanced Learning

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Although corresponding technological and didactical models have been known for decades, the digitization of teaching has hardly advanced beyond simple non-interactive formats (e.g. downloadable slides are provided within a learning management system). The COVID-19 crisis is changing this situation dramatically, creating a high demand for highly interactive formats and fostering exchange between conversation partners about the course content. Systems are required that are able to communicate with students verbally, to answer their questions, and to check the students’ knowledge. While technological advances have made such systems possible in principle, the game stopper is the large amount of manual work and knowledge that must be put into designing such a system and feeding it the right content.

In this publication, we present a first system to overcome the aforementioned drawback by automatically generating a corresponding dialog system from slide-based presentations, such as PowerPoint, OpenOffice, or Keynote, which can be dynamically adapted to the respective students and their needs. Our first experiments confirm the proof of concept and reveal that such a system can be very handy for both respective groups, learners and lecturers, alike. The limitations of the developed system, however, also reminds us that many challenges need to be addressed to improve the feasibility and quality of such systems, in particular in the understanding of semantic knowledge.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Research on Remote Online Teaching Assistant System Based on Human-Computer Interaction

next chapter Research on the Fusion Pattern Recognition System Based on the Concept of Production Education Integration and Application of Generative Countermeasure Network

Amazon: Alexa in higher education (2020). https://aws.amazon.com/de/education/alexa-edu/higher-education/

Atapattu, T., Falkner, K., Falkner, N.: A comprehensive text analysis of lecture slides to generate concept maps. Comput. Educ. 115, 96–113 (2017)CrossRef

Bocklisch, T., Faulkner, J., Pawlowski, N., Nichol, A.: Rasa: open source language understanding and dialogue management. arXiv preprint arXiv:1712.05181 (2017)

Cer, D.M., et al.: Universal sentence encoder. arXiv abs/1803.11175 (2018)

Ch, D.R., Saha, S.K.: Automatic multiple choice question generation from text: a survey. IEEE Trans. Learn. Technol. 13(1), 14–25 (2020). https://doi.org/10.1109/TLT.2018.2889100

Damonte, M., Goel, R., Chung, T.: Practical semantic parsing for spoken language understanding. arXiv preprint arXiv:1903.04521 (2019)

D’Mello, S.K., Dowell, N., Graesser, A.: Does it really matter whether students’ contributions are spoken versus typed in an intelligent tutoring system with natural language? J. Exp. Psychol. Appl. 17(1), 1 (2011)CrossRef

Fiorella, L., Stull, A.T., Kuhlmann, S., Mayer, R.E.: Instructor presence in video lectures: the role of dynamic drawings, eye contact, and instructor visibility. J. Educ. Psychol. 111(7), 1162 (2019)CrossRef

Følstad, A., Skjuve, M., Brandtzaeg, P.B.: Different chatbots for different purposes: towards a typology of chatbots to understand interaction design. In: Bodrunova, S., et al. (eds) INSCI 2018. LNCS, vol. 11551, pp. 145–156. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-17705-8_13

10.

Fügen, C., et al.: Advances in lecture recognition: the ISL RT-06s evaluation system. In: Ninth International Conference on Spoken Language Processing (2006)

11.

Gašević, D., Dawson, S., Rogers, T., Gasevic, D.: Learning analytics should not promote one size fits all: the effects of instructional conditions in predicting academic success. Internet High. Educ. 28, 68–84 (2016)

12.

Graesser, A.C.: Conversations with autotutor help students learn. Int. J. Artif. Intell. Educ. 26(1), 124–132 (2016)CrossRef

13.

Hobert, S., Berens, F.: Small talk conversations and the long-term use of chatbots in educational settings – experiences from a field study. In: Følstad, A., et al. (eds.) CONVERSATIONS 2019. LNCS, vol. 11970, pp. 260–272. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-39540-7_18

14.

Hobert, S., Meyer von Wolff, R.: Say hello to your new automated tutor–a structured literature review on pedagogical conversational agents (2019)

15.

Jacob, L., Lachner, A., Scheiter, K.: Learning by explaining orally or in written form? Text complexity matters. Learn. Instr. 68, 101344 (2020)

16.

Kelsey, E., Ray, F., Brown, D., Robson, R.: Design of a domain-independent, interactive, dialogue-based tutor for use within the GIFT framework. In: Generalized Intelligent Framework for Tutoring (GIFT) Users Symposium (Giftsym3), pp. 161–168 (2015)

17.

Kim, K., Boelling, L., Haesler, S., Bailenson, J., Bruder, G., Welch, G.F.: Does a digital assistant need a body? The influence of visual embodiment and social behavior on the perception of intelligent virtual agents in ar. In: 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 105–114. IEEE (2018)

18.

Kolss, M., Wolfel, M., Kraft, F., Niehues, J., Paulik, M., Waibel, A.: Simultaneous german-English lecture translation. In: International Workshop on Spoken Language Translation (IWSLT) 2008 (2008)

19.

Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. arXiv preprint arXiv:1603.01360 (2016)

20.

Monz, C.: Machine learning for query formulation in question answering. Nat. Lang. Eng. 17(4), 425–454 (2011)CrossRef

21.

Pearl, C.: Designing Voice user Interfaces: Principles of Conversational Experiences. O’Reilly Media, Inc., Sebastopol (2016)

22.

Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., Zettlemoyer, L.: Deep contextualized word representations. arXiv preprint arXiv:1802.05365 (2018)

23.

Rus, V., D’Mello, S., Hu, X., Graesser, A.: Recent advances in intelligent systems with conversational dialogue. AI Mag. 34, 42–54 (2013)

24.

Schlobinski, P., Siever, T.: Sprachliche kommunikation in der digitalen welt. Eine repräsentative Umfrage, durchgeführt von forsa (1619–1021) (2018)

25.

Seufert, S., Meier, C., Soellner, M., Rietsche, R.: A pedagogical perspective on big data and learning analytics: a conceptual model for digital learning support. Technol. Knowl. Learn. 24(4), 599–619 (2019)CrossRef

26.

Wang, Y., Sumiya, K.: Semantic ranking of lecture slides based on conceptual relationship and presentational structure. Procedia Comput. Sci. 1(2), 2801–2810 (2010)CrossRef

27.

Wei, C., Yu, Z., Fong, S.: How to build a chatbot: chatbot framework and its capabilities. In: Proceedings of the 2018 10th International Conference on Machine Learning and Computing, pp. 369–373 (2018)

28.

Winkler, R., Söllner, M.: Towards empowering educators to create their own smart personal assistants. In: Proceedings of the 53rd Hawaii International Conference on System Sciences (2020)

29.

Wölfel, M.: Robust automatic transcription of lectures. KIT Scientific Publishing (2009)

30.

Wölfel, M., Schlippe, T., Stitz, A.: Voice driven type design. In: 2015 International Conference on Speech Technology and Human-Computer Dialogue (SpeD), pp. 1–9. IEEE (2015)

31.

Yang, Y., et al.: Multilingual universal sentence encoder for semantic retrieval (2019)

32.

Zellou, G., Cohn, M.: Social and functional pressures in vocal alignment: differences for human and voice-AI interlocutors (2020)

33.

Zhang, Y., Tuo, M., Yin, Q., Qi, L., Wang, X., Liu, T.: Keywords extraction with deep neural network model. Neurocomputing 383, 113–121 (2020)CrossRef

34.

Zhang, Z., Takanobu, R., Zhu, Q., Huang, M., Zhu, X.: Recent advances and challenges in task-oriented dialog systems. Sci. China Technol. Sci., 1–17 (2020)

Title: Towards the Automatic Generation of Pedagogical Conversational Agents from Lecture Slides
Author: Matthias Wölfel
Publisher: Springer International Publishing
Book: Multimedia Technology and Enhanced Learning
Print ISBN: 978-3-030-82564-5

Electronic ISBN: 978-3-030-82565-2

Copyright Year: 2021
DOI: https://doi.org/10.1007/978-3-030-82565-2_18

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner