Top

Journal of Computing in Higher Education

Published in:

09-09-2022

Using chatbots to support student goal setting and social presence in fully online activities: learner engagement and perceptions

Authors: Khe Foon Hew, Weijiao Huang, Jiahui Du, Chengyuan Jia

Published in: Journal of Computing in Higher Education | Issue 1/2023

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Although fully online learning is now the ‘new normal’ in many parts of the world, its implementation is often beset by challenges such as the lack of student self-regulation, and the sense of isolation. In this paper, we explored the use of chatbots to support student goal setting (Study 1) and social presence (Study 2) in online activities. In Study 1, participants in a fully online course were invited to complete a goal setting activity prior to attending class via a goal-setting chatbot. The chatbot engaged participants with five questions developed based on the SMART (specific, measurable, achievable, realistic, and timely) goal setting framework. In Study 2, English-as-Foreign-Language participants in a fully online course were tasked to complete listening practices. The learning buddy chatbot was designed based on the social presence framework (interpersonal communication, open communication, cohesive communication) to guide students through listening exercises. In both Study 1 and 2, we evaluated participants’ behavioral engagement by measuring their conversation records with the chatbots, as well as participants’ perceived usefulness and ease of use of the chatbots. We also gathered in-depth interview data concerning the participants’ perceptions of interacting with the chatbots. Overall, our findings found positive learner experiences with both chatbots with regard to the chatbots’ perceived usefulness and perceived ease of use. We also provided suggestions for instructors to apply chatbots in teaching and learning.

previous article Re-thinking the online distance instruction based on students’ feedback

next article Design and effects of the teacher-student interaction model in the online learning spaces

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Adam, M., Wessel, M., & Benlian, A. (2021). AI-based chatbots in customer service and their effects on user compliance. Electronic Markets, 31(2), 427–445. https://doi.org/10.1007/s12525-020-00414-7CrossRef

Azran, S. (2019). Google Dialogflow vs Microsoft LUIS vs IBM Watson Assistant – conversational AI comparison. https://medium.com/@samuel_22971/google-dialogflow-vs-microsoft-luis-vs-ibm-watson-assistant-conversational-ai-comparision-eb2d374f1413

Bączek, M., Zagańczyk-Bączek, M., Szpringer, M., Jaroszyński, A., & Wożakowska-Kapłon, B. (2021). Students’ perception of online learning during the COVID-19 pandemic. Medicine. https://doi.org/10.1097/MD.0000000000024821CrossRef

Barrot, J. S., Llenares, I. I., & del Rosario, L. S. (2021). Students’ online learning challenges during the pandemic and how they cope with them: The case of the Philippines. Education and Information Technologies, 26(6), 7321–7338. https://doi.org/10.1007/s10639-021-10589-xCrossRef

Bjerke, M. B., & Renger, R. (2017). Being smart about writing SMART objectives. Evaluation and Program Planning, 61, 125–127. https://doi.org/10.1016/j.evalprogplan.2016.12.009CrossRef

Cauldwell, R. (2018). A syllabus for listening: Decoding. Speech in Action.

Cavinato, A. G., Hunter, R. A., Ott, L. S., & Robinson, J. K. (2021). Promoting student interaction, engagement, and success in an online environment. Analytical and Bioanalytical Chemistry, 413, 1513–1520. https://doi.org/10.1007/s00216-021-03178-xCrossRef

Chametzky, B. (2021). Communication in online learning: Being meaningful and reducing isolation. In I. Management Association (Eds.), Research anthology on developing effective online learning courses (pp. 1184–1205). IGI Global. https://doi.org/10.4018/978-1-7998-8047-9.ch058

Chipchase, L., Davidson, M., Blackstock, F., Bye, R., Clothier, P., Klupp, N., Nickson, W., Turner, D., & Williams, M. (2017). Conceptualising and measuring student disengagement in higher education: A synthesis of the literature. International Journal of Higher Education, 6(2), 31–42. https://doi.org/10.5430/ijhe.v6n2p31CrossRef

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 318–339.CrossRef

Doran, G. T. (1981). There’s a SMART way to write management’s goals and objectives. Management Review, 70(11), 35–36.

Følstad, A., & Brandtzæg, P. B. (2017). Chatbots and the New World of HCI. Interactions, 24(4), 38–42.CrossRef

Følstad, A., & Taylor, C. (2020). Conversational repair in chatbots for customer service: The effect of expressing uncertainty and suggesting alternatives. Lecture Notes in Computer ScienceChatbot research and design. CONVERSATIONS 2019. (Vol. 11970). Springer. https://doi.org/10.1007/978-3-030-39540-7_14CrossRef

García Botero, G., Questier, F., & Zhu, C. (2019). Self-directed language learning in a mobile-assisted, out-of-class context: Do students walk the talk? Computer Assisted Language Learning, 32(1–2), 71–97.CrossRef

Garrison, D. R. (2011). E-learning in the 21st century: A framework for research and practice. Taylor & Francis.CrossRef

Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2), 87–105. https://doi.org/10.1016/s1096-7516(00)00016-6CrossRef

Gierdowski, D. C. (2019). ECAR study of undergraduate students and information technology. EDUCAUSE Center for Analysis and Research. https://library.educause.edu/resources/2019/10/2019-study-of-undergraduate-students-and-information-technology

Handoko, E., Gronseth, S. L., McNeil, S. G., Bonk, C. J., & Robin, B. R. (2019). Goal setting and MOOC completion: A study on the role of self-regulated learning in student performance in massive open online courses. The International Review of Research in Open and Distributed Learning. https://doi.org/10.19173/irrodl.v20i4.4270CrossRef

Hartley, K., & Bendixen, L. D. (2001). Educational research in the internet age: Examining the role of individual characteristics. Educational Researcher, 30(9), 22–26. https://doi.org/10.3102/0013189X030009022CrossRef

He, Y., Chen, Q., & Kitkuakul, S. (2018). Regulatory focus and technology acceptance: Perceived ease of use and usefulness as efficacy. Cogent Business & Management. https://doi.org/10.1080/23311975.2018.1459006CrossRef

Holzwarth, M., Janiszewski, C., & Neumann, M. M. (2006). The influence of avatars on online consumer shopping behavior. Journal of Marketing, 70(4), 19–36. https://doi.org/10.1509/jmkg.70.4.019CrossRef

Huang, W., Hew, K. F., & Fryer, L. K. (2022). Chatbots for language learning—Are they really useful? A systematic review of chatbot-supported language learning. Journal of Computer Assisted Learning, 38(1), 237–257. https://doi.org/10.1111/jcal.12610CrossRef

Insider Intelligence. (2021). Chatbot market in 2021: Stats, trends, and companies in the growing AI chatbot industry. Insider. Retrieved from https://www.businessinsider.com/chatbot-market-stats-trends

Jia, C., & Hew, K. F. (2022). Supporting lower-level processes in EFL listening: The effect on learners’ listening proficiency of a dictation program supported by a mobile instant messaging app. Computer Assisted Language Learning, 35(1–2), 141–168.CrossRef

Jia, C., Hew, K. F., Bai, S., & Huang, W. (2022). Adaptation of a conventional flipped course to an online flipped format during the Covid-19 pandemic: Student learning performance and engagement. Journal of Research on Technology in Education, 54(2), 281–301.CrossRef

Keller, J. M. (1987). Development and use of the ARCS model of instructional design. Journal of Instructional Development, 10(3), 2–10.CrossRef

Kennedy, C., & Levy, M. (2009). Sustainability and computer-assisted language learning: Factors for success in a context of change. Computer Assisted Language Learning, 22(5), 445–463.CrossRef

Lachmann, P., & Kiefel, A. (2012). Recommending learning activities as strategy for enabling self-regulated learning. In 2012 IEEE 12th international conference on advanced learning technologies (pp. 704–705). IEEE. https://doi.org/10.1109/ICALT.2012.222

Latham, G. P., & Locke, E. A. (1991). Self-regulation through goal setting. Organizational Behavior and Human Decision Process, 50(20), 212–247. https://doi.org/10.1016/0749-5978(91)90021-KCrossRef

LiveChat. (2022). Chatbot best practices. https://www.chatbot.com/chatbot-best-practices/

Mayer, R. E. (2017). Using multimedia for e-learning. Journal of Computer Assisted Learning, 33(5), 403–423.CrossRef

Mead, J. (2019). What session length can tell you about your chatbot performance. Inf. Commun. https://inform-comms.com/session-length-chatbot-performance

Moos, D. C., & Bonde, C. (2016). Flipping the classroom: Embedding self-regulated learning prompts in videos. Technology, Knowledge and Learning, 21(2), 225–242. https://doi.org/10.1007/s10758-015-9269-1CrossRef

Nass, C., & Moon, Y. (2000). Machines and mindlessness: Social responses to computers. Journal of Social Issues, 56(1), 81–103. https://doi.org/10.1111/0022-4537.00153CrossRef

Nayyar, D. A. (2019). Chatbots and the open source tools you can use to develop them. Open Source For You. Retrieved from https://opensourceforu.com/2019/01/chatbots-andthe-open-source-tools-you-can-use-to-develop-them/

Nushi, M., & Orouji, F. (2020). Investigating EFL teachers’ views on listening difficulties among their learners: The case of Iranian context. SAGE Open. https://doi.org/10.1177/2158244020917393CrossRef

Nussbaumer, A., Albert, D., & Kirschenmann, U. (2011). Technology-mediated support for self-regulated learning in open responsive learning environments. In 2011 IEEE global engineering education conference (EDUCON) (pp. 421–427). IEEE. https://doi.org/10.1109/EDUCON.2011.5773171

Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015). Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 42(5), 533–544.CrossRef

Pedrotti M., & Nistor N. (2019) How students fail to self-regulate their online learning experience. In M. Scheffel, J. Broisin, V. Pammer-Schindler, A. Ioannou, J. Schneider (Eds.), Transforming learning with meaningful technologies (Vol. 11722, pp. 377–385). EC-TEL 2019. Lecture Notes in Computer Science. Springer. https://doi.org/10.1007/978-3-030-29736-7_28

Phillips, C. (2018). Chatbot analytics 101: The essential metrics you need to track. Chatbots Magazine. https://chatbotsmagazine.com/chatbot-analytics-101-e73ba7013f00

Pomerantz. J., & Brooks, D. C. (2017). ECAR study of faculty and information technology. Research report. ECAR. https://library.educause.edu/-/media/files/library/2017/10/facultyitstudy2017.pdf

Reja, U., Manfreda, K. J., Hlebec, V., & Vehovar, V. (2003). Open-ended vs. close-ended questions in web questionnaires. Developments in Applied Statistics. http://mrvar.fdv.uni-lj.si/pub/mz/mz19/reja.pdf

Richardson, J. C., Maeda, Y., Lv, J., & Caskurlu, S. (2017). Social presence in relation to students’ satisfaction and learning in the online environment: A meta-analysis. Computers in Human Behavior, 71, 402–417. https://doi.org/10.1016/j.chb.2017.02.001CrossRef

Schwarzenberg, P., & Navon, J. (2020). Supporting goal setting in flipped classroom. Interactive Learning Environments, 28(6), 671–684. https://doi.org/10.1080/10494820.2019.1707691CrossRef

Shih, K. P., Chen, H. C., Chang, C. Y., & Kao, T. C. (2010). The development and implementation of scaffolding-based self-regulated learning system for e/m-learning. Educational Technology & Society, 13(1), 80–93.

Singh, J., Joesph, M. H., & Jabbar, K. B. A. (2019). Rule-based chabot for student enquiries. Journal of Physics: Conference Series, 1228(1), 012060.

Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook messenger. Computers & Education, 151, 103862. https://doi.org/10.1016/j.compedu.2020.103862CrossRef

Starr-Glass, D. (2020). Encouraging engagement: Video-conference augmentation of online distance learning environments. On the Horizon, 28(3), 125–132. https://doi.org/10.1108/oth-06-2020-0020CrossRef

Thorat, S. A., & Jadhav, V. (2020). A review on implementation issues of rule-based chatbot systems. In Proceedings of the international conference on innovative computing & communications (ICICC).

Tu, C.-H., & McIsaac, M. (2002). The relationship of social presence and interaction in online classes. American Journal of Distance Education, 16(3), 131–150. https://doi.org/10.1207/S15389286AJDE1603_2CrossRef

van Alten, D. C., Phielix, C., Janssen, J., & Kester, L. (2020). Self-regulated learning support in flipped learning videos enhances learning outcomes. Computers & Education, 158, 104000. https://doi.org/10.1016/j.compedu.2020.104000CrossRef

Weizenbaum, J. (1966). ELIZA—A computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36–45. https://doi.org/10.1145/365153.365168CrossRef

Whiteside, A., Dikkers, A., & Lewis, S. (2014). The power of social presence for learning. EDUCAUSE Review Online. Retrieved from https://er.educause.edu/articles/2014/5/the-power-of-social-presence-for-learning

Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., & Drachsler, H. (2021). Are We There Yet?—A systematic literature review on chatbots in education. Frontiers in Artificial Intelligence, 4, 654924. https://doi.org/10.3389/frai.2021.654924CrossRef

Wong, J., Baars, M., Davis, D., Van Der Zee, T., Houben, G. J., & Paas, F. (2019). Supporting self-regulated learning in online learning environments and MOOCs: A systematic review. International Journal of Human-Computer Interaction, 35(4–5), 356–373. https://doi.org/10.1080/10447318.2018.1543084CrossRef

Wong, J., Baars, M., He, M., de Koning, B. B., & Paas, F. (2021). Facilitating goal setting and planning to enhance online self-regulation of learning. Computers in Human Behavior, 124, 106913. https://doi.org/10.1016/j.chb.2021.106913CrossRef

Wut, T.-M., & Xu, J. (2021). Person-to-person interactions in online classroom settings under the impact of COVID-19: A social presence theory perspective. Asia Pacific Education Review, 22, 371–383. https://doi.org/10.1007/s12564-021-09673-1CrossRef

Xavier, M., & Meneses, J. (2020). Dropout in online higher education: A scoping review from 2014 to 2018. Barcelona: eLearn Center, Universitat Oberta de Catalunya. https://doi.org/10.7238/uoc.dropout.factors.2020

Yao, M. (2016). 5 metrics every chatbot developer needs to track. VentureBeat, The Machine. https://venturebeat.com/2016/10/04/5-metrics-every-chatbot-developer-needs-to-track/

Yılmaz, R., & Keser, H. (2017). The impact of interactive environment and metacognitive support on academic achievement and transactional distance in online learning. Journal of Educational Computing Research, 55(1), 95–122. https://doi.org/10.1177/0735633116656453CrossRef

Zimmerman, B. J. (2013). From cognitive modeling to self-regulation: A social cognitive career path. Educational Psychologist, 48(3), 135–147. https://doi.org/10.1080/00461520.2013.794676CrossRef

Zuckerman, A. (2020). IBM Watson vs Dialogflow comparison 2021. https://comparecamp.com/ibm-watson-vs-dialogflow-comparison/

Title: Using chatbots to support student goal setting and social presence in fully online activities: learner engagement and perceptions
Authors: Khe Foon Hew
Weijiao Huang
Jiahui Du
Chengyuan Jia
Publication date: 09-09-2022
Publisher: Springer US
Published in: Journal of Computing in Higher Education / Issue 1/2023
Print ISSN: 1042-1726
Electronic ISSN: 1867-1233
DOI: https://doi.org/10.1007/s12528-022-09338-x

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 1/2023

Re-thinking the online distance instruction based on students’ feedback

Examining key factors of beginner’s continuance intention in blended learning in higher education

Design and effects of the teacher-student interaction model in the online learning spaces

The relations between self-reported perceptions of learning environment, observational learning strategies, and academic outcome

Flipped classroom: motivational affordances of spherical video-based immersive virtual reality in support of pre-lecture individual learning in pre-service teacher education

Adapting Under Pressure: A Case Study in Scaling Faculty Development for Emergency Remote Teaching

Premium Partner