Kristin Stephens-Martinez
Marti A. Hearst
Armando Fox
ABSTRACT
For an instructor who is teaching a massive open online course (MOOC), what is the best way to understand their class? What is the best way to view how the students are interacting with the content while the course is running? To help prepare for the next iteration, how should the course's data be best analyzed after the fact? How do these instructional monitoring needs differ between online courses with tens of thousands of students and courses with only tens? This paper reports the results of a survey of 92 MOOC instructors who answered questions about which information they find useful in their course, with the end goal of creating an information display for MOOC instructors.
The main findings are: (i) quantitative data sources such as grades, although useful, are not sufficient; understanding the activity in discussion forums and student surveys was rated useful for all use cases by a large majority of respondents, (ii) chat logs were not seen as useful, (iii) for the most part, the same sources of information were seen as useful as found in surveys of smaller online courses, (iv) mockups of existing and novel visualization techniques were responded to positively for use both while the course is running and for planning a revision of the course, and (v) a wide range of views was expressed about other details.
- Khan Academy coach demo. http://www.khanacademy.org/coach/demo.Google Scholar
- Aguilar, D. A. G., Theron, R., and Penalvo, F. J. G. Semantic spiral timelines used as support for e-learning. J. UCS 15, 7 (2009), 1526--1545.Google Scholar
- Breslow, L., Pritchard, D. E., DeBoer, J., Stump, G. S., Ho, A. D., and Seaton, D. Studying learning in the worldwide classroom: Research into edX's first MOOC. Research & Practice in Assessment 8 (2013), 13--25.Google Scholar
- Coetzee, D., Fox, A., Hearst, M. A., and Hartmann, B. Should your MOOC forum use a reputation system? In Proceedings of the 2014 Conference on Computer-Supported Cooperative Work (2014). Google ScholarDigital Library
- Cotton, K. Monitoring student learning in the classroom. Northwest Regional Educational Laboratory, 1988.Google Scholar
- Gaudioso, E., Hernandez-del-Olmo, F., and Montero, M. Enhancing e-learning through teacher support: two experiences. Education, IEEE Transactions on 52, 1 (2009), 109--115. Google ScholarDigital Library
- Gibbs,W. J., Olexa, V., and Bernas, R. S. A visualization tool for managing and studying online communications. Educational Technology & Society 9, 3 (2006), 232--243.Google Scholar
- Goldberg, M. W. Student participation and progress tracking for web-based courses using WebCT. In Proceedings of the Second International NA WEB Conference (1996), 5--8.Google Scholar
- Goldberg, M. W., Salari, S., and Swoboda, P. World wide webcourse tool: An environment for building www-based courses. Computer Networks and ISDN Systems 28, 7 (1996), 1219--1231. Google ScholarDigital Library
- Grover, S., Pea, R., and Cooper, S. Promoting active learning & leveraging dashboards for curriculum assessment in an OpenEdX introductory CS course for middle school. In Learning @ Scale, Work in Progress, ACM (2014). Google ScholarDigital Library
- Hardless, C., and Nulden, U. Visualizing learning activities to support tutors. In CHI'99 extended abstracts on Human factors in computing systems, ACM (1999), 312--313. Google ScholarDigital Library
- Hardy, J., Antonioletti, M., and Bates, S. e-learner tracking: Tools for discovering learner behavior. In The IASTED International Conference on Web-base Education (2004).Google Scholar
- Huang, J., Piech, C., Nguyen, A., and Guibas, L. Syntactic and functional variability of a million code submissions in a machine learning mooc. In AIED 2013 Workshops Proceedings Volume (2013), 25.Google Scholar
- Kizilcec, R. F., Piech, C., and Schneider, E. Deconstructing disengagement: analyzing learner subpopulations in massive open online courses. In Proceedings of the Third International Conference on Learning Analytics and Knowledge, ACM (2013), 170--179. Google ScholarDigital Library
- Mazza, R., and Dimitrova, V. Coursevis: Externalising student information to facilitate instructors in distance learning. In Proceedings of the International conference in Artificial Intelligence in Education, Sydney, Australia (2003), 117--129.Google Scholar
- Mazza, R., and Dimitrova, V. Informing the design of a course data visualisator: an empirical study. In 5th International Conference on New Educational Environments (ICNEE 2003) (2003), 215--220.Google Scholar
- Mazza, R., and Dimitrova, V. Coursevis: A graphical student monitoring tool for supporting instructors in web-based distance courses. International Journal of Human-Computer Studies 65, 2 (2007), 125--139. Google ScholarDigital Library
- Mazza, R., and Milani, C. Exploring usage analysis in learning systems: Gaining insights from visualisations.In AIED05 workshop on Usage analysis in learningsystems, Citeseer (2005), 65--72.Google Scholar
- Romero, C., and Ventura, S. Educational data mining: a review of the state of the art. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on 40, 6 (2010), 601--618. Google ScholarDigital Library
- Williams, F. P., and Conlan, O. Visualizing narrative structures and learning style information in personalized e-learning systems. In Advanced Learning Technologies, 2007. ICALT 2007. Seventh IEEE International Conference on, IEEE (2007), 872--876.Google ScholarCross Ref
- Zinn, C., and Scheuer, O. Getting to know your student in distance learning contexts. In Innovative Approaches for Learning and Knowledge Sharing. Springer, 2006, 437--451. Google ScholarDigital Library
Index Terms
- Monitoring MOOCs: which information sources do instructors value?
Recommendations
Understanding Student Motivation, Behaviors and Perceptions in MOOCs
CSCW '15: Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social ComputingMassive Open Online Courses (MOOCs) have recently experienced rapid development and garnered significant attention from various populations. Despite the wide recognition of MOOCs as an important opportunity within educational practices, there are still ...
Retention in MOOCs: Understanding Users' Motivations, Perceptions and Activity Trajectories
CHI EA '15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing SystemsMassive Open Online Courses (MOOCs) have experienced rapid development and achieved substantial attention from learners, educators and practitioners. Although MOOCs have been widely accepted, they have been plagued by tremendously high drop-out rate. ...
MOOCs and their impact on academics
Koli Calling '14: Proceedings of the 14th Koli Calling International Conference on Computing Education ResearchMassive Open Online Courses (MOOCs) have recently become a hot topic in the academic world, launching a wide ranging discussion on a number of issues. In this research, we surveyed academics' awareness, attitudes, perceptions, and experiences of MOOCs. ...
Comments