Samiha Marwan
Veronica Catete
Thomas W. Price
Tiffany Barnes
ABSTRACT
Providing students with adaptive feedback on their progress on programming problems has been shown to motivate students and improve their performance, but little is known about how such feedback might impact student self-regulated learning during programming. Self-regulated learning (SRL) involves student planning a task, monitoring their progress, and reflecting on the outcome. We explored students’ SRL behaviors, particularly progress monitoring, when programming using each of three different scaffolds. The first scaffold is a subgoal checklist for the given programming task, the second adds automated, binary completion feedback on each subgoal, and the third adaptively reflects an automated percent progress estimate of student progress on each. Through interviews and programming logs from 17 students solving a problem in a block-based programming environment, we investigated the extent to which each scaffold supported student SRL. Our qualitative study results suggest that all three scaffolds could be useful for student SRL, but students felt that a combination of the checklist and progress feedback provided them with a balance of autonomy and motivation to persevere in programming. Furthermore, our results suggest that explaining how the automated feedback system works may have encouraged students to reason about the feedback they receive, which was a key intended outcome to improve SRL during programming.
- Michael Ball. 2018. Lambda: An Autograder for snap. Technical Report. Electrical Engineering and Computer Sciences University of California at Berkeley.Google Scholar
- Brett A. Becker, Kyle Goslin, and Graham Glanville. 2018. The Effects of Enhanced Compiler Error Messages on a Syntax Error Debugging Test. (2018).Google Scholar
- Robert Bodily, Judy Kay, Vincent Aleven, Ioana Jivet, Dan Davis, Franceska Xhakaj, and Katrien Verbert. 2018. Open learner models and learning analytics dashboards: a systematic review. In Proceedings of the 8th international conference on learning analytics and knowledge. 41–50.Google ScholarDigital Library
- Susan Bull, Abdallatif S Abu-Issa, Harpreet Ghag, and Tim Lloyd. 2005. Some Unusual Open Learner Models.. In AIED. 104–111.Google Scholar
- Michelene TH Chi. 2009. Active-constructive-interactive: A conceptual framework for differentiating learning activities. Topics in cognitive science 1, 1 (2009), 73–105.Google Scholar
- Lindy Crawford and Leanne R Ketterlin-Geller. 2008. Improving math programming for students at risk: Introduction to the special topic issue. Remedial and Special Education 29, 1 (2008), 5–8.Google ScholarCross Ref
- Wanda Dann, Dennis Cosgrove, Don Slater, Dave Culyba, and Steve Cooper. 2012. Mediated transfer: Alice 3 to java. In Proceedings of the 43rd ACM technical symposium on Computer Science Education. 141–146.Google ScholarDigital Library
- Barbara Di Eugenio, Davide Fossati, Stellan Ohlsson, and David Cosejo. 2009. Towards explaining effective tutorial dialogues. In Annual Meeting of the Cognitive Science Society. 1430–1435.Google Scholar
- Roberta E Dihoff, Gary M Brosvic, Michael L Epstein, and Michael J Cook. 2004. Provision of feedback during preparation for academic testing: Learning is enhanced by immediate but not delayed feedback. The Psychological Record 54, 2 (2004), 207–231.Google ScholarCross Ref
- Yihuan Dong, Samiha Marwan, Veronica Catete, Thomas W. Price, and Tiffany Barnes. 2019. Defining Tinkering Behavior in Open-ended Block-based Programming Assignments. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education. ACM, 1204–1210.Google ScholarDigital Library
- Katrina Falkner, Rebecca Vivian, and Nickolas JG Falkner. 2014. Identifying computer science self-regulated learning strategies. In Proceedings of the 2014 conference on Innovation & technology in computer science education. 291–296.Google ScholarDigital Library
- Davide Fossati, Barbara Di Eugenio, STELLAN Ohlsson, Christopher Brown, and Lin Chen. 2015. Data driven automatic feedback generation in the iList intelligent tutoring system. Technology, Instruction, Cognition and Learning 10, 1 (2015), 5–26.Google Scholar
- Dan Garcia, Brian Harvey, and Tiffany Barnes. 2015. The beauty and joy of computing. ACM Inroads 6, 4 (2015), 71–79.Google ScholarDigital Library
- Nuno Gil Fonseca, Luís Macedo, and António José Mendes. 2018. Supporting differentiated instruction in programming courses through permanent progress monitoring. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education. 209–214.Google ScholarDigital Library
- Jamie Gorson and Eleanor O’Rourke. 2020. Why Do CS1 Students Think They’re Bad at Programming? Investigating Self-Efficacy and Self-Assessments at Three Universities. In Proceedings of the 2020 ACM Conference on International Computing Education Research (Virtual Event, New Zealand) (ICER ’20). Association for Computing Machinery, New York, NY, USA, 170–181. https://doi.org/10.1145/3372782.3406273Google ScholarDigital Library
- Luke Gusukuma, Austin Cory Bart, Dennis Kafura, and Jeremy Ernst. 2018. Misconception-driven feedback: Results from an experimental study. In Proceedings of the 2018 ACM Conference on International Computing Education Research. 160–168.Google ScholarDigital Library
- Luke Gusukuma, Dennis Kafura, and Austin Cory Bart. 2017. Authoring feedback for novice programmers in a block-based language. In 2017 IEEE Blocks and Beyond Workshop (B&B). IEEE, 37–40.Google ScholarCross Ref
- I-H Hsiao, Sergey Sosnovsky, and Peter Brusilovsky. 2010. Guiding students to the right questions: adaptive navigation support in an E-Learning system for Java programming. Journal of Computer Assisted Learning 26, 4 (2010), 270–283.Google ScholarCross Ref
- David E Johnson. 2016. ITCH: Individual Testing of Computer Homework for Scratch Assignments. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education. ACM, New York, NY, 223–227.Google ScholarDigital Library
- Samad Kardan and Cristina Conati. 2015. Providing adaptive support in an interactive simulation for learning: An experimental evaluation. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, 3671–3680.Google ScholarDigital Library
- Vive Kumar, Philip Winne, Allyson Hadwin, John Nesbit, Dianne Jamieson-Noel, Tom Calvert, and Behzad Samin. 2005. Effects of self-regulated learning in programming. In Fifth IEEE International Conference on Advanced Learning Technologies (ICALT’05). IEEE, 383–387.Google ScholarDigital Library
- Dastyni Loksa and Andrew J Ko. 2016. The role of self-regulation in programming problem solving process and success. In Proceedings of the 2016 ACM conference on international computing education research. 83–91.Google ScholarDigital Library
- Dastyni Loksa, Andrew J Ko, Will Jernigan, Alannah Oleson, Christopher J Mendez, and Margaret M Burnett. 2016. Programming, problem solving, and self-awareness: Effects of explicit guidance. In Proceedings of the 2016 CHI conference on human factors in computing systems. 1449–1461.Google ScholarDigital Library
- Lauren Margulieux and Richard Catrambone. 2017. Using learners’ self-explanations of subgoals to guide initial problem solving in app inventor. In Proceedings of the 2017 ACM Conference on International Computing Education Research. 21–29.Google ScholarDigital Library
- Samiha Marwan, Anay Dombe, and Thomas W. Price. 2020. Unproductive Help-seeking in Programming: What it is and How to Address it?. In The Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science (ITiCSE’20). ACM, New York, NY.Google ScholarDigital Library
- Samiha Marwan, Ge Gao, Susan Fisk, Thomas W. Price, and Tiffany Barnes. 2020. Adaptive Immediate Feedback Can Improve Novice Programming Engagement and Intention to Persist in Computer Science. In Proceedings of the International Computing Education Research Conference (forthcoming).Google ScholarDigital Library
- Samiha Marwan, Joseph Jay Williams, and Thomas W. Price. 2019. An Evaluation of the Impact of Automated Programming Hints on Performance and Learning. In Proceedings of the 2019 ACM Conference on International Computing Education Research. ACM, 61–70.Google ScholarDigital Library
- Samiha Marwan, Nicholas Lytle, Joseph Jay Williams, and Thomas W. Price. 2019. The Impact of Adding Textual Explanations to Next-step Hints in a Novice Programming Environment. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education. ACM, 520–526.Google ScholarDigital Library
- Samiha Marwan, Thomas W Price, M. Chi, and Tiffany Barnes. 2020. Immediate Data-Driven Positive Feedback Increases Engagement on Programming Homework for Novices. In Educational Data Mining in Computer Science Education (CSEDM) Workshop @ EDM’20.Google Scholar
- Antonija Mitrovic and Brent Martin. 2007. Evaluating the effect of open student models on self-assessment. International Journal of Artificial Intelligence in Education 17, 2(2007), 121–144.Google ScholarDigital Library
- Antonija Mitrovic, Stellan Ohlsson, and Devon K Barrow. 2013. The effect of positive feedback in a constraint-based intelligent tutoring system. Computers & Education 60, 1 (2013), 264–272.Google ScholarDigital Library
- Roxana Moreno and Richard E Mayer. 1999. Cognitive principles of multimedia learning: The role of modality and contiguity.Journal of educational psychology 91, 2 (1999), 358.Google Scholar
- Alannah Oleson, Meron Solomon, and Amy J Ko. 2020. Computing Students’ Learning Difficulties in HCI Education. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 1–14.Google ScholarDigital Library
- Thomas W Price, Yihuan Dong, and Tiffany Barnes. 2016. Generating Data-Driven Hints for Open-Ended Programming.International Educational Data Mining Society (2016).Google Scholar
- Thomas W. Price, Yihuan Dong, and Dragan Lipovac. 2017. iSnap: Towards Intelligent Tutoring in Novice Programming Environments. In Proceedings of the ACM Technical Symposium on Computer Science Education. ACM, New York, NY.Google ScholarDigital Library
- Thomas W Price, Yihuan Dong, Rui Zhi, Benjamin Paaßen, Nicholas Lytle, Veronica Cateté, and Tiffany Barnes. 2019. A comparison of the quality of data-driven programming hint generation algorithms. International Journal of Artificial Intelligence in Education 29, 3(2019), 368–395.Google ScholarCross Ref
- Thomas W. Price, Zhongxiu Liu, Veronica Catete, and Tiffany Barnes. 2017. Factors Influencing Students’ Help-Seeking Behavior while Programming with Human and Computer Tutors. In Proceedings of the International Computing Education Research Conference. ACM, New York, NY.Google ScholarDigital Library
- Thomas W. Price, Rui Zhi, and Tiffany Barnes. 2017. Hint Generation Under Uncertainty: The Effect of Hint Quality on Help-Seeking Behavior. In Proceedings of the International Conference on Artificial Intelligence in Education.Google ScholarCross Ref
- R Keith Sawyer. 2005. The Cambridge handbook of the learning sciences. Cambridge University Press.Google Scholar
- Mary Catherine Scheeler, Kathy L Ruhl, and James K McAfee. 2004. Providing performance feedback to teachers: A review. Teacher education and special education 27, 4 (2004), 396–407.Google ScholarCross Ref
- Dale H. Schunk. 1995. Self-efficacy, motivation, and performance. Journal of Applied Sport Psychology 7, 2 (1995), 112–137. https://doi.org/10.1080/10413209508406961 arXiv:https://doi.org/10.1080/10413209508406961Google ScholarCross Ref
- Preya Shabrina, Samiha Marwan, Min Chi, Thomas W Price, and Tiffany Barnes. 2020. The Impact of Data-driven Positive Programming Feedback: When it Helps, What Happens when it Goes Wrong, and How Students Respond. In Educational Data Mining in Computer Science Education (CSEDM) Workshop @ EDM’20.Google Scholar
- Valerie J Shute. 2008. Focus on formative feedback. Review of educational research 78, 1 (2008), 153–189.Google Scholar
- Daniel Toll, Anna Wingkvist, and Morgan Ericsson. 2020. Current State and Next Steps on Automated Hints for Students Learning to Code. In 2020 IEEE Frontiers in Education Conference (FIE). IEEE, 1–5.Google ScholarDigital Library
- Wengran Wang, Rui Zhi, Alexandra Milliken, Nicholas Lytle, and Thomas W. Price. 2020. Crescendo: Engaging Students to Self-Paced Programming Practices. In To be published in the 51st ACM Technical Symposium on Computer Science Education (SIGCSE ’20). ACM, New York, NY.Google ScholarDigital Library
- Philip H Winne and Allyson F Hadwin. 2013. nStudy: Tracing and supporting self-regulated learning in the Internet. In International handbook of metacognition and learning technologies. Springer, 293–308.Google Scholar
Recommendations
Promoting self-regulated learning in web-based learning environments
Self-regulated learning with the Internet or hypermedia requires not only cognitive learning strategies, but also specific and general meta-cognitive strategies. The purposes of the Study2000 project, carried out at the TU Dresden, were to develop and ...
Work in progress: How do first-year engineering students develop as self-directed learners?
FIE '12: Proceedings of the 2012 IEEE Frontiers in Education Conference (FIE)Although self-direction is among the most critical skills required of today's engineering graduates, the complex processes through which individuals develop the attitudes, beliefs, and skills of lifelong, self-directed learners remains unclear. In this ...
Developing Web-based assessment strategies for facilitating junior high school students to perform self-regulated learning in an e-Learning environment
This research refers to the self-regulated learning strategies proposed by Pintrich (1999) in developing a multiple-choice Web-based assessment system, the Peer-Driven Assessment Module of the Web-based Assessment and Test Analysis system (PDA-WATA). ...
Comments