Rebecca Smith
ABSTRACT
The software development process often follows a circuitous path, littered with mistakes and backtracks. This is particularly true for novice programmers, who typically navigate through a variety of errors en route to their final solution. This paper presents a quantitative analysis of a large dataset of Python programs written by novice students. The analysis paints a multifaceted picture of the errors that students encounter, providing insight into the distribution, duration, and evolution of these errors. Ultimately, this paper aims to incite further conversation on the mistakes made by novice programmers, and to inform the decisions instructors make as they help students overcome these mistakes.
- Alireza Ahadi, Vahid Behbood, Arto Vihavainen, Julia Prior, and Raymond Lister. 2016. Students' Syntax Mistakes in Writing Seven Different Types of SQL Queries and its Applications to Predicting Students' Success. In Proceedings of the 47th Technical Symposium on Computer Science Education (SIGCSE '16). Memphis, TN. Google Scholar
Digital Library
- Alireza Ahadi, Raymond Lister, Shahil Lal, and Arto Hellas. 2018. Learning Programming, Syntax Errors and Institution-Specific Factors. In Proceedings of the 20th Australasian Computing Education Conference (ACE '18). Brisbane, Queensland, Australia. Google ScholarDigital Library
- Marzieh Ahmadzadeh, Dave Elliman, and Colin Higgins. 2005. An Analysis of Patterns of Debugging Among Novice Computer Science Students. In Proceedings of the 10th International Conference on Innovation and Technology in Computer Science Education (ITiCSE '05). Caparica, Portugal. Google ScholarDigital Library
- Amjad Altadmri and Neil C. C. Brown. 2015. 37 Million Compilations: Investigating Novice Programming Mistakes in Large-Scale Student Data. In Proceedings of the 46th Technical Symposium on Computer Science Education (SIGCSE '15). Kansas City, MO. Google ScholarDigital Library
- Brett A. Becker and Catherine Mooney. 2016. Categorizing Compiler Error Messages with Principal Component Analysis. In Proceedings of the 12th China-Europe International Symposium on Software Engineering Education (CEISEE '16). Shenyang, China.Google Scholar
- Jeff Bulmer, Angie Pinchbeck, and Bowen Hui. 2018. Visualizing Code Patterns in Novice Programmers. In Proceedings of the 23rd Western Canadian Conference on Computing Education (WCCCE '18). Victoria, BC, Canada. Google ScholarDigital Library
- Yuliya Cherenkova, Daniel Zingaro, and Andrew Petersen. 2014. Identifying Challenging CS1 Concepts in a Large Problem Dataset. In Proceedings of the 45th Technical Symposium on Computer Science Education (SIGCSE '14). Atlanta, GA. Google ScholarDigital Library
- Paul Denny, Andrew Luxton-Reilly, and Dave Carpenter. 2014. Enhancing Syntax Error Messages Appears Ineffectual. In Proceedings of the 19th International Conference on Innovation and Technology in Computer Science Education (ITiCSE '14). Uppsala, Sweden. Google ScholarDigital Library
- Paul Denny, Andrew Luxton-Reilly, and Ewan Tempero. 2012. All Syntax Errors Are Not Equal. In Proceedings of the 17th International Conference on Innovation and Technology in Computer Science Education (ITiCSE '12). Haifa, Israel. Google ScholarDigital Library
- Stephen H. Edwards, Nischel Kandru, and Mukund B. M. Rajagopal. 2017. Investigating Static Analysis Errors in Student Java Programs. In Proceedings of the 13th International Computing Education Conference (ICER '17). Tacoma, WA. Google ScholarDigital Library
- Andrew Ettles, Andrew Luxton-Reilly, and Paul Denny. 2018. Common Logic Errors Made by Novice Programmers. In Proceedings of the 20th Australasian Computing Education Conference (ACE '18). Brisbane, Queensland, Australia. Google ScholarDigital Library
- Juha Helminen, Petri Ihantola, and Ville Karavirta. 2013. Recording and Analyzing In-Browser Programming Sessions. In Proceedings of the 13th Koli Calling International Conference on Computing Education Research (Koli Calling '13). Koli, Finalnd. Google ScholarDigital Library
- James Jackson, Michael Cobb, and Curtis Carter. 2005. Identifying Top Java Errors for Novice Programmers. In Proceedings of the 35th Frontiers in Education Conference (FIE '05). Indianapolis, IN.Google ScholarCross Ref
- Matthew C. Jadud. 2005. A First Look at Novice Compilation Behavior Using BlueJ. Computer Science Education, Vol. 15, 1 (March 2005).Google ScholarCross Ref
- Sarah K. Kummerfeld and Judy Kay. 2003. The Neglected Battlefields of Syntax Errors. In Proceedings of the 5th Australasian Computing Education Conference (ACE '03). Adelaide, Australia. Google ScholarDigital Library
- Einari Kurvinen, Niko Hellgren, Erkki Kaila, Mikko-Jussi Laakso, and Tapio Salakoski. 2016. Programming Misconceptions in an Introductory Level Programming Course Exam. In Proceedings of the 21st International Conference on Innovation and Technology in Computer Science Education (ITiCSE '16). Arequipa, Peru. Google ScholarDigital Library
- Davin McCall and Michael Kolling. 2014. Meaningful Categorisation of Novice Programmer Errors. In Proceedings of the 44th Frontiers in Education Conference (FIE '14). Madrid, Spain.Google ScholarCross Ref
- Ioana Tuugalei Chan Mow. 2012. Analyses of Student Programming Errors in Java Programming Courses. Journal of Emerging Trends in Computing and Information Sciences, Vol. 3, 5 (May 2012).Google Scholar
- David Pritchard. 2015. Frequency Distribution of Error Messages. In Proceedings of the 6th Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU '15). Pittsburgh, PA. Google ScholarDigital Library
- Esther Shein. 2015. Python for Beginners. Commun. ACM, Vol. 58, 3 (Feb. 2015), 19--21. Google ScholarDigital Library
- Emily S. Tabano, Maria Mercedes T. Rodrigo, and Matthew C. Jadud. 2011. Predicting At-Risk Novice Java Programmers Through the Analysis of Online Protocols. In Proceedings of the 7th International Computing Education Conference (ICER '11). Providence, RI. Google ScholarDigital Library
- Terry Tang, Rebecca Smith, Scott Rixner, and Joe Warren. 2016. Data-Driven Test Case Generation for Automated Programming Assessment. In Proceedings of the 21st International Conference on Innovation and Technology in Computer Science Education (ITiCSE '16). Arequipa, Peru. Google ScholarDigital Library
- Arto Vihavainen, Juha Helminen, and Petri Ihantola. 2014. How Novices Tackle Their First Lines of Code in an IDE: Analysis of Programming Session Traces. In Proceedings of the 14th Koli Calling International Conference on Computing Education Research (Koli Calling '14). Koli, Finland. Google ScholarDigital Library
- Joe Warren, Scott Rixner, John Greiner, and Stephen Wong. 2014. Facilitating Human Interaction in an Online Programming Course. In Proceedings of the 45th Technical Symposium on Computer Science Education (SIGCSE '14). Atlanta, GA. Google ScholarDigital Library
Index Terms
- The Error Landscape: Characterizing the Mistakes of Novice Programmers
Recommendations
Compiler Error Messages Considered Unhelpful: The Landscape of Text-Based Programming Error Message Research
ITiCSE-WGR '19: Proceedings of the Working Group Reports on Innovation and Technology in Computer Science EducationDiagnostic messages generated by compilers and interpreters such as syntax error messages have been researched for over half of a century. Unfortunately, these messages which include error, warning, and run-time messages, present substantial difficulty ...
Teaching Students To Use Programming Error Messages
CompEd 2023: Proceedings of the ACM Conference on Global Computing Education Vol 2Research shows many students struggle to use programming error and warning messages effectively. Instead of using these messages as aids to debug and fix their code, some students have negative emotional reactions to seeing 'angry red text'. Not ...
Applying Software Engineering Anti-patterns to Programming Error Messages
SIGCSE 2023: Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 2Programming error messages (PEMs) have long been a hindrance to novice programmers. This work aims to establish a catalog of PEM anti-patterns--- common, reoccurring features of PEMs that make them unhelpful or actively harmful to programmers. The goal ...
Comments