Christopher Watson
ABSTRACT
Whilst working on an upcoming meta-analysis that synthesized fifty years of research on predictors of programming performance, we made an interesting discovery. Despite several studies citing a motivation for research as the high failure rates of introductory programming courses, to date, the majority of available evidence on this phenomenon is at best anecdotal in nature, and only a single study by Bennedsen and Caspersen has attempted to determine a worldwide pass rate of introductory programming courses.
In this paper, we answer the call for further substantial evidence on the CS1 failure rate phenomenon, by performing a systematic review of introductory programming literature, and a statistical analysis on pass rate data extracted from relevant articles. Pass rates describing the outcomes of 161 CS1 courses that ran in 15 different countries, across 51 institutions were extracted and analysed. An almost identical mean worldwide pass rate of 67.7% was found. Moderator analysis revealed significant, but perhaps not substantial differences in pass rates based upon: grade level, country, and class size. However, pass rates were found not to have significantly differed over time, or based upon the programming language taught in the course. This paper serves as a motivation for researchers of introductory programming education, and provides much needed quantitative evidence on the potential difficulties and failure rates of this course.
- T. Beaubouef and J. Mason. Why the high attrition rate for computer science students: some thoughts and observations. SIGCSE Bulletin, 37(2):103--106, 2005. Google Scholar
Digital Library
- J. Bennedsen and M. E. Caspersen. Failure rates in introductory programming. SIGCSE Bulletin, 39(2):32--36, 2007. Google ScholarDigital Library
- R. Bornat, S. Dehnadi, and . Simon. Mental models, consistency and programming aptitude. In Proc. Australasian Computing Education, pages 53--61, 2008. Google ScholarDigital Library
- K. B. Bruce. Controversy on how to teach cs 1: a discussion on the sigcse-members mailing list. SIGCSE Bulletin, 37(2):111--117, 2005. Google ScholarDigital Library
- D. Crookes. Educators call for reform in how programming is taught in schools, 2013. Retrieved Nov. 19, 2013 fromtextithttp://ind.pn/1evOJgl.Google Scholar
- A. Ehlert and C. Schulte. Comparison of oop first and oop later: first results regarding the role of comfort level. In Proc. ITiCSE, pages 108--112. ACM, 2010. Google ScholarDigital Library
- Higher Education Funding Council. Data about demand and supply in higher education subjects, 2013. Retrieved Jan. 5, 2013 fromtextithttp://bit.ly/19WRiLi.Google Scholar
- T. Jenkins. On the difficulty of learning to program. In Proceedings of the 3rd Annual Conference of the LTSN Centre for Information and Computer Sciences, volume 4, pages 53--58, 2002.Google Scholar
- C. McDowell, L. Werner, H. E. Bullock, and J. Fernald. Pair programming improves student retention, confidence, and program quality. Communications of the ACM, 49(8):90--95, 2006. Google ScholarDigital Library
- A. J. Mendes, L. Paquete, A. Cardoso, and A. Gomes. Increasing student commitment in introductory programming learning. In Proc. Frontiers in Education, pages 1--6. IEEE, 2012. Google ScholarDigital Library
- US Bureau of Labor Statistics. Computer and information technology occupations, 2013. Retrieved Nov. 19, 2013 fromtextithttp://1.usa.gov/1a63neI.Google Scholar
- C. Watson, F. W. Li, and J. L. Godwin. Bluefix: using crowd-sourced feedback to support programming students in error diagnosis and repair. In Proc. ICWL, pages 228--239. Springer, 2012. Google ScholarDigital Library
- C. Watson, F. W. Li, and J. L. Godwin. Predicting performance in an introductory programming course by logging and analyzing student programming behavior. In Proc. ICALT, pages 319--323. IEEE, 2013. Google ScholarDigital Library
- C. Watson, F. W. Li, and J. L. Godwin. No tests required: comparing traditional and dynamic predictors of programming success. In Proc. SIGCSE, pages 469--474. ACM, 2014. Google ScholarDigital Library
- C. Watson, F. W. Li, and R. W. Lau. Learning programming languages through corrective feedback and concept visualisation. In Proc. ICWL, pages 11--20. Springer, 2011. Google ScholarDigital Library
- S. Zweben. Computing degree and enrollment trends. Computing Research Association, 2011.Google Scholar
Index Terms
- Failure rates in introductory programming revisited
Recommendations
Pass Rates in Introductory Programming and in other STEM Disciplines
ITiCSE-WGR '19: Proceedings of the Working Group Reports on Innovation and Technology in Computer Science EducationVast numbers of publications in computing education begin with the premise that programming is hard to learn and hard to teach. Many papers note that failure rates in computing courses, and particularly in introductory programming courses, are higher ...
Natural language in introductory programming: an experimental study
ITiCSE '11: Proceedings of the 16th annual joint conference on Innovation and technology in computer science educationAlthough characterized as being "high level", classical programming languages such as Pascal and C have a grammar that is very different from natural language. In this research field, two main streams are noteworthy, one of them is characterized by an ...
A Survey of Introductory Programming Courses in Ireland
ITiCSE '19: Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science EducationBetween January and April of 2018, a comprehensive survey of introductory programming courses was undertaken across all sectors of Irish third-level institutions (universities, institutes of technology, and private colleges). The survey instrument was ...
Reviews
Andrew Brooks
A systematic review of introductory programming literature yielded data for 161 CS1 courses. An average worldwide pass rate of 67.7 percent was found. Pass rates, however, varied considerably, with a low of 23.1 percent and a high of 96 percent. No significant improvement in CS1 pass rates was found over time. While pass rates were found to vary considerably by country, the average pass rates of three of the four most sampled countries (USA, UK, and Australia) were not found to be statistically different. Though average pass rates for C and C++ courses were the poorest, at 61.1 percent and 56.2 percent respectively, they were not found to be statistically different from courses using other programming languages. Defining classes to be small if they had less than 30 students, the average pass rate of 80.1 percent for small classes was found to be statistically different from the average pass rate of 65.4 percent for larger classes. Also, the average pass rate of 79.9 percent for other educational institutions was found to be statistically different from the average pass rate of 66.4 percent for universities. Differences in admission standards are not addressed, nor is any attempt made to explain the lowest and highest pass rates observed. A graph charting class size against pass rate would have been useful. Though the authors suggest that small class sizes coupled with active learning approaches might be the best way to teach CS1, no analysis of pedagogy in the 161 CS1 courses examined is presented. This paper is recommended to computer science faculty. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments