Vilma Kangas
Juho Leinonen
ABSTRACT
We have constructed a tool, CrowdSorcerer, in which students create programming assignments, their model solutions and associated test cases using a simple input-output format. We have used the tool as a part of an introductory programming course with normal course activities such as programming assignments and a final exam. In our work, we focus on whether creating programming assignments and associated tests correlate with students' performance in a testing-related exam question. We study this through an analysis of the quality of student-written tests within the tool, measured using the number of test cases, line coverage and mutation coverage, and students' performance in testing related exam question, measured using exam points. Finally, we study whether previous programming experience correlates with how students act within the tool and within the testing related exam question.
- Elena Garc'ia Barriocanal, Miguel-Ángel Sicilia Urbán, Ignacio Aedo Cuevas, and Paloma Diaz Pérez. 2002. An experience in integrating automated unit testing practices in an introductory programming course. ACM SIGCSE Bulletin, Vol. 34, 4 (2002), 125--128. Google Scholar
Digital Library
- Susan Bergin and Ronan Reilly. 2005. Programming: Factors that Influence Success. In SIGCSE '05 Proceedings of the 36th SIGCSE technical symposium on Computer science education. ACM, 411--415. http://eprints.maynoothuniversity.ie/8209/ Google ScholarDigital Library
- Barry W. Boehm. 1984. Software Engineering Economics. IEEE Trans. Softw. Eng., Vol. 10, 1 (Jan. 1984), 4--21. Google ScholarDigital Library
- David Carrington. 1996. Teaching Software Testing. In Proceedings of the 2Nd Australasian Conference on Computer Science Education (ACSE '97). ACM, New York, NY, USA, 59--64. Google ScholarDigital Library
- Ruth C Clark, Frank Nguyen, and John Sweller. 2006. Efficiency in learning: Evidence-based guidelines to manage cognitive load .Pfeiffer, John Wiley & Sons.Google Scholar
- Benjamin S. Clegg, José Miguel Rojas, and Gordon Fraser. 2017. Teaching Software Testing Concepts Using a Mutation Testing Game. In Proceedings of the 39th International Conference on Software Engineering: Software Engineering and Education Track (ICSE-SEET '17). IEEE Press, Piscataway, NJ, USA, 33--36. Google ScholarDigital Library
- T. Cowling. 2012. Stages in teaching software testing. In 2012 34th International Conference on Software Engineering (ICSE). 1185--1194. Google ScholarDigital Library
- Paul Denny, Diana Cukierman, and Jonathan Bhaskar. 2015. Measuring the Effect of Inventing Practice Exercises on Learning in an Introductory Programming Course. In Proceedings of the 15th Koli Calling Conference on Computing Education Research (Koli Calling '15). ACM, New York, NY, USA, 13--22. Google ScholarDigital Library
- Paul Denny, Andrew Luxton-Reilly, Ewan Tempero, and Jacob Hendrickx. 2011. CodeWrite: Supporting Student-driven Practice of Java. In Proceedings of the 42Nd ACM Technical Symposium on Computer Science Education (SIGCSE '11). ACM, New York, NY, USA, 471--476. Google ScholarDigital Library
- Stephen H. Edwards. 2003. Improving Student Performance by Evaluating How Well Students Test Their Own Programs. J. Educ. Resour. Comput., Vol. 3, 3, Article 1 (Sept. 2003). Google ScholarDigital Library
- Stephen H. Edwards and Zalia Shams. 2014. Comparing Test Quality Measures for Assessing Student-written Tests. In Companion Proceedings of the 36th International Conference on Software Engineering (ICSE Companion 2014). ACM, New York, NY, USA, 354--363. Google ScholarDigital Library
- Dianne Hagan and Selby Markham. 2000. Does It Help to Have Some Programming Experience Before Beginning a Computing Degree Program?. In Proceedings of the 5th Annual SIGCSE/SIGCUE ITiCSE conference on Innovation and Technology in Computer Science Education (ITiCSE '00). ACM, New York, NY, USA, 25--28. Google ScholarDigital Library
- David Janzen and Hossein Saiedian. 2008. Test-driven Learning in Early Programming Courses. In Proceedings of the 39th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '08). ACM, New York, NY, USA, 532--536. Google ScholarDigital Library
- Vesa Lappalainen, Jonne Itkonen, Ville Isomöttönen, and Sami Kollanus. 2010. ComTest: a tool to impart TDD and unit testing to introductory level programming. In Proceedings of the fifteenth annual conference on Innovation and technology in computer science education. ACM, 63--67. Google ScholarDigital Library
- Andrew Luxton-Reilly, Daniel Bertinshaw, Paul Denny, Beryl Plimmer, and Robert Sheehan. 2012a. The Impact of Question Generation Activities on Performance. In Proceedings of the 43rd ACM Technical Symposium on Computer Science Education (SIGCSE '12). ACM, New York, NY, USA, 391--396. Google ScholarDigital Library
- Andrew Luxton-Reilly, Paul Denny, Beryl Plimmer, and Robert Sheehan. 2012b. Activities, Affordances and Attitude: How Student-generated Questions Assist Learning. In Proceedings of the 17th ACM Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE '12). ACM, New York, NY, USA, 4--9. Google ScholarDigital Library
- Will Marrero and Amber Settle. 2005. Testing First: Emphasizing Testing in Early Programming Courses. In Proceedings of the 10th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education (ITiCSE '05). ACM, New York, NY, USA, 4--8. Google ScholarDigital Library
- Glenford J. Myers. 1979. Art of Software Testing .John Wiley & Sons, Inc., New York, NY, USA. Google ScholarDigital Library
- Nea Pirttinen, Vilma Kangas, Irene Nikkarinen, Henrik Nygren, Juho Leinonen, and Arto Hellas. 2018a. Crowdsourcing Programming Assignments with CrowdSorcerer. In Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE 2018). ACM, New York, NY, USA, 326--331. Google ScholarDigital Library
- Nea Pirttinen, Vilma Kangas, Henrik Nygren, Juho Leinonen, and Arto Hellas. 2018b. Analysis of Students' Peer Reviews to Crowdsourced Programming Assignments. In Proceedings of the 18th Koli Calling International Conference on Computing Education Research. ACM. Google ScholarDigital Library
- Christopher Watson, Frederick W.B. Li, and Jamie L. Godwin. 2014. No Tests Required: Comparing Traditional and Dynamic Predictors of Programming Success. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education (SIGCSE '14). ACM, New York, NY, USA, 469--474. Google ScholarDigital Library
- Susan Wiedenbeck, Deborah Labelle, and Vennila N. R. Kain. 2004. Factors Affecting Course Outcomes in Introductory Programming. (05 2004), 97--110.Google Scholar
- Brenda Cantwell Wilson and Sharon Shrock. 2001. Contributing to Success in an Introductory Computer Science Course: A Study of Twelve Factors. In Proceedings of the Thirty-second SIGCSE Technical Symposium on Computer Science Education (SIGCSE '01). ACM, New York, NY, USA, 184--188. Google ScholarDigital Library
Index Terms
- Does Creating Programming Assignments with Tests Lead to Improved Performance in Writing Unit Tests?
Recommendations
Crowdsourcing programming assignments with CrowdSorcerer
ITiCSE 2018: Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science EducationSmall automatically assessed programming assignments are an often used resource for learning programming. Creating sufficiently large amounts of such assignments is, however, time consuming. As a consequence, offering large quantities of practice ...
Extracting Inline Tests from Unit Tests
ISSTA 2023: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and AnalysisWe recently proposed inline tests for validating individual program statements; they allow developers to provide test inputs, expected outputs, and test oracles immediately after a target statement. But, existing code can have many target statements. ...
Retrofitting unit tests for parameterized unit testing
FASE'11/ETAPS'11: Proceedings of the 14th international conference on Fundamental approaches to software engineering: part of the joint European conferences on theory and practice of softwareRecent advances in software testing introduced parameterized unit tests (PUT), which accept parameters, unlike conventional unit tests (CUT), which do not accept parameters. PUTs are more beneficial than CUTs with regards to fault-detection capability, ...
Comments