Allison Elliott Tew
Mark Guzdial
ABSTRACT
A primary goal of many CS education projects is to determine the extent to which a given intervention has had an impact on student learning. However, computing lacks valid assessments for pedagogical or research purposes. Without such valid assessments, it is difficult to accurately measure student learning or establish a relationship between the instructional setting and learning outcomes.
We developed the Foundational CS1 (FCS1) Assessment instrument, the first assessment instrument for introductory computer science concepts that is applicable across a variety of current pedagogies and programming languages. We applied methods from educational and psychological test development, adapting them as necessary to fit the disciplinary context. We conducted a large scale empirical study to demonstrate that pseudo-code was an appropriate mechanism for achieving programming language independence. Finally, we established the validity of the assessment using a multi-faceted argument, combining interview data, statistical analysis of results on the assessment, and CS1 exam scores.
- American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. Standards for educational and psychological testing. American Educational Research Association, Washington, DC, 1999.Google Scholar
- J. Cohen. Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates, Hillsdale, NJ, 2nd edition, 1988.Google Scholar
- A. M. Decker. How Students Measure Up: An Assessment Instrument for Introductory Computer Science. PhD thesis, University at Bu ffalo (SUNY), Buff alo, NY, 2007.Google Scholar
- K. Goldman, P. Gross, C. Heeren, G. Herman, L. Kaczmarczyk, M. C. Loui, and C. Zilles. Identifying important and difficult concepts in introductory computing courses using a Delphi process. In SIGCSE '08: Proceedings of the 39th ACM Technical Symposium on Computer Science Education, pages 256--260, 2008. Google ScholarDigital Library
- R. K. Hambleton, H. Swaminathan, and H. J. Rogers. Fundamentals of item response theory. Sage Publications, Newbury Park, CA, 1991.Google Scholar
- D. Hestenes, M. Wells, and G. Swackhamer. Force concept inventory. The Physics Teacher, 30:141--158, March 1992.Google ScholarCross Ref
- L. C. Kaczmarczyk, E. R. Petrick, J. P. East, and G. L. Herman. Identifying student misconceptions of programming. In SIGCSE '10: Proceedings of the 41st ACM Technical Symposium on Computer Science Education, 2010. Google ScholarDigital Library
- M. T. Kane. Validation. In R. L. Brennen, editor, Educational Measurement, pages 17--64. American Council on Education/Praeger Publishers, Westport, CT, 4th edition, 2006.Google Scholar
- J. C. Libarkin and S. Anderson. Assessment of learning in entry-level geoscience courses: Results from the geoscience concept inventory. Journal of Geoscience Education, 53:394--401, 2005.Google ScholarCross Ref
- M. D. Miller, R. L. Linn, and N. E. Gronlund. Validity. In Measurement and assessment in teaching, pages 80--99. Pearson Education, Upper Saddle River, NJ, 10th edition, 2009.Google Scholar
- P. A. Moss, B. J. Girard, and L. C. Haniford. Validity in Educational Assessment. Review of Research in Education, 30(1):109--162, 2006.Google Scholar
- A. E. Tew. Assessing fundamental introductory computing concept knowledge in a language independent manner. PhD thesis, Georgia Institute of Technology, Atlanta, GA, 2010. Google ScholarDigital Library
- A. E. Tew and M. Guzdial. Developing a validated assessment of fundamental CS1 concepts. In SIGCSE '10: Proceedings of the 41st ACM Technical Symposium on Computer Science education, 2010. Google ScholarDigital Library
Index Terms
- The FCS1: a language independent assessment of CS1 knowledge
Recommendations
Uses, Revisions, and the Future of Validated Assessments in Computing Education: A Case Study of the FCS1 and SCS1
ICER 2021: Proceedings of the 17th ACM Conference on International Computing Education ResearchValidated instruments of knowledge are important for creating an accepted and shared measurement for the most important part of education – student learning. The Force Concept of Inventory from Physics Education Research has had a significant impact ...
Replication, Validation, and Use of a Language Independent CS1 Knowledge Assessment
ICER '16: Proceedings of the 2016 ACM Conference on International Computing Education ResearchComputing education lags other discipline-based education research in the number and range of validated assessments available to the research community. Validated assessments are important for researchers to reduce experimental error due to flawed ...
Developing a validated assessment of fundamental CS1 concepts
SIGCSE '10: Proceedings of the 41st ACM technical symposium on Computer science educationPrevious studies of student programming ability have raised questions about students' ability to problem solve, read and analyze code, and understand introductory computing concepts. However, it is unclear whether these results are the product of ...
Comments