Top

Educational Assessment, Evaluation and Accountability

Published in:

24-02-2023

One size does not fit all: a concurrent analysis of the framework for teaching and the Classroom Strategies Assessment System

Authors: Christopher M. Dudek, Linda A. Reddy, Ryan J. Kettler

Published in: Educational Assessment, Evaluation and Accountability | Issue 3/2023

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

The present study examined the concurrent validity between two different classroom observational assessments, the Danielson Framework for Teaching (FFT: Danielson 2013) and the Classroom Strategies Assessment System (CSAS; Reddy & Dudek 2014). Twenty-four administrators from 8 high-poverty charter schools conducted 324 annual teacher evaluations for 221 Kindergarten through 12th grade teachers across three school years. Pearson correlations were computed between each measure’s calculated summative scores. Small to medium positive and two negative correlations (i.e., CSAS Vague Directives and Behavior Corrective Feedback) were found between FFT Domains 2 (Classroom Environment) and 3 (Instruction) and CSAS Strategy Counts. Negative medium to large correlations were found between FFT Domains 2 and 3 and CSAS Strategy Rating scale discrepancy scores. Overall, findings revealed that the FFT and CSAS scores offer conceptually similar and dissimilar constructs, and yield complementary information for educator evaluation practices. Implications for research, practice, and policy are presented.

previous article A Florida Teacher Wrongfully Terminated? Alleged (Mis)Uses of Value-Added Model (VAM) Estimates for High-Stakes Teacher Evaluation Decisions

next article Enacting teacher evaluation in Norwegian compulsory education: teachers’ perceptions of possibilities and constraints

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Alberto, P. A., & Troutman, A. C. (2006). Applied behavior analysis for teachers. Saddle River, NJ: Pearson Education.

American Educational Research Association on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testinng, American Psychological Association, National Council. (2014). Standards for educational and psychological testing. American Educational Research Association.

Blank, C., & Shavit, Y. (2016). The association between student reports of classmates’ disruptive behavior and student achievement. AERA Open, 2, 1–17.CrossRef

Borman, G. D., & Dowling, N. M. (2008). Teacher attrition and retention: A meta-analytic and narrative review of the research. Review of Educational Research, 78, 367–409.CrossRef

Boyd, D., Grossman, P., Hammerness, K., Lankford, H., Loeb, S., Ronfeldt, M., & Wyckoff, J. (2012). Recruiting effective math teachers, evidence from New York City. American Educational Research Journal, 49, 1008–1047.CrossRef

Brophy, J., & Good, T. (1986). Teacher behavior and student achievement. In M. Wittrock (Ed.), Handbook of research on teaching ((3rd ed) ed., pp. 328–375). Macmillan.

Cohen, J., & Goldhaber, D. (2016). Building a more complete understanding of teacher evaluation using classroom observations. Educational Researcher, 45(6), 378–387.CrossRef

Danielson, C. (2013). The framework for teaching evaluation instrument. The Danielson Group.

Danielson, C. (2007). Enhancing professional practice: A frame-work for teaching (2nd ed.). Association for Supervision and Curriculum Development.

Darling-Hammond, L., & Bransford, J. (2007). Preparing teachers for a changing world: What teachers should learn and be able to do. John Wiley & Sons.

Dee, T. S., & Wyckoff, J. (2015). Incentives, selection, and teacher performance: Evidence from IMPACT. Journal of Policy Analysis and Management, 34(2), 267–297.CrossRef

Dudek, C. M., Kettler, R. J., Reddy, L. A., & Kurz, A. (2019). Use of multiple assessments for measuring teacher practice: An initial investigation for educational practice. International Journal of Humanities and Social Science, 8, 2220–8488. https://doi.org/10.30845/ijhss.v8n10p2CrossRef

Flower, A., McKenna, J. W., & Haring, C. D. (2017). Behavior and classroom management: Are teacher preparation programs really preparing our teachers? Preventing School Failure: Alternative Education for Children and Youth, 61(2), 163–169.CrossRef

Freeman, J., Simonsen, B., Briere, D. E., & MacSuga-Gage, A. S. (2014). Pre-service teacher training in classroom management: A review of state accreditation policy and teacher preparation programs. Teacher Education and Special Education, 37(2), 106–120.CrossRef

Gallagher, H. A. (2004). Vaughn Elementary’s innovative teacher evaluation system: Are teacher evaluation scores related to growth in student achievement? Peabody Journal of Education, 79, 79–107.CrossRef

Garrett, R., & Steinberg, M. P. (2015). Examining teacher effectiveness using classroom observation scores: Evidence from the randomization of teachers to students. Educational Evaluation and Policy Analysis, 37, 224–242.CrossRef

Goe, L., Bell, C., & Little, O. (2008). Approaches to evaluating teacher effectiveness: A research synthesis. National Comprehensive Center for Teacher Quality.

Halpin, P. F., & Kieffer, M. J. (2015). Describing profiles of instructional practice: A new approach to analyzing classroom observation data. Educational Researcher, 44, 263–277.CrossRef

Hattie, J. (2009). Visible learning: A synthesis of meta-analyses relating to achievement. Routledge.

Hattie, J., Biggs, J., & Purdie, N. (1996). Effects of learning skills interventions on student learning: A meta-analysis. Review of Educational Research, 66, 99–136.CrossRef

Hill, H. C., Charalambous, C. Y., & Kraft, M. A. (2012). When rater reliability is not enough: Teacher observation systems and a case for the generalizability study. Educational Researcher, 41, 56–64.CrossRef

Ho, A. D., & Kane, T. J. (2013). The reliability of classroom observations by school personnel. Bill & Melinda Gates Foundation.

Holtzapple, E. (2003). Criterion-related validity evidence for a standards-based teacher evaluation system. Journal of Personnel Evaluation in Education, 17, 207–219.CrossRef

Isore, M. (2009). Teacher evaluation: Current practices in OECD countries and a literature review. In OECD Education Working Papers No. 23. OECD Publishing Retrieved from http://www.oecd.org/edu/ workingpapers

Jacob, B. A., & Lefgren, L. (2008). Can principals identify effective teachers? Evidence on subjective performance evaluation in education. Journal of Labor Economics, 1, 101–136. https://doi.org/10.1086/522974CrossRef

Jacob, B. A., & Lefgren, L. (2006). When principals rate teachers: The best and the worst stand out. Education Next, 2, 58–65 Retrieved from https://www.questia.com/library/journal/1G1-143817756/when-principals-rate-teachers-the-best-and-the-worst-stand

Jacob, B. A., & Lefgren, L. (2005). Principals as agents: Subjective performance measurement in education. In Working paper no. 11463, National Bureau of Economic Research. National Bureau of Economic Research.

Jin, L., & Cortazzi, M. (1998). Dimensions of dialogue: Large classes in China. International Journal of Educational Research, 29, 739–761.CrossRef

Jones, N. D., & Brownell, M. T. (2014). Assessing the viability of using classroom observations in the evaluation of special education teachers. Assessment for Effective Intervention, 39, 112–124.CrossRef

Kane, T. J., & Staiger, D. O. (2012). Gathering feedback for teaching: Combining high-quality observations with student surveys and achievement gains (MET Research Paper). Bill and Melinda Gates Foundation Retrieved from http://www.metproject.org/downloads/MET_Gathering_Feedback_Research_Paper.pdf

Ko, J., Sammons, P., & Bakkum, L. (2013). Effective teaching: A review of research and evidence. In Hong Kong: Hong Kong Institute of Education. CfBT Education Trust.

Kounin, J. S. (1970). Discipline and group management in classrooms. Holt, Rinehart & Winston.

Kimball, S., White, B., Milanowski, A., & Borman, G. (2004). Examining the relationship between teacher evaluation and student assessment results in Washoe County. Peabody Journal of Education, 79, 54–78.CrossRef

Lash, A., Tran, L., & Huang, M. (2016). Examining the validity of ratings from a classroom observation instrument for use in a district’s teacher evaluation system (REL 2016–135). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory West.

Lekwa, A. J., Reddy, L. A., & Shernoff, E. S. (2019). Measuring teacher practices and student academic engagement: A predictive validity study. School Psychology Quarterly, 34(1), 109–118. https://doi.org/10.1037/spq0000268CrossRef

Mantzicopoulos, P. Y., French, B. F., Patrick, H., Watson, J. S., & Ahn, I. (2018). The stability of kindergarten teachers’ effectiveness: A generalizability study comparing the Framework for Teaching and the Classroom Assessment Scoring System. Educational Assessment, 23, 24–46.CrossRef

McFarland, J., Hussar, B., de Brey, C., Snyder, T., Wang, X., Wilkinson-Flicker, S., Gebrekristos, S., Zhang, J., Rathbun, A., Barmer, A., Bullock Mann, F., & Hinz, S. (2017). The Condition of Education 2017 (NCES 2017-144). In Department of Education. National Center for Education Statistics Retrieved 1/22/2018 from https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2017144

Milanowski, A. T. (2011). Validity research on teacher evaluation systems based on the framework for teaching. Consortium for Policy Research in Education.

Milanowski, A. (2004). The relationship between teacher performance evaluation scores and student achievement: Evidence from Cincinnati. Peabody Journal of Education, 79(4), 33–53.CrossRef

Milanowski, A., Kimball, S. M., & White, B. (2004). The relationship between standards-based teacher evaluation scores and student achievement. University of Wisconsin-Madison, Consortium for Policy Research in Education.

Nelson, P., Reddy, L. A., Dudek, C. M., & Lekwa, A. (2017). Observer and student ratings of the class environment: A preliminary investigation of convergence. School Psychology Quarterly, 31(3), 1–15.

Neuman, S., & Cunningham, L. (2009). The impact of professional development and coaching on early language and literacy instructional practices. American Education Research Journal, 46(2), 532–566. https://doi.org/10.3102/0002831208328088CrossRef

Patrick, H., French, B. F., & Mantzicopoulos, P. (2020). The reliability of framework for teaching scores in Kindergarten. Journal of Psychoeducational Assessment, 38(7), 831–845. https://doi.org/10.1177/0734282920910843CrossRef

Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2008). Classroom Assessment Scoring System: Manual K-3. Paul J. Brookes.

Peterson, L. (1995). Stop and think learning: A teacher’s guide for motivating children to learn: including those with special needs. The Australian Council for Educational Research.

Praetorius, A., Lenske, G., & Helmke, A. (2012). Observer ratings of instructional quality: Do they fulfill what they promise? Learning and Instruction, 31, 2–12.CrossRef

Pratt, D., Kelly, M., & Wong, W. (1999). Chinese conceptions of effective teaching in Hong Kong: Toward culturally sensitive evaluation of teaching. International Journal of Lifelong Education, 18(4), 241–258.

Reddy, L. A., & Dudek, C. M. (2014). Teacher progress monitoring of instructional and behavioral management practices: An evidence-based approach to improving classroom practices. International Journal of Education and Psychology, 2, 71–84.

Reddy, L. A., Fabiano, G. A., & Dudek, C. M. (2013). Concurrent validity of the Classroom Strategies Scale for elementary school – Observer form. Journal of Psychoeducational Assessment, 31, 258–270.CrossRef

Reddy, L. A., Fabiano, G. A., Dudek, C. M., & Hsu, L. (2013a). Development and construct validity of the Classroom Strategies Scale– Observer Form. School Psychology Quarterly, 28, 317–341.CrossRef

Reddy, L. A., Fabiano, G. A., & Dudek C. M. &. Hsu L, (2013b). Predictive validity of the Classroom Strategies Scale-Observer form on statewide testing scores: A preliminary investigation. School Psychology Quarterly, 28, 301-316.CrossRef

Reddy, L. A., Lekwa, A., Dudek, C. M., Kettler, R. J., & Hua, A. (2020). Evaluation of teacher practices and student achievement in high-poverty schools. Journal of Psychoeducational Assessment., 38, 816–830. https://doi.org/10.1177/0734282920913394CrossRef

Reddy, L. A., Shernoff, E., & Lekwa, A. (2021). A randomized controlled trial of instructional coaching in high-poverty schools: Examining teacher practices and student outcomes. Journal of School Psychology, 86, 151–168. https://doi.org/10.1016/j.jsp.2021.04.001CrossRef

Rowe, K. (2006). Effective teaching practices for students with and without learning difficulties: Issues and implications. Australian Journal of Learning Disabilities, 11(3), 99–115.CrossRef

Sartain, L., Stoelinga, S. R., & Brown, E. R. (2011). Rethinking teacher evaluation in Chicago: Lessons learned from classroom observations, principal-teacher conferences, and district implementation (Research Report) (November). Consortium on Chicago School Research at the University of Chicago.

Sass, T. R., Hannaway, J., Xu, Z., Figlio, D. N., & Feng, L. (2012). Value added of teachers in high-poverty schools and lower poverty schools. Journal of Urban Economics, 72(2-3), 104–122.CrossRef

Steinberg, M. P., & Kraft, M. A. (2017). The sensitivity of teacher performance ratings to the design of teacher evaluation systems. Educational Researcher, 46(7), 378–396.CrossRef

Steinberg, M. P., & Sartain, L. (2015). Does teacher evaluation improve school performance? Experimental evidence from Chicago’s excellence in teacher project. Education Finance and Policy, 10(4), 535–572.CrossRef

Sutherland, K. S., Wehby, J. H., & Yoder, P. J. (2002). Examination of the relationship between teacher praise and opportunities for students with EBD to respond to academic requests. Journal of Emotional and Behavioral Disorders, 10(1), 13. https://doi.org/10.1177/106342660201000102CrossRef

Sutherland, K. S., & Oswald, D. P. (2005). The relationship between teacher and student behavior in classrooms for students with emotional and behavioral disorders: Transactional processes. Journal of Child and Family Studies, 14, 1–14. https://doi.org/10.1007/s10826-005-1106-zCrossRef

Tyler, J. H. (2011). Designing high quality evaluation systems for high school teachers. Center for American Progress.

Tyler, J. H., Taylor, E. S., Kane, T. J., & Wooten, A. L. (2010). Using student performance data to identify effective classroom practices. American Economic Review: Papers and Proceedings, 100, 256–260.CrossRef

Walberg, H. J. (1986). Syntheses of research on teaching. In M. Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 570–602). Macmillan.

Walker, H. M., Ramsay, E., & Gresham, F. M. (2004). Antisocial behavior in school: Evidence-based practices (2nd ed.). Wadsworth/Thomson Learning.

Walker, H. M., Ramsey, E., & Gresham, F. M. (2003). Heading off disruptive behavior: How early intervention can reduce defiant behavior and win back teaching time. American Educator, 26(4), 6–45.

Dudek, C. M., Reddy, L. A., & Lekwa, A. L. (2018). Measuring teacher practices to inform student achievement in high poverty schools: An initial predictive validity Study. Contemporary School Psychology. https://doi.org/10.1007/s40688-018-0196-8

Reddy, LA., Hua, A., Dudek, C.D. Kettler, R.J., Lekwa, A., Arnold-Berkovits, I., & Crouse, K. (2019). Use of observational measures to predict student achievement in high-poverty schools. Studies in Educator Evaluation, 62, 197-208. https://doi.org/10.1016/j.stueduc.2019.05.001

Title: One size does not fit all: a concurrent analysis of the framework for teaching and the Classroom Strategies Assessment System
Authors: Christopher M. Dudek
Linda A. Reddy
Ryan J. Kettler
Publication date: 24-02-2023
Publisher: Springer Netherlands
Published in: Educational Assessment, Evaluation and Accountability / Issue 3/2023
Print ISSN: 1874-8597
Electronic ISSN: 1874-8600
DOI: https://doi.org/10.1007/s11092-023-09405-6

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 3/2023

Theorising a contextual framework for moderation of internal assessment: development and opportunities

Analyses of quality assurance and development in education

Validation and invariance of the Conceptions of Assessment-III Abridged (COA-IIIA) among pre-service and in-service teachers in the United States

Enacting teacher evaluation in Norwegian compulsory education: teachers’ perceptions of possibilities and constraints

A Florida Teacher Wrongfully Terminated? Alleged (Mis)Uses of Value-Added Model (VAM) Estimates for High-Stakes Teacher Evaluation Decisions