Abstract
The present study examined the predictive validity of a classroom observation measure, the Classroom Strategies Assessment System (CSAS)-Observer Form, as a predictor of student performance on statewide tests of mathematics and English language arts. The CSAS is a teacher practice assessment that measures evidence-based instructional and behavioral management practices (Reddy and Dudek 2014). This study presents a sample of 35 teachers and 829 third through eighth grade students from six urban high poverty schools. Six school administrators trained to criterion on using the CSAS conducted three classroom observations for each teacher as part of their yearly evaluation process. Zero-order correlations revealed negative relationships between CSAS Rating Scale discrepancy scores (i.e., ∑ | recommended frequency-frequency ratings |) and mathematics and English language arts proficiency scores. A series of two-level hierarchical generalized linear models were fitted to data to assess whether CSAS Instructional Strategy and Behavioral Management Strategy Rating Scale Total and Composite discrepancy scores predicted statewide mathematics and English language arts proficiency scores. Results indicated that CSAS Total and Instructional and Behavior Management Composite discrepancy scores significantly predicted both mathematics and English language arts proficiency scores, suggesting that larger discrepancies on observer ratings of what teachers did versus what should have been done were associated with lower proficiency scores. Results offer evidence of the utility of measuring teacher practices via the CSAS to inform student achievement in high poverty settings. Implications for school psychological practice, teacher evaluation, and professional development are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Belfiore, P. J., Auld, R., & Lee, D. L. (2005). The disconnection in poor urban education: equal access and a pedagogy of risk-taking. Psychology in the Schools, 42, 855–863.
Boller, K., Atkins-Burnett, S., Malone, L. M., Baxter, G. P., & West, J. (2010). Compendium of Student, Teacher, and Classroom Measures Used in NCEE Evaluations of Educational Interventions. Volume I. Measures Selection Approaches and Compendium Development Methods (NCEE 2010-4012). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.
Borman, G. D., & Dowling, N. M. (2008). Teacher attrition and retention: a meta-analytic and narrative review of the research. Review of Educational Research, 78, 367–409.
Boyd, D., Grossman, P., Hammerness, K., Lankford, H., Loeb, S., Ronfeldt, M., & Wyckoff, J. (2012). Recruiting effective math teachers, evidence from New York City. American Educational Research Journal, 49, 1008–1047.
Brophy, J. E., & Good, T. (1986). Teacher behavior and student achievement. In M. C. Wittrock (Ed.), Handbook of research in teaching (3rd ed., pp. 328–375). New York: Macmillan.
Cantrell, S. (2013). Ensuring fair and reliable measures of effective teaching. Seattle: Bill and Melinda Gates Foundation.
Civic Impulse (2018). H.R. 1 — 111th Congress: American Recovery and Reinvestment Act of 2009. Retrieved from https://www.govtrack.us/congress/bills/111/hr1.
Coutinho, S. A., Wiemer-Hastings, K., Skowronski, J. J., & Britt, M. A. (2005). Metacognition, need for cognition and use of explanations during ongoing learning and problem solving. Learning and Individual Differences, 15, 321–337.
Creemers, B. P. M. (1994). The effective classroom. London: Cassell.
Darling-Hammond, L. (1993). Reframing the school reform agenda. Phi Delta Kappan, 74(10), 752–761.
Darling-Hammond, L. (1995). Restructuring Schools for Student Success. Daedalus, 124(4), 153–162.
Darling-Hammond, L. (2000). Teacher quality and student achievement: a review of state policy evidence. Educational Policy Analysis and Archives, 8, 1–44.
Darling-Hammond, L. (2003). Keeping good teachers: Why it matters, what leaders can do. Educational Leadership, 60(8), 6–13.
Darling-Hammond, L., Chung Wei, R., Andree, A., Richardson, N, & Orphanos, S. (2009). Professional learning in the learning profession: a status report on teacher development in the United States and abroad. National Staff Development Council. Retrieved from http://leaningforward.org/docs/pdf/nsdcstudy2009.pdf.
Dudek, C.M. (2017). A multi measure approach to classroom observational assessment: A preliminary investigation of concurrent validity. Paper presented at the annual meeting of the Association for Education Finance and Policy, Washington, D. C.
Elementary and Secondary Education Act, 20 U. S. C, 6313 (2001).
ESSA (2015). Every Student Succeeds Act of 2015, Pub. L. No. 114-95 § 114 Stat. 1177 (2015-2016).
Espinosa, L. M. (2005). Curriculum and assessment considerations for young children from culturally linguistically, and economically diverse backgrounds. Psychology in the Schools, 42, 837–853.
Espinosa, L., & Laffey, J. (2003). Urban primary teacher perceptions of children with challenging behaviors. Journal of Children and Poverty, 9(2), 136–156.
Gable, R. A., Hester, P. H., Rock, M. L., & Hughes, K. G. (2009). Back to basics: rules, praise, ignoring, and reprimands revisited. Intervention in School and Clinic, 44, 195–205.
Gage, N. L. (1978). The scientific basis of the art of teaching. New York: Teachers College Press.
Hart, B., & Risley, T. R. (1995). Meaningful differences in the everyday experience of young American children. Baltimore: Paul H. Brookes Publishing Company.
Hattie, J., Biggs, J., & Purdie, N. (1996). Effects of learning skills interventions on students learning: a meta-analysis. Review of Educational Research, 66, 99–136.
Kane, T. J., & Staiger, D. O. (2008). Estimating teacher impacts on student achievement: An experiment evaluation (NBER Working Paper No. 14607). Cambridge: National Bureau of Economic Research.
Kane, T. J., & Staiger, D. O. (2012). Gathering feedback for teaching: combining high-quality observations with student surveys and achievement gains (MET Research Paper). Seattle: Bill and Melinda Gates Foundation. Retrieved from http://www.metproject.org/downloads/MET_Gathering_Feedback_Research_Paper.pdf.
Krei, M. S. (1998). Intensifying the barriers: the problems of inequitable teacher allocation in low-income urban schools. Urban Education, 33(1), 71–94. https://doi.org/10.1177/0042085998033001005.
Lankford, H., Loeb, S., & Wyckoff, J. (2002). Teacher sorting and the plight of urban schools: a descriptive analysis. Education Evaluation and Policy Analysis.
Leigh, A. (2010). Estimating teacher effectiveness from two-year changes in students’ test scores. Economics of Education Review, 29, 480–488. https://doi.org/10.1016/j.econedurev.2009.10.010.
Lekwa, A. J., Reddy, L. A., Dudek, C., & Hua, A. (2018). Predictive validity of the Classroom Strategies Assessment System for gains in reading and mathematics. School Psychology Quarterly.
Marzano, R.J. (1998). A Theory-Based Meta-Analysis of Research on Instruction. Mid-continent Aurora, Colorado: Regional Educational Laboratory. Retrieved from http://eric.ed.gov/?id=ED427087.
Marzano, R. J., Pickering, D. J., & Pollock, J. E. (2001). Classroom instruction that works: research-based strategies for every teacher. Alexandria: Association for Supervision and Curriculum Development.
McDonald Connor, C., Son, S. H., Hindman, A., & Morrison, F. J. (2005). Teacher qualifications, classroom practices, and family characteristics: complex effects on first graders’ language and early reading. Journal of School Psychology, 43, 343–375. https://doi.org/10.1016/j.jsp.2005.06.001.
McFarland, J., Hussar, B., de Brey, C., Snyder, T., Wang, X., Wilkinson-Flicker, S., Gebrekristos, S., Zhang, J., Rathbun, A., Barmer, A., Bullock Mann, F., and Hinz, S. (2017). The condition of education 2017 (NCES 2017–144). U.S. Department of Education Washington, DC: National Center for Education Statistics Retrieved 3/2/2018 from https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2017144.
McLeskey, J., Billingsley, B. S., & McLeskey, J. (2008). How does the quality and stability of the teaching force influence the research to practice gap. Remedial and Special Education, 29(5), 293–305.
Mendro, R. L. (1998). Student achievement and school and teacher accountability. Journal of Personnel Evaluation in Education, 12(3), 257–267.
National Council on Teacher Quality (2015). State of states: evaluating teaching, leading, and learning. Washington DC: national Council on Teacher Quality. Retrieved from: http://www.nctq.org/dms/Stage/StageofStates2015.
Northwest Evaluation Association. (2011). Technical Manual for Measures of Academic Progress (MAP®) and Measures of Academic Progress for Primary Grades (MPG®). Portland: Author.
Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26, 237–257. https://doi.org/10.3102/01623737026003237.
Odden, A., Borman, G., & Fermanich, M. (2004). Assessing teacher, classroom, and school effects, including fiscal effects. Peabody Journal of Education, 79, 4–32. https://doi.org/10.1207/s15327930pje7904_2.
Pennington, K., & Mead, S. (2016). For good measure?: Teacher evaluation policy in the ESSA era. Washington, DC: Bellwether Education Partners.
Raudenbush, S., & Bryk, A. (2002). Hierarchical linear models: applications and data analysis methods (2nd ed.). Thousand Oaks: Sage.
Reddy, L. A., & Dudek, C. (2014). Teacher progress monitoring of instructional and behavioral management practices: an evidence-based approach to improving classroom practices. International Journal of School and Educational Psychology, 2, 71–84.
Reddy, L. A., Fabiano, G. A., & Dudek, C. M. (2012). Concurrent validity of the Classroom Strategies Scale for elementary school—Observer Form. Journal of Psychoeducational Assessment, 31, 258–270.
Reddy, L. A., Fabiano, G., Dudek, C., & Hsu, L. (2013a). Development and construct validation of the classroom strategies scale-observer form. School Psychology Quarterly, 28, 317–341.
Reddy, L. A., Fabiano, G., Dudek, C., & Hsu, L. (2013b). Predictive validity of the Classroom Strategies Scale-Observer Form on statewide testing. School Psychology Quarterly, 28, 301–316.
Reddy, L.A., Dudek, C.M., Peters, S., Alperin, A., Kettler, R.J., & Kurz, A. (2015). Educational Assessment, Evaluation, and Accountability, 30,47.
Reddy, L. A., Dudek, C. M., & Lekwa, A. (2017). Classroom strategies coaching model: integration of formative assessment and instructional coaching. Theory Into Practice, 56(1), 45–55.
Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrica, 73, 417–458. https://doi.org/10.1111/j.1468-0262.2005.00584.x.
Rockoff, J. E. (2004). The impact of individual teachers on student achievement: evidence from panel data. American Economic Review, 94, 247–252. https://doi.org/10.1257/0002828041302244.
Sanders, W. L., & Rivers, J. C. (1996). Cumulative and residual effects of teachers on future student academic achievement. Knoxville: YN University of Tennessee Value-Added Research and Assessment Center.
Santos, J. R. A. (1999). Cronbach’s alpha: a tool for assessing the reliability of scales. Journal of Extension, 37(2), 1–5.
Sawchuk, S. (2016). ESSA loosens reins on teacher evaluations, qualifications. Education Week. Retrieved from: http://www.edweek.org/ew/articles/2016/01/06/essa-loosens-reins-on-teacher-evaluations-qualifications.html?cmp=eml-enl-eu-news1.
Scheerens, J., & Bosker, R. (1997). The foundations of educational effectiveness. Oxford: Pergamon.
Shernoff, E. S., Lekwa, A. J., Reddy, L. A., Coccaro, C. (2017). Examining teachers’ attitudes and experiences with coaching to inform research-based practice: An iterative developmental design study. Journal of Educational and Psychological Consultation, 27, 459–485.
Smith, S. (2004). The effects of targeted positive behavior support strategies on preschoolers externalizing behavior. Unpublished doctoral dissertation, University of Missouri—Columbia.
Stichter, J. P., Lewis, T. J., Whittaker, T. A., Richter, M., Johnson, N., & Trussell, R. (2009). Assessing teacher use of opportunities to respond and effective classroom management strategies. Journal of Positive Behavior Interventions, 11, 68–81.
Stormont, M. (2007). Fostering resilience in your children at risk for failure: strategies for grades K-3. Upper Saddle River: Pearson.
Stormont, M., Covington, S., & Lewis, T. J. (2006). Using data to inform systems: assessing teacher implementation of key features of program-wide positive behavioral support in head start classrooms. Beyond Behavior, 15(3), 10–14.
Sugai, G., & Horner, R. (2002). The evolution of discipline practices: school-wide positive behavior supports. Child and Family Behavior Therapy, 24(1–2), 23–50. https://doi.org/10.1300/J019v24n01_03.
Sutherland, K. S., Alder, N., & Gunter, P. L. (2003). The effect of varying rates of opportunities to respond to academic requests on the classroom behavior of students with EBD. Journal of Emotional and Behavioral Disorders, 11, 239–248. https://doi.org/10.1177/10634266030110040501.
Tomlinson, C. A., & Edison, C. C. (2003). Differentiation in practice: a resource guide for differentiating curriculum, Grades K–5. Alexandria: Association for Supervision and Curriculum Development.
Walberg, H. J. (1986). Synthesis of research on teaching. In M. Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 214–229). New York: Macmillan.
Wang, M. C. (1991). Productive teaching and instruction: assessing the knowledge base. Phi Delta Kappan, 71, 470–478.
Wright, L., Sanders, S. P., & Horn, S. P. (1997). Teacher and classroom context effects on student achievement: implications for teacher evaluation. Journal of Personnel Evaluation in Education, 11(1), 57–67.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Ethical Approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Additional information
The current study was implemented as part of the School System Improvement (SSI) Project, a collaboration between multiple universities and charter schools funded by the US Department of Education’s Office of Innovation and Improvement as part of the Teacher Incentive Fund program (awarded to Rutgers, The State University of New Jersey; #S374A120060, PI Reddy, Kettler, & Kurz). The positions and opinions expressed in this article are solely those of the author.
Rights and permissions
About this article
Cite this article
Dudek, C.M., Reddy, L.A. & Lekwa, A. Measuring Teacher Practices to Inform Student Achievement in High Poverty Schools: a Predictive Validity Study. Contemp School Psychol 23, 290–303 (2019). https://doi.org/10.1007/s40688-018-0196-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40688-018-0196-8