nach oben

Educational Assessment, Evaluation and Accountability

Erschienen in:

11.09.2017

Does a teacher’s classroom observation rating vary across multiple classrooms?

verfasst von: Xiaoxuan Lei, Hongli Li, Audrey J. Leroux

Erschienen in: Educational Assessment, Evaluation and Accountability | Ausgabe 1/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Classroom observations have been increasingly used for teacher evaluations, and it is important to examine the measurement quality and the use of observation ratings. When a teacher is observed in multiple classrooms, his or her observation ratings may vary across classrooms. In that case, using ratings from one classroom per teacher may not be adequate to represent a teacher’s instructional quality. Drawing on the Measures of Effective Teaching (MET) dataset, this study examined the variation of a teacher’s classroom observation ratings across his or her multiple classrooms. The results indicate that the math classrooms accounted for 4.9 to 14.7% of the variance in the classroom observation ratings and English language arts (ELA) classrooms accounted for 6.7 to 15.5% of the variance in the ratings. The results of this study suggest that teachers’ multiple classrooms should be taken into consideration when classroom observation ratings are used to evaluate teachers in high-stakes settings.

Vorheriger Artikel Resistance to classroom observation in the context of teacher evaluation: teachers’ and department heads’ experiences and perspectives

Nächster Artikel Teachers’ and school administrators’ attitudes and beliefs of teacher evaluation: a preliminary investigation of high poverty school districts

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nur mit Berechtigung zugänglich

Abbott, M. L., & Fouts, J. T. (2003). Constructivist teaching and student achievement: The results of a school-level classroom observation study in Washington. Lynnwood, WA: Seattle Pacific University, Washington School Research Center.

Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48.CrossRef

Bell, C. A., Gitomer, D. H., McCaffrey, D. F., Hamre, B. K., Pianta, R. C., & Qi, Y. (2012). An argument approach to observation protocol validity. Educational Assessment, 17(2–3), 62–87.CrossRef

Beretvas, S. N. (2011). Cross-classified and multiple membership models. In J. J. Hox & J. K. Roberts (Eds.), Handbook of advanced multilevel analysis (pp. 313–344). New York: Routledge.

Berliner, D. C. (2014). Exogenous variables and value-added assessment: A fatal flaw. Teachers College Record, 116(1), 1–31.

Cadima, J., Peixoto, C., & Leal, T. (2014). Observed classroom quality in first grade: Associations with teacher, classroom, and school characteristics. European Journal of Psychology of Education, 29(1), 139–158.CrossRef

Chetty, R., Friedman, J. N., & Rockoff, J. E. (2011). The long-term impacts of teachers: Teacher value-added and student outcomes in adulthood (working paper no. no. w17699). Washington, DC: National Bureau of Economic Research.CrossRef

Darling-Hammond, L. (2000). Teacher quality and student achievement: a review of state policy evidence. Education Policy Analysis Archives, 8(1), 1–44.CrossRef

Darling-Hammond, L. (2010). Evaluating teacher effectiveness: How teacher performance assessments can measure and improve teaching? Washington, DC: Center for American Progress.

Darling-Hammond, L. (2012). Creating a comprehensive system for evaluating and supporting effective teaching. Stanford, CA: Stanford Center for Opportunity Policy in Education.

Evertson, C. M., Anderson, C. W., Anderson, L. M., & Brophy, J. E. (1980). Relationships between classroom behaviors and student outcomes in junior high mathematics and English classes. American Educational Research Journal, 17(1), 43–60.CrossRef

Every Student Succeeds Act of 2015, Pub. L. No. 114-95, § 114 Stat. 1177 (2015–2016).

Goe, L., Bell, C., & Little, O. (2008). Approaches to evaluating teacher effectiveness: a research synthesis. Washington, DC: National Comprehensive Center for Teacher Quality.

Goldstein, H. (2003). Multilevel statistical models (3rd ed.). New York, NY: Hodder Arnold.

Hansen, M., Lemke, M., & Sorensen, N. (2013). Combining multiple performance measures: do common approaches undermine districts’ personnel evaluation systems? Washington DC: American Institutes for Research.

Hattie, J. A. C. (2009). Visible learning: a synthesis of 800+ meta-analyses on achievement. Abingdon: Routledge.

Hattie, J. (2012). Visible learning for teachers: maximizing impact on learning. Abingdon: Routledge.

Hill, H. C., Charalambous, C. Y., Blazar, D., McGinn, D., Kraft, M. A., Beisiegel, M., et al. (2012). Validating arguments for observational instruments: attending to multiple sources of variation. Educational Assessment, 17(2–3), 88–106.CrossRef

Hox, J. J., Moerbeek, M., & van de Schoot, R. (2010). Multilevel analysis: techniques and applications. New York, NY: Routledge.

Hull, J. (2013). Trends in teacher evaluation: how states are measuring teacher performance. Alexandria, VA: Center for Public Education.

Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 17–64). Westport, CT: Praeger Publishers.

Kane, T. J., & Staiger, D. O. (2012). Gathering feedback for teaching: combining high-quality observations with student surveys and achievement gains. Seattle, WA: Bill & Melinda Gates Foundation.

Kelcey, B., McGinn, D., & Hill, H. (2013, March). Measurement of classroom teaching quality with Item Response Theory. Washington, DC: Paper presented at the Society for Research on Educational Effectiveness.

Kennedy, M. M. (2006). From teacher quality to quality teaching. Educational Leadership, 63(6), 14–19.

Kennedy, M. M. (2010). Attribution error and the quest for teacher quality. Educational Researcher, 39(8), 591–598.CrossRef

Leo, S. F., & Lachlan-Haché, L. (2012). Creating summative educator effectiveness scores: approaches to combining measures. Washington, DC: American Institutes for Research.

Lockwood, J. R., Louis, T. A., & McCaffrey, D. F. (2002). Uncertainty in rank estimation: implications for value-added modeling accountability systems. Journal of Educational and Behavioral Statistics, 27(3), 255–270.CrossRef

Luo, W., & Kwok, O. (2009). The impacts of ignoring a crossed factor in analyzing cross-classified data. Multivariate Behavioral Research, 44(2), 182–212.CrossRef

Marzano, R. J., & Toth, M. (2013). Teacher evaluation that makes a difference: a new model for teacher growth and student achievement. Alexandria, VA: Association for Supervision and Curriculum Development.

McGuinn, P. (2012). The state of teacher evaluation reform: state education agency capacity and the implementation of new teacher–evaluation systems. Washington, DC: Center for American Progress.CrossRef

Meyers, J. L., & Beretvas, S. N. (2006). The impact of inappropriate modeling of cross-classified data structures. Multivariate Behavioral Research, 41(4), 473–497.CrossRef

Mihaly, K., & McCaffrey, D. F. (2014). Grade-level variation in observational measures of teacher effectiveness. In T. J. Kane, K. A. Kerr, & R. C. Pianta (Eds.), Designing teacher evaluation systems: new guidance from the measures of effective teaching project (pp. 9–49). San Francisco, CA: Jossey-Bass.

Mihaly, K., McCaffrey, D. F., Staiger, D. O., & Lockwood, J. R. (2013). A composite estimator of effective teaching. Seattle, WA: Bill & Melinda Gates Foundation.

Morgan, G. B., Hodge, K. J., Trepinski, T. M., & Anderson, L. W. (2014). The stability of teacher performance and effectiveness: implications for policies concerning teacher evaluation. Education Policy Analysis Archives, 22(95), 1–21.

Murphy, D. L., & Beretvas, S. N. (2015). A comparison of teacher effectiveness measures calculated using three multilevel models for raters effects. Applied Measurement in Education, 28(3), 219–236.CrossRef

No Child Left Behind Act of 2001. (2002). Pub. L. No. 107–110, 115 Stat. 1425.

Park, Y. S., Chen, J., & Holtzman, S. L. (2014). Evaluating efforts to minimize rater bias in scoring classroom observations. In T. J. Kane, K. A. Kerr, & R. C. Pianta (Eds.), Designing teacher evaluation systems: new guidance from the measures of effective teaching project (pp. 381–414). San Francisco, CA: Jossey-Bass.

Partee, G. L. (2012). Using multiple evaluation measures to improve teacher effectiveness: state strategies from round 2 of no child left behind act waivers. Washington, DC: Center for American Progress.

Pianta, R. C., & Hamre, B. K. (2009). Conceptualization, measurement, and improvement of classroom processes: Standardized observation can leverage capacity. Educational Researcher, 38(2), 109–119.CrossRef

Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2008). The classroom assessment scoring system: K-3 manual. Baltimore, MD: Paul Brookes Publishing.

Polikoff, M. S. (2015). The stability of observational and student survey measures of teaching effectiveness. American Journal of Education, 121(2), 183–212.CrossRef

R Development Core Team. (2014). R: a language and environment for statistical computing (version 3.1.2) [computer software]. Vienna: R Foundation for Statistical Computing.

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: applications and data analysis methods. Thousand Oaks, CA: Sage.

Reform Support Network, U.S. Department of Education. (2013). Race to the top at a glance: Evaluations of teacher effectiveness: State requirements for classroom observations. Retrieved from https://www2.ed.gov/about/inits/ed/implementation-support-unit/tech-assist/evaluations-of-teacher-effectiveness.pdf.

Rockoff, J. (2004). The impact of individual teachers on student achievement: evidence from panel data. American Economic Review, 94(2), 247–252.CrossRef

Sanders, W. L., Wright, S. P., & Horn, S. P. (1997). Teacher and classroom context effects on student achievement: implications for teacher evaluation. Journal of Personnel Evaluation in Education, 11(1), 57–67.CrossRef

Smolkowski, K., & Gunn, B. (2012). Reliability and validity of the classroom observations of student-teacher interactions (COSTI) for kindergarten reading instruction. Early Childhood Research Quarterly, 27(2), 316–328.CrossRef

Staiger, D. O., & Rockoff, J. E. (2010). Searching for effective teachers with imperfect information. Journal of Economic Perspectives, 24(3), 97–117.CrossRef

Steele, J. L., Hamilton, L. S., & Stecher, B. (2010). Incorporating student performance measures into teacher evaluation systems. Santa Monica, CA: RAND Corporation.

Steinberg, M., & Garrett, R. (2016). Classroom composition and measured teacher performance: what do teacher observation scores really measure? Educational Evaluation and Policy Analysis, 38(2), 293–317.CrossRef

Theall, K. P., Scribner, R., Broyles, S., Yu, Q., Chotalia, J., Simonsen, N., et al. (2011). Impact of small group size on neighbourhood influences in multilevel models. Journal of Epidemiology and Community Health, 65(8), 688–695.CrossRef

U.S. Department of Education (2012). ESEA flexibility. Flexibility to improve student academic achievement and increase the quality of instruction. Retrieved from http://www2.ed.gov/policy/elsec/guid/esea-flexibility/index.html.

U.S. Department of Education (2013). The RESPECT Project: Envisioning a teaching profession for the 21st century (Vision Statement). Retrieved from http://www.ed.gov/teaching/national-conversation/vision.

Wallace, M. L. (2015). Modeling cross-classified data with and without the crossed factors’ random effects’ interaction (Unpublished doctoral dissertation). Austin, TX: University of Texas at Austin.

Waxman, H. C., & Huang, S. L. (1999). Classroom observation research and the improvement of teaching. In H. C. Waxman & H. J. Walberg (Eds.), New directions for teaching practice and research (pp. 107–129). Berkeley, CA: McCutchan.

White, M., & Rowan, B. (2013). User guide to measures of effective teaching longitudinal database (MET LDB). Ann Arbor, MI: Inter-university Consortium for Political and Social Research.

Whitehurst, G. J., Chingos, M. M., & Lindquist, K. M. (2014). Evaluating teachers with classroom observations: Lessons learned in four districts. Washington, DC: Brown Center on Education Policy at Brookings.

Titel: Does a teacher’s classroom observation rating vary across multiple classrooms?
verfasst von: Xiaoxuan Lei
Hongli Li
Audrey J. Leroux
Publikationsdatum: 11.09.2017
Verlag: Springer Netherlands
Erschienen in: Educational Assessment, Evaluation and Accountability / Ausgabe 1/2018
Print ISSN: 1874-8597
Elektronische ISSN: 1874-8600
DOI: https://doi.org/10.1007/s11092-017-9269-x

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 1/2018

Teacher evaluation: the need for valid measures and increased teacher involvement

Resistance to classroom observation in the context of teacher evaluation: teachers’ and department heads’ experiences and perspectives

Teachers’ and school administrators’ attitudes and beliefs of teacher evaluation: a preliminary investigation of high poverty school districts

Standards and assessment: coherence from the teacher’s perspective