Top

Educational Assessment, Evaluation and Accountability

Published in:

13-02-2019

Reframing conversations about teacher quality: school and district administrators’ perceptions of the validity, reliability, and justifiability of a new teacher evaluation system

Authors: Noelle A. Paufler, Chris Clark

Published in: Educational Assessment, Evaluation and Accountability | Issue 1/2019

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

In changing accountability contexts, policymakers are engaging in international dialogue and collaborative efforts with new opportunities to reframe conversations about how to measure teacher quality and to (re) design and implement evaluation systems accordingly to ensure that they are fair, useable, feasible, and accurate. This study examined the lived experiences of school and district administrators in a large, fast-growth, suburban district in the USA regarding their districts’ new teacher evaluation system to better understand their perceptions of the system’s validity and reliability such that justifiable conclusions may be drawn about teachers’ effectiveness. Given concerns regarding validity and reliability, administrators generally discouraged external, high-stakes uses of evaluation results but valued the evaluation process and the data it provides for supporting teacher growth. As part of a larger study including teachers, findings can inform policymakers seeking to reform teacher evaluation frameworks to emphasize professional growth over high-stakes consequences.

previous article Underperforming teachers: the impact on co-workers and their responses

next article Classroom observation systems in context: A case for the validation of observation systems

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Adnot, M., Dee, T., Katz, V., & Wyckoff, J. (2017). Teacher turnover, teacher quality, and student achievement in DCPS. Educational Evaluation and Policy Analysis, 39(1), 54–76. https://doi.org/10.3102/0162373716663646.CrossRef

Amrein-Beardsley, A. (2008). Methodological concerns about the education value-added assessment system. Educational Researcher, 37(2), 65–75. https://doi.org/10.3102/0013189X08316420.CrossRef

Amrein-Beardsley, A. (2014). Rethinking value-added models in education: critical perspectives on tests and assessment-based accountability. New York, NY: Routledge.

Amrein-Beardsley, A., & Collins, C. (2012). The SAS education value-added assessment system (SAS® EVAAS®) in the Houston Independent School District (HISD): intended and unintended consequences. Education Policy Analysis Archives, 20. https://doi.org/10.14507/epaa.v20n12.2012.

Baker, E. L., Barton, P. E., Darling-Hammond, L., Haertel, E., Ladd, H. F., Linn, R. L., et al. (2010). Problems with the use of student test scores to evaluate teachers. Washington, DC: Economic Policy Institute Retrieved from http://www.epi.org/publication/bp278/. Accessed 5 May 2013.

Ballou, D., & Springer, M. G. (2015). Using student test scores to measure teacher performance: some problems in the design and implementation of evaluation systems. Educational Researcher, 44(2), 77–86. https://doi.org/10.3102/0013189X15574904.CrossRef

Berliner, D. C. (2018). Between Scylla and Charybdis: reflections on and problems associated with the evaluation of teachers in an era of metrification. Education Policy Analysis Archives, 26(54). https://doi.org/10.14507/epaa.26.3820.

Borg, S. (2018). Teacher evaluation: global perspectives and their implications for English language teaching. A literature review. London, UK: British Council Retrieved from https://www.teachingenglish.org.uk/sites/teacheng/files/pub_Teacher_evaluation_Global_perspectives_implications_ELT.pdf. Accessed 21 Dec 2018.

Braun, H. I. (2005). Using student progress to evaluate teachers: a primer on value-added models. Princeton, NJ: Educational Testing Service Retrieved from http://www.ets.org/Media/Research/pdf/PICVAM.pdf. Accessed 5 May 2013.

Briggs, D., & Domingue, B. (2011). Due diligence and the evaluation of teachers: a review of the value-added analysis underlying the effectiveness rankings of Los Angeles Unified School District teachers by the Los Angeles Times. Boulder, CO: National Education Policy Center Retrieved from http://nepc.colorado.edu/publication/due-diligence. Accessed 4 Feb 2019.

Cannata, M., Rubin, M., Goldring, E., Grissom, J. A., Neumerski, C. M., Drake, T. A., & Schuermann, P. (2017). Using teacher effectiveness data for information-rich hiring. Educational Administration Quarterly, 53(2), 180–222. https://doi.org/10.1177/0013161X16681629.CrossRef

Centre for Development and Enterprise (CDE). (2015a). Teacher evaluation: lessons from other countries [number 8]. Johannesburg, South Africa: Author. Retrieved from https://www.cde.org.za/teacher-evaluation-lessons-from-other-countries/. Accessed 22 Dec 2018.

Centre for Development and Enterprise (CDE). (2015b). Teacher evaluation in South African schools [number 9]. Johannesburg, South Africa: Author Retrieved from https://www.cde.org.za/teacher-evaluation-in-south-african-schools/. Accessed 22 Dec 2018.

Collins, C., & Amrein-Beardsley, A. (2014). Putting growth and value-added models on the map: a national overview. Teachers College Record, 116(1), 1-34. Retrieved from http://www.tcrecord.org/Content.asp?ContentId=17291. Accessed 04 Feb 2019.

Corcoran, S. P. (2010). Can teachers be evaluated by their students’ test scores? Should they be? The use of value-added measures of teacher effectiveness in policy and practice. Providence, RI: Annenberg Institute for School Reform Retrieved from http://www.annenberginstitute.org/publications/can-teachers-be-evaluated-their-students%E2%80%99-test-scores-should-they-be-use-value-added-me. Accessed 5 May 2013.

Cuevas, R., Ntoumanis, N., Fernandez-Bustos, J. G., & Bartholomew, K. (2018). Does teacher evaluation based on student performance predict motivation, well-being, and ill-being? Journal of School Psychology, 68, 154–162. https://doi.org/10.1016/j.jsp.2018.03.005.CrossRef

Danielson, C. (2013). The framework for teaching evaluation instrument. Princeton, NJ: The Danielson Group Retrieved from http://danielsongroup.org/download/?download=448. Accessed 4 Feb 2019.

Darling-Hammond, L. (2013). Getting teacher evaluation right: what really matters for effectiveness and improvement. New York, NY: Teachers College.

Darling-Hammond, L. (2015). Can value added add value to teacher evaluation? Educational Researcher, 44(2), 132–137. https://doi.org/10.3102/0013189X15575346.CrossRef

Darling-Hammond, L., Amrein-Beardsley, A., Haertel, E., & Rothstein, J. (2012). Evaluating teacher evaluation. Phi Delta Kappan, 93(6), 8–15. https://doi.org/10.1177/003172171209300603.CrossRef

Derrington, M. L. (2016). Implementing teacher evaluation: lattice of leadership. Journal of Research on Leadership Education, 11(2), 181–199. https://doi.org/10.1177/1942775116658689.CrossRef

Derrington, M. L., & Campbell, J. W. (2017). Teacher evaluation policy tools: principals’ selective use in instructional leadership. Leadership and Policy in Schools., 17, 568–590. https://doi.org/10.1080/15700763.2017.1326143.CrossRef

Derrington, M. L., & Campbell, J. W. (2018). High-stakes teacher evaluation policy: US principals’ perspectives and variations in practice. Teachers and Teaching: Theory and Practice, 24(3), 246–263. https://doi.org/10.1080/13540602.2017.1421164.CrossRef

Dodson, R. L. (2017). An analysis of principals’ perceptions of the primary teaching evaluation system used in eight U.S. states. International Journal of Education Policy and Leadership, 12(5), 1–22. https://doi.org/10.22230/ijelp.2017v12n5a773.CrossRef

Donaldson, M. L. (2011). Principals’ approaches to developing teacher quality: constraints and opportunities in hiring, assigning, evaluating, and developing teachers. Washington, DC: Center for American Progress Retrieved from http://cdn.americanprogress.org/wp-content/uploads/issues/2011/02/pdf/principal_report.pdf. Accessed 8 Aug 2013.

Erickson, F. (1986). Qualitative methods in research on teaching. In M. C. Wittrock (Ed.), Handbook of research on teaching (pp. 119–161). New York, NY: Macmillan.

Every Student Succeeds Act (ESSA) of 2015, Pub. L. No. 114-95, § 129 Stat. 1802. (2015).

Fink, A. (1995). Evaluation for education and psychology. Thousand Oaks, CA: Sage.

Finster, M., & Milanowski, A. (2018). Teacher perceptions of a new performance evaluation system and their influence on practice: a within- and between-school level analysis. Education Policy Analysis Archives, 26(41). https://doi.org/10.14507/epaa.26.3500.

Flores, M. A., & Derrington, M. L. (2017). School principals’ views of teacher evaluation policy: lessons learned from two empirical studies. International Journal of Leadership in Education, 20(4), 416–431. https://doi.org/10.1080/13603124.2015.1094144.CrossRef

Glaser, B., & Strauss, A. (1967). The discovery of grounded theory: strategies for qualitative research. Chicago, IL: Aldine.

Goldring, E., Grissom, J. A., Rubin, M., Neumerski, C. M., Cannata, M., Drake, T., & Schuermann, P. (2015). Make room value added: principals’ human capital decisions and the emergence of teacher observation data. Educational Researcher, 44(2), 96–104. https://doi.org/10.3102/0013189X15575031.CrossRef

Harris, D. N. (2011). Value-added measures in education: what every educator needs to know. Cambridge, MA: Harvard Education Press.

Harris, D. N., Ingle, W. K., & Rutledge, S. A. (2014). How teacher evaluation methods matter for accountability: a comparative analysis of teacher effectiveness ratings by principals and teacher value-added measures. American Educational Journal, 51(1), 73–112. https://doi.org/10.3102/0002831213517130.CrossRef

Hazi, H. M. (2017). VAM under scrutiny: teacher evaluation litigation in the states. The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 90(5–6), 184–190. https://doi.org/10.1080/00098655.2017.1366803.CrossRef

Herlihy, C., Karger, E., Pollard, C., Hill, H. C., Kraft, M. A., Williams, M., & Howard, S. (2014). State and local efforts to investigate the validity and reliability of scores from teacher evaluation systems. Teachers College Record, 116, 1–28.

Hopkins, P. (2016). Teacher voice: how teachers perceive evaluations and how leaders can use this knowledge to help teachers grow professionally. NASSP Bulletin, 100(1), 5–25. https://doi.org/10.1177/0192636516670771.CrossRef

Hopkins, E., Hendry, H., Garrod, F., McClare, S., Pettit, D., Smith, L., Burrell, H., & Temple, J. (2016). Teachers’ views of the impact of school evaluation and external inspection processes. Improving Schools, 19(1), 52–61. https://doi.org/10.1177/1365480215627894.CrossRef

Jiang, J. Y., Sporte, S. E., & Luppescu, S. (2015). Teacher perspectives on evaluation reform: Chicago’s REACH students. Educational Researcher, 44(2), 105–116. https://doi.org/10.3102/0013189X15575517.CrossRef

Johnson, S. M. (2005). The prospects for teaching as a profession. In L. V. Hedges & B. Schneider (Eds.), The social organization of schooling (pp. 72–90). New York, NY: Russell Sage Foundation.

Joint Committee on Standards for Educational Evaluation. (2008). The personnel evaluation standards (2nd ed.). Thousand Oaks, CA: Sage.

Kane, M. T. (2008). Terminology, emphasis, and utility in validation. Educational Researcher, 37(2), 76–82. https://doi.org/10.3102/0013189X08315390.CrossRef

Kraft, M. A., & Gilmour, A. F. (2016). Can principals promote teacher development as evaluators? A case study of principals’ views and experiences. Educational Administration Quarterly, 52(5), 711–753. https://doi.org/10.1177/001316IXI16653445.CrossRef

Kraft, M. A., & Gilmour, A. F. (2017). Revisiting the widget effect: teacher evaluation reforms and the distribution of teacher effectiveness. Educational Researcher, 46(5), 234–249. https://doi.org/10.3102/0013189X17718797.CrossRef

Lavigne, A. L. (2014). Exploring the intended and unintended consequences of high-stakes teacher evaluation on schools, teachers, and students. Teachers College Record, 116, 1–29 Retrieved from http://www.tcrecord.org/library/abstract.asp?contentid=17294. Accessed 24 Jan 2015.

Lavigne, A. L., & Chamberlain, R. W. (2017). Teacher evaluation in Illinois: school leaders’ perceptions and practices. Educational Assessment, Evaluation and Accountability, 29, 179–209. https://doi.org/10.1007/s11092-016-9250-0.CrossRef

Lavigne, A. L., & Good, T. L. (2014). Teacher and student evaluation: moving beyond the failure of school reform. New York, NY: Routledge.

Liu, S., & Zhao, D. (2013). Teacher evaluation in China: latest trends and future directions. Educational Assessment, Evaluation and Accountability, 25(3), 231–250. https://doi.org/10.1007/s11092-013-9168-8.CrossRef

Loewus, L. (2017). Are states changing course on teacher evaluation? Test-score growth plays lesser role in six states. Education Week, 37(13), 1–17 Retrieved from https://www.edweek.org/ew/articles/2017/11/15/are-states-changing-course-on-teacher. Accessed 28 Feb 2018.

Martinez, F., Taut, S., & Schaaf, K. (2016). Classroom observation for evaluating and improving teaching: an international perspective. Studies in Educational Evaluation, 49, 15–29. https://doi.org/10.1016/j.stueduc.2016.03.002.CrossRef

Messick, S. (1975). The standard problem: Meaning and values in measurement and evaluation. American Psychologist, 30, 955–966. https://doi.org/10.1037/0003-066X.30.10.955.CrossRef

Messick, S. (1980). Test validity and the ethics of assessment. American Psychologist, 35(11), 1012–1027. https://doi.org/10.1037/0003-066X.35.11.1012.CrossRef

Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: Sage Publications.

No Child Left Behind (NCLB) Act of 2001, Public Law 107–110, § 115 Stat. 1425. (2002).

Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York, NY: McGraw Hill.

O’Pry, S. C., & Schumacher, G. (2012). New teachers’ perceptions of a standards-based performance appraisal system. Educational Assessment, Evaluation and Accountability, 24(4), 325–350. https://doi.org/10.1007/s11092-012-9148-4.CrossRef

Organisation for Economic Co-operation and Development (OECD). (2013). Teachers for the 21st century: Using evaluation to improve teaching. Paris: OECD Publishing.

Papay, J. P. (2010). Different tests, different answers: the stability of teacher value-added estimates across outcome measures. American Educational Research Journal, 48(1), 163–193. https://doi.org/10.3102/00002831210362589.CrossRef

Paufler, N. A. (2018a). Declining morale, diminishing autonomy, and decreasing value: principal reflections on a high-stakes teacher evaluation system. International Journal of Education Policy & Leadership, 13(8). https://doi.org/10.22230/ijepl.2018v13n8a831.

Paufler, N. A. (2018b). The value a teacher evaluation system adds in practice: school administrator and teacher perceptions of system effectiveness. Manuscript submitted for publication.

Popham, W. J. (1988). Educational evaluation (2nd ed.). Englewood Cliffs, NJ: Prentice Hall.

Reddy, L. A., Dudek, C. M., Peters, S., Alperin, A., Kettler, R. J., & Kurz, A. (2018). Teachers’ and school administrators’ attitudes and beliefs of teacher evaluation: a preliminary investigation of high poverty school districts. Educational Assessment, Evaluation and Accountability, 30, 47–40. https://doi.org/10.1007/s11092-017-9263-3.CrossRef

Reid, D. B. (2018). School principals acting as middle leaders implementing new teacher evaluation systems. School Leadership & Management., 1–17. https://doi.org/10.1080/13632434.2018.1508013.

Saldaña, J. (2013). The coding manual for qualitative researchers (2nd ed.). London: Sage.

Smith, M. L. (1997). Mixing and matching: methods and models. New Directions for Evaluation, 74, 73–85. https://doi.org/10.1002/ev.1073.CrossRef

Stake, R. E. (1978). The case study method in social inquiry. Educational Researcher, 7(2), 5–8. https://doi.org/10.3102/0013189X007002005.CrossRef

Stake, R. E., & Trumbull, D. (1982). Naturalistic generalizations. Review Journal of Philosophy and Social Science, 7, 1–12.

Stewart, V. (2013). Teacher quality: the 2013 International Summit on the Teaching Profession. Asia Society: Partnership for Global Learning Retrieved from https://asiasociety.org/files/teachingsummit2013.pdf. Accessed 21 Dec 2018.

Stewart, V. (2015). Implementing highly effective teacher policy and practice. Asia Society: Partnership for Global Learning Retrieved from https://asiasociety.org/files/2015-istp-report.pdf. Accessed 22 Dec 2018.

Stewart, V. (2016). Teachers’ professional learning and growth: creating the conditions to achieve quality teaching for excellent learning outcomes. Asia Society: Partnership for Global Learning Retrieved from https://asiasociety.org/files/2016-istp-report-small.pdf. Accessed 22 Dec 2018.

Stewart, V. (2018). New challenges and opportunities facing the teaching profession in public education. Asia Society: Partnership for Global Learning Retrieved from https://asiasociety.org/sites/default/files/inline-files/2018-international-summit-on-the-teaching-profession-edu-istp.pdf.

Strauss, A. L., & Corbin, J. (1995). Basics of qualitative research: grounded theory procedures and techniques. Newbury Park, CA: Sage Publications.

Taut, S., & Sun, Y. (2014). The development and implementation of a national, standards-based, multi-method teacher performance assessment system in Chile. Education Policy Analysis Archives, 22(71). https://doi.org/10.14507/epaa.v22n71.2014.

Taylor, E. S., & Tyler, J. H. (2012). Can teacher evaluation improve teaching? Education Next, 12(4) Retrieved from https://www.educationnext.org/can-teacher-evaluation-improve-teaching/. Accessed 4 Feb 2019.

Tucker, P. D., & Stronge, J. H. (2005). Linking teacher evaluation and student learning. Alexandria, VA: Association for Supervision and Curriculum Development.

United Nations Educational, Scientific and Cultural Organization (UNESCO). (2016). Education 2030. Incheon declaration and framework for action. Retrieved from https://unesdoc.unesco.org/ark:/48223/pf0000245656. Accessed 22 Dec 2018.

United States Department of Education. (2009). Race to the Top program: executive summary. Retrieved from http://www2.ed.gov/programs/racetothetop/executive-summary.pdf. Accessed 6 July 2013.

United States Department of Education. (2010). Teacher Incentive Fund. Retrieved from http://www2.ed.gov/programs/teacherincentive/index.html. Accessed 6 July 2013.

Weisberg, D., Sexton, S., Mulhern, J., & Keeling, D. (2009). The widget effect: our national failure to acknowledge and act of differences in teacher effectiveness (2nd ed.). Brooklyn, NY: The New Teacher Project (TNTP). Retrieved from http://tntp.org/ideas-and-innovations/view/the-widget-effect. Accessed 31 Jan 2018.

Will, M. (2016). Assessing quality of teaching staff still complex despite ESSA’s leeway. Education Week, 36(16), 31–32 Retrieved from http://www.edweek.org/ew/articles/2017/01/04/assessing-quality-of-teaching-staff-still-complex.html?intc=EW-QC17-TOC&_ga=1.138540723.1051944855.1481128421. Accessed 9 Apr 2017.

World Bank. (2013). What matters most in teacher policies? A framework for building a more effective teaching profession. Retrieved from https://openknowledge.worldbank.org/bitstream/handle/10986/20143/901820NWP0no4000Box385307B00PUBLIC0.pdf?sequence=1&isAllowed=y. Accessed 4 Feb 2019.

World Bank. (2018). World development report: learning to realize education’s promise. Washington, DC: World Bank Retrieved from www.worldbank.org/en/publication/wdr2018. Accessed 4 Feb 2019.

Title: Reframing conversations about teacher quality: school and district administrators’ perceptions of the validity, reliability, and justifiability of a new teacher evaluation system
Authors: Noelle A. Paufler
Chris Clark
Publication date: 13-02-2019
Publisher: Springer Netherlands
Published in: Educational Assessment, Evaluation and Accountability / Issue 1/2019
Print ISSN: 1874-8597
Electronic ISSN: 1874-8600
DOI: https://doi.org/10.1007/s11092-019-09292-w

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 1/2019

Using the dynamic approach to school improvement to promote quality and equity in education: a European study

Top-down and bottom-up approaches to improve educational quality: their intended and unintended consequences

Classroom observation systems in context: A case for the validation of observation systems

Teacher log of students’ opportunity to learn and classroom observation: an initial investigation of convergence

Underperforming teachers: the impact on co-workers and their responses