Top

Educational Assessment, Evaluation and Accountability

Published in:

17-08-2021

An item response theory and Rasch analysis of the NUDKS: a data literacy scale

Authors: Pamela S. Trantham, Jonathon Sikorski, R. J. de Ayala, Beth Doll

Published in: Educational Assessment, Evaluation and Accountability | Issue 1/2022

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

There is an extensive need for school systems to reliably assess the data literacy and data use skills of their educators. To address this need, the current study seeks to refine the NU Data Knowledge Scale (NUDKS) for assessing teacher data literacy for classroom data. A data-based decision-making framework provides the theoretical underpinnings for the instrument. The study’s objective is to refine the NUDKS such that items are located at various points along the data literacy continuum. In this fashion, the NUDKS should be able to measure teacher data literacy throughout the data literacy continuum. To this end, item response theory is used to provide the estimates of the items’ locations and teacher data literacy. Analyses revealed that the NUDKS conformed to the Rasch model. To facilitate the future use of the NUDKS, concordance tables were created to provide a quick determination of teacher data literacy.

previous article Representations of student performance data in local education policy

next article Correction to: An application of importance-performance analysis to students’ evaluation of teaching

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

Andrich, D. (2004). Controversy and the Rasch model: A characteristic of incompatible paradigms? Medical Care, 42, 17–116.CrossRef

Bakx, A., Baartman, L., & van Schilt-Mol, T. (2014). Development and evaluation of a summative assessment program for senior teacher competence. Studies in Educational Evaluation, 40, 50–62.CrossRef

Booher-Jennings, J. (2005). Below the bubble:“educational triage” and the Texas accountability system. American Educational Research Journal, 42, 231–268.CrossRef

Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model: Fundamental measurement in the human sciences. Lawrence Erlbaum Associates Publishers.CrossRef

Christoforidou, M., Kyriakides, L., Antoniou, P., & Creemers, B. P. M. (2014). Searching for stages of teacher’s skills in assessment. Studies in Educational Evaluation, 40, 1–11.CrossRef

de Ayala, R. (2009). The theory and practice of item response theory. The Guilford Press.

de Ayala, R. (2013). The IRT tradition and its applications. FoundationsIn T. D. Little (Ed.), Oxford Handbook of Quantitative Methods (Vol. I, pp. 144–168). Oxford University Press.

Doll, B., Haack, K., Kosse, S., Osterloh, M., Siemers, E., & Pray, B. (2005). The dilemma of pragmatics: Why schools don’t use quality team consultation practices. Journal of Educational and Psychological Consultation, 16, 127–155.CrossRef

Doll, B., Franta, E., Thomas, A., Chapla, B., & Sikorski, J. (2014, April). Nu data: Preparing educators to use data well. Paper presented at the American Educational Reserach Association.

Doll, B., Horn, C., & Shope, R. (2010). Using data to foster the school success of students with disabilities (Grant No. R324A110131). IES.

Duncan, A. (2009c, June). Robust data gives up the roadmap to reform. Keynote address at the Fourth Annual IES Research Conference, Retrieved from http://www2.ed.gov/news/speeches/2009/06/06082009.pdf

Dunn, K. E., Airola, D. T., Lo, W.-J., & Garrison, M. (2013). Becoming data driven: The influence of teachers’ sense of efficacy on concerns related to data-driven decision making. Journal of Experimental Education, 81, 222–241.CrossRef

Gettinger, J. B., Mulford, L., & Hoffman, A. (2010). Prevention and early intervention for preschool children at risk for learning and behavior problems. In Doll, W. Pfohl & J. Yoon (Eds.), Handbook of youth prevention science (pp. 349–374). Routledge.

Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Kluwer-Nijhoff.CrossRef

Ingram, D., Louis, K. S., & Schroeder, R. G. (2004). Accountability policies and teacher decision making: Barriers to the use of data to improve practice. Teachers College Record, 106, 1258–1287.CrossRef

Jacobs, J., Gregory, A., Hoppey, D., & Yendol-Hoppey, D. (2009). Data literacy: Understanding teachers’ data use in a context of accountability and response to intervention. Action in Teacher Education, 31, 41–55.CrossRef

Jimerson, J. B. (2014). Thinking about data: Exploring the development of mental models for “data use” among teachers and school leaders. Studies in Educational Evaluation, 42, 5–14.CrossRef

Kerr, K. A., Marsh, J. A., Ikemoto, G. S., Darilek, H., & Barney, H. (2006). Strategies to promote data use for instructional improvement: Actions, outcomes, and lessons from three urban districts. American Journal of Education, 112, 496–552.CrossRef

Kosse, S., & Doll, B. (2006, March). A comparison of traditional student assistance teams and response to intervention. A paper presented at the Annual Convention of the National Association of School Psychologists.

Lai, M. K., & Hsiao, S. (2014). Developing data collection and management systems for decision-making: What professional development is required? Studies in Educational Evaluation, 42, 63–67.CrossRef

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Erlbaum.

Love, N. (2004). Taking data to new depths. Journal of Staff Development, 25, 22–26.

Love, N. (2011). Data literacy for teachers. Dude Publishing.

Mandinach, E. B., & Gummer, E. (2012). Navigating the landscape of data literacy: it is complex. Washington, DC and Portland, OR: WestEd and Education Northwest.

Mandinach, E. B., & Gummer, E. S. (2013a). Building educators’ data literacy: Differing perspectives. The Journal of Educational Research & Policy Studies, 13, 4–8.

Mandinach, E. B., & Gummer, E. S. (2013b). A systemic view of implementing data literacy in educator preparation. Educational Researcher, 42, 30–37.CrossRef

McCutchen, D., Abbott, R. D., Green, L. B., Beretvas, S. N., Cox, S., Potter, N. S., ... Gray, A. L. (2002). Beginning literacy: links among teacher knowledge, teacher practice, and student learning. Journal of Learning Disabilities, 35, 69-86

Mead, R.J. (2008) A Rasch primer: the measurement theory of Georg Rasch. Psychometrics services research memorandum 2008–001. Data Recognition Corporation.

Means, B., Chen, E., DeBarger, A., & Padilla, C. (2011). Teachers’ ability to use data to inform instruction: challenges and supports. Office of Planning, Evaluation and Policy Development, US Department of Education (pp. 1–122). US Department of Education.

Mertler, C. A., & Campbell, C. (2005, April). Measuring teachers’ knowledge & application of classroom assessment concepts: Development of the assessment literacy inventory. Paper presented at the American Educational Research Association

Sikorski, J. (2015). Examination of the NU Data Knowledge Scale. (Doctoral dissertation, University of Nebraska – Lincoln). Retrieved from https://digitalcommons.unl.edu/dissertations/AAI10125684/

Sikorski, J., Doll, B., Thomas, A., Franta, E., & Kenney, C. (2013). Nu data: building educators’ data use in schools. The Researcher: Theory and Practice, 23–27.

Sikorski, J., Franta, E., & Doll, B. (2014, February). Measuring educators’ data literacy: the NU data knowledge scale. Paper presented at the National Association of School Psychologist Annual Convention.

Staman, L., Visscher, A. J., & Luyten, H. (2014). The effects of professional development on the attitudes, knowledge and skills for data-driven decision making. Studies in Educational Evaluation, 42, 79–9. 1.1016/j.stueduc.2013.11.002

Suen, H. K., Lei, P.-W., & Li, H. (2011). Data analysis for effective decision making. In M. A. Bray & T. J. Kehle (Eds.), The oxford handbook of school psychology (pp. 140–168). Oxford University Press.

U.S. Department of Education. (2008). Teachers’ use of student data systems to improve instruction: 2005 to 2007.

Wayman, J. C., & Jimerson, J. B. (2014). Teacher needs for data-related professional learning. Studies in Educational Evaluation, 42, 25–34.CrossRef

Wilson, M. (2005). Constructing measures: An item response modeling approach. Erlbaum.

Wright, B. D. (1968). Sample-free test calibration and person measurement. In Proceedings of the 1967 Invitational Conference on Testing Problems. Educational Testing Service.

Wright, B. D., & Stone, M. H. (1979). Best test design. MESA Press.

Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8, 125–145.CrossRef

Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30, 187–213.CrossRef

Zimowski, M. F., Muraki, E., Mislevy, R. J., & Bock, R. D. (1996). BILOG-MG [Computer software]. Scientific Software International.

Title: An item response theory and Rasch analysis of the NUDKS: a data literacy scale
Authors: Pamela S. Trantham
Jonathon Sikorski
R. J. de Ayala
Beth Doll
Publication date: 17-08-2021
Publisher: Springer Netherlands
Published in: Educational Assessment, Evaluation and Accountability / Issue 1/2022
Print ISSN: 1874-8597
Electronic ISSN: 1874-8600
DOI: https://doi.org/10.1007/s11092-021-09372-w

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 1/2022

Data-driven approaches to education governance and their implications

Correction to: An application of importance-performance analysis to students’ evaluation of teaching

“I am here for the students”: principals’ perception of accountability amid work intensification

School accountability and teacher stress: international evidence from the OECD TALIS study

Principals may inflate teacher evaluation scores to achieve important goals

Representations of student performance data in local education policy