Skip to main content
Erschienen in: Learning Environments Research 2/2009

01.07.2009 | Original Paper

Statistical tests conducted with school environment data: The effect of teachers being clustered in schools

verfasst von: Jeffrey P. Dorman

Erschienen in: Learning Environments Research | Ausgabe 2/2009

Einloggen

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This article discusses the effect of clustering on statistical tests conducted with school environment data. Because most school environment studies involve the collection of data from teachers nested within schools, the hierarchical nature to these data cannot be ignored. In particular, this article considers the influence of intraschool correlations on tests of statistical significance conducted with the individual teacher as the unit of analysis. Theory that adjusts t test scores for nested data in two-group comparisons is presented and applied to school environment data. This article demonstrates that Type I error rates inflate greatly as the intraschool correlation increases. Because data analysis techniques that recognise the clustering of teachers in schools are essential, it is recommended that either multilevel analysis or adjustments to statistical parameters be undertaken in school environment studies involving nested data.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Aitkin, M., & Longford, N. (1986). Statistical modelling issues in school effectiveness studies (with discussion). Journal of the Royal Statistical Society. Series A (General), 149, 1–43. doi:10.2307/2981882.CrossRef Aitkin, M., & Longford, N. (1986). Statistical modelling issues in school effectiveness studies (with discussion). Journal of the Royal Statistical Society. Series A (General), 149, 1–43. doi:10.​2307/​2981882.CrossRef
Zurück zum Zitat Alker, H. R. (1969). A typology of ecological fallacies. In H. Dogan & S. Rokkan (Eds.), Quantitative ecological analysis in the social sciences (pp. 69–86). London: MIT press. Alker, H. R. (1969). A typology of ecological fallacies. In H. Dogan & S. Rokkan (Eds.), Quantitative ecological analysis in the social sciences (pp. 69–86). London: MIT press.
Zurück zum Zitat Anderson, C. S. (1982). The search for school climate: A review of the research. Review of Educational Research, 52, 368–420. Anderson, C. S. (1982). The search for school climate: A review of the research. Review of Educational Research, 52, 368–420.
Zurück zum Zitat Anstine-Templeton, R., Johnson, C. E., Lee, H., & Wan, G. (2006). Becoming more efficient: One college’s use of the SLEQ. In D. L. Fisher & M. S. Khine (Eds.), Contemporary approaches to research on learning environments (pp. 313–336). Singapore: World Scientific. Anstine-Templeton, R., Johnson, C. E., Lee, H., & Wan, G. (2006). Becoming more efficient: One college’s use of the SLEQ. In D. L. Fisher & M. S. Khine (Eds.), Contemporary approaches to research on learning environments (pp. 313–336). Singapore: World Scientific.
Zurück zum Zitat Bailey, M. (1979). The art of positive principalship. Momentum, 10(2), 46–47. Bailey, M. (1979). The art of positive principalship. Momentum, 10(2), 46–47.
Zurück zum Zitat Brookover, W. B., Schweitzer, J. H., Schneider, J. M., Beady, C. H., Flood, P. K., & Wisenbaker, J. M. (1978). Elementary school social climate and school achievement. American Educational Research Journal, 15, 301–318. Brookover, W. B., Schweitzer, J. H., Schneider, J. M., Beady, C. H., Flood, P. K., & Wisenbaker, J. M. (1978). Elementary school social climate and school achievement. American Educational Research Journal, 15, 301–318.
Zurück zum Zitat Burstein, L. (1980). The analysis of multi-level data in educational research and evaluation. In D. C. Berliner (Ed.), Review of research in education, Vol. 8 (pp. 158–233). Washington, DC: American Educational Research Association. Burstein, L. (1980). The analysis of multi-level data in educational research and evaluation. In D. C. Berliner (Ed.), Review of research in education, Vol. 8 (pp. 158–233). Washington, DC: American Educational Research Association.
Zurück zum Zitat Cresswell, J., & Fisher, D. L. (1999, April). A school level environment study in Australia. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada. Cresswell, J., & Fisher, D. L. (1999, April). A school level environment study in Australia. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada.
Zurück zum Zitat Cronbach, L. J. (1976). Research on classrooms and schools: Formulation of questions, design and analysis (Occasional paper of the Stanford Evaluation Consortium). Palo Alto, CA: Stanford University. Cronbach, L. J. (1976). Research on classrooms and schools: Formulation of questions, design and analysis (Occasional paper of the Stanford Evaluation Consortium). Palo Alto, CA: Stanford University.
Zurück zum Zitat den Brok, P., Brekelmans, M., & Wubbels, T. (2006). Multilevel issues in research using students’ perceptions of learning environments: The case of the Questionnaire on Teacher Interaction. Learning Environments Research, 9, 199–213. doi:10.1007/s10984-006-9013-9.CrossRef den Brok, P., Brekelmans, M., & Wubbels, T. (2006). Multilevel issues in research using students’ perceptions of learning environments: The case of the Questionnaire on Teacher Interaction. Learning Environments Research, 9, 199–213. doi:10.​1007/​s10984-006-9013-9.CrossRef
Zurück zum Zitat Docker, J. G., Fraser, B. J., & Fisher, D. L. (1989). Differences in the psychosocial work environment of different types of schools. Journal of Research in Childhood Education, 4, 5–17.CrossRef Docker, J. G., Fraser, B. J., & Fisher, D. L. (1989). Differences in the psychosocial work environment of different types of schools. Journal of Research in Childhood Education, 4, 5–17.CrossRef
Zurück zum Zitat Dorman, J. P. (1994). A study of school and classroom environments in Queensland Catholic secondary schools. Unpublished PhD thesis, Curtin University of Technology, Perth, Australia. Dorman, J. P. (1994). A study of school and classroom environments in Queensland Catholic secondary schools. Unpublished PhD thesis, Curtin University of Technology, Perth, Australia.
Zurück zum Zitat Dorman, J. P. (1996a). School environment questionnaire: An instrument developed for Australian Catholic secondary schools. Journal of Christian Education, 39(1), 31–44. Dorman, J. P. (1996a). School environment questionnaire: An instrument developed for Australian Catholic secondary schools. Journal of Christian Education, 39(1), 31–44.
Zurück zum Zitat Dorman, J. P. (2002). Classroom environment research: Progress and possibilities. Queensland Journal of Educational Research, 18, 112–140. Dorman, J. P. (2002). Classroom environment research: Progress and possibilities. Queensland Journal of Educational Research, 18, 112–140.
Zurück zum Zitat Dorman, J. P., Fraser, B. J., & McRobbie, C. J. (1997). Relationship between school-level and classroom-level environment in secondary schools. Journal of Educational Administration, 35, 74–91. doi:10.1108/09578239710156999.CrossRef Dorman, J. P., Fraser, B. J., & McRobbie, C. J. (1997). Relationship between school-level and classroom-level environment in secondary schools. Journal of Educational Administration, 35, 74–91. doi:10.​1108/​0957823971015699​9.CrossRef
Zurück zum Zitat Ely, M. J. (1971). The management of schools in New South Wales 1848–1880: Local initiative suppressed? Journal of Educational Administration, 9, 79–97. doi:10.1108/eb009659.CrossRef Ely, M. J. (1971). The management of schools in New South Wales 1848–1880: Local initiative suppressed? Journal of Educational Administration, 9, 79–97. doi:10.​1108/​eb009659.CrossRef
Zurück zum Zitat Fisher, D. L., & Fraser, B. J. (1983). Use of WES to assess science teachers’ perceptions of school environment. European Journal of Science Education, 5, 231–233. Fisher, D. L., & Fraser, B. J. (1983). Use of WES to assess science teachers’ perceptions of school environment. European Journal of Science Education, 5, 231–233.
Zurück zum Zitat Fisher, D. L., Fraser, B. J., & Wubbels, T. (1993). Interpersonal teacher behavior and school environment. Do you know what you look like? In T. Wubbels & J. Levy (Eds.), Interpersonal relationships in education (pp. 103–112). London: Falmer Press. Fisher, D. L., Fraser, B. J., & Wubbels, T. (1993). Interpersonal teacher behavior and school environment. Do you know what you look like? In T. Wubbels & J. Levy (Eds.), Interpersonal relationships in education (pp. 103–112). London: Falmer Press.
Zurück zum Zitat Fraser, B. J. (1994). Research on classroom and school climate. In D. Gabel (Ed.), Handbook of research on science teaching and learning (pp. 493–541). New York: Macmillan. Fraser, B. J. (1994). Research on classroom and school climate. In D. Gabel (Ed.), Handbook of research on science teaching and learning (pp. 493–541). New York: Macmillan.
Zurück zum Zitat Fraser, B. J. (1998). Science learning environments: Assessments, effects and determinants. In B. J. Fraser & K. G. Tobin (Eds.), International handbook of science education (pp. 527–564). Dordrecht, The Netherlands: Kluwer. Fraser, B. J. (1998). Science learning environments: Assessments, effects and determinants. In B. J. Fraser & K. G. Tobin (Eds.), International handbook of science education (pp. 527–564). Dordrecht, The Netherlands: Kluwer.
Zurück zum Zitat Fraser, B. J. (2007). Classroom learning environments. In S. K. Abell & N. G. Lederman (Eds.), Handbook of research on science education (pp. 103–124). Mahwah, NJ: Erlbaum. Fraser, B. J. (2007). Classroom learning environments. In S. K. Abell & N. G. Lederman (Eds.), Handbook of research on science education (pp. 103–124). Mahwah, NJ: Erlbaum.
Zurück zum Zitat Fraser, B. J., & Rentoul, A. J. (1982). Relationships between school-level and classroom-level environment. The Alberta Journal of Educational Research, 28, 212–225. Fraser, B. J., & Rentoul, A. J. (1982). Relationships between school-level and classroom-level environment. The Alberta Journal of Educational Research, 28, 212–225.
Zurück zum Zitat Goldstein, H. (2004). Some observations on the definition and estimation of effect sizes. In I. Schagen & K. Elliot (Eds.), But what does it mean? The use of effect sizes in educational research (pp. 67–71). Slough, UK: NFER. Goldstein, H. (2004). Some observations on the definition and estimation of effect sizes. In I. Schagen & K. Elliot (Eds.), But what does it mean? The use of effect sizes in educational research (pp. 67–71). Slough, UK: NFER.
Zurück zum Zitat Gregoire, M., & Algina, J.(2000, April). Reconceptualizing the debate on school climate and students’ academic motivation and achievement: A multilevel analysis. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Gregoire, M., & Algina, J.(2000, April). Reconceptualizing the debate on school climate and students’ academic motivation and achievement: A multilevel analysis. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
Zurück zum Zitat Halawah, I. (2005). The relationship between effective communication of high school principal and school climate. Education, 126, 334–345. Halawah, I. (2005). The relationship between effective communication of high school principal and school climate. Education, 126, 334–345.
Zurück zum Zitat Halpin, A. W., & Croft, D. B. (1963). Organisational climate of schools. Chicago: Midwest Administration Center, University of Chicago. Halpin, A. W., & Croft, D. B. (1963). Organisational climate of schools. Chicago: Midwest Administration Center, University of Chicago.
Zurück zum Zitat Hox, J. J. (2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Erlbaum. Hox, J. J. (2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Erlbaum.
Zurück zum Zitat Hudley, C., Daoud, A., Polanco, T., Wright-Castro, R., & Hershberg, R.(2003, April). Student engagement, school climate, and future expectations in high school. Paper presented at the biennial meeting of the Society for Research in Child Development, Tampa, FL. Hudley, C., Daoud, A., Polanco, T., Wright-Castro, R., & Hershberg, R.(2003, April). Student engagement, school climate, and future expectations in high school. Paper presented at the biennial meeting of the Society for Research in Child Development, Tampa, FL.
Zurück zum Zitat Johnson, B., & Stevens, J. J. (2001). Exploratory and confirmatory factor analysis of the School Level Environment Questionnaire (SLEQ). Learning Environments Research, 4, 325–344. doi:10.1023/A:1014486821714.CrossRef Johnson, B., & Stevens, J. J. (2001). Exploratory and confirmatory factor analysis of the School Level Environment Questionnaire (SLEQ). Learning Environments Research, 4, 325–344. doi:10.​1023/​A:​1014486821714.CrossRef
Zurück zum Zitat Kelley, R. C., Thornton, B., & Daugherty, R. (2005). Relationships between measures of leadership and school climate. Education, 126, 17–25. Kelley, R. C., Thornton, B., & Daugherty, R. (2005). Relationships between measures of leadership and school climate. Education, 126, 17–25.
Zurück zum Zitat Kilian, J. M., Fish, M. C., & Maniago, E. B. (2007). Making schools safe: A system-wide school intervention to increase student prosocial behaviors and enhance school climate. Journal of Applied School Psychology, 23, 1–30. doi:10.1300/J370v23n01_01.CrossRef Kilian, J. M., Fish, M. C., & Maniago, E. B. (2007). Making schools safe: A system-wide school intervention to increase student prosocial behaviors and enhance school climate. Journal of Applied School Psychology, 23, 1–30. doi:10.​1300/​J370v23n01_​01.CrossRef
Zurück zum Zitat Kitsantas, A., Ware, H. W., & Martinez-Arias, R. (2004). Students’ perceptions of school safety: Effects by community, school environment, and substance use variables. The Journal of Early Adolescence, 24, 412–430. doi:10.1177/0272431604268712.CrossRef Kitsantas, A., Ware, H. W., & Martinez-Arias, R. (2004). Students’ perceptions of school safety: Effects by community, school environment, and substance use variables. The Journal of Early Adolescence, 24, 412–430. doi:10.​1177/​0272431604268712​.CrossRef
Zurück zum Zitat Loukas, A., & Murphy, J. L. (2007). Middle school student perceptions of school climate: Examining protective functions on subsequent adjustment problems. Journal of School Psychology, 45, 293–309. doi:10.1016/j.jsp.2006.10.001.CrossRef Loukas, A., & Murphy, J. L. (2007). Middle school student perceptions of school climate: Examining protective functions on subsequent adjustment problems. Journal of School Psychology, 45, 293–309. doi:10.​1016/​j.​jsp.​2006.​10.​001.CrossRef
Zurück zum Zitat Moos, R. H. (1986). Work environment scale manual (2nd ed.). Palo Alto, CA: Consulting Psychologists Press. Moos, R. H. (1986). Work environment scale manual (2nd ed.). Palo Alto, CA: Consulting Psychologists Press.
Zurück zum Zitat Noonan, J. (2004). School climate and the safe school: Seven contributing factors. Educational Horizons, 83(1), 61–65. Noonan, J. (2004). School climate and the safe school: Seven contributing factors. Educational Horizons, 83(1), 61–65.
Zurück zum Zitat Peterson, R. L., & Skiba, R. (2001). Creating school climates that prevent violence. Social Studies, 92, 167–175. Peterson, R. L., & Skiba, R. (2001). Creating school climates that prevent violence. Social Studies, 92, 167–175.
Zurück zum Zitat Rasbash, J., Steele, F., Browne, W., & Prosser, B. (2005). A user’s guide to MLwiN version 2.0. Bristol, UK: Centre for Multilevel Modelling, University of Bristol. Rasbash, J., Steele, F., Browne, W., & Prosser, B. (2005). A user’s guide to MLwiN version 2.0. Bristol, UK: Centre for Multilevel Modelling, University of Bristol.
Zurück zum Zitat Raudenbush, S., & Bryk, A. (2002). Hierarchical linear models. Thousand Oaks, CA: Sage. Raudenbush, S., & Bryk, A. (2002). Hierarchical linear models. Thousand Oaks, CA: Sage.
Zurück zum Zitat Raudenbush, S. W., & Willms, J. D. (Eds.). (1991). Schools, classrooms and pupils: International studies of schooling from multilevel perspective. San Diego, CA: Academic. Raudenbush, S. W., & Willms, J. D. (Eds.). (1991). Schools, classrooms and pupils: International studies of schooling from multilevel perspective. San Diego, CA: Academic.
Zurück zum Zitat Rentoul, A. J., & Fraser, B. J. (1983). Development of a school-level environment questionnaire. Journal of Educational Administration, 21(1), 21–39. doi:10.1108/eb009866.CrossRef Rentoul, A. J., & Fraser, B. J. (1983). Development of a school-level environment questionnaire. Journal of Educational Administration, 21(1), 21–39. doi:10.​1108/​eb009866.CrossRef
Zurück zum Zitat Rowe, K. J. (2007). Practical multilevel analysis with MLwiN & LISREL: An integrated course (7th ed.). Melbourne, Australia: Australian Council for Educational Research. Rowe, K. J. (2007). Practical multilevel analysis with MLwiN & LISREL: An integrated course (7th ed.). Melbourne, Australia: Australian Council for Educational Research.
Zurück zum Zitat Snijders, T. A. B., & Bosker, R. J. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London: Sage. Snijders, T. A. B., & Bosker, R. J. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London: Sage.
Zurück zum Zitat Tabachnick, B. G., & Fidell, L. S. (2007). Understanding multivariate statistics (5th ed.). Boston: Pearson. Tabachnick, B. G., & Fidell, L. S. (2007). Understanding multivariate statistics (5th ed.). Boston: Pearson.
Zurück zum Zitat Vail, K. (2005). Create great school climate. Education digest, 71(4), 11. Vail, K. (2005). Create great school climate. Education digest, 71(4), 11.
Zurück zum Zitat Vyskocil, J. R., & Goens, G. A. (1979). Collective bargaining and supervision: A matter of climate. Educational Leadership, 37, 175–177. Vyskocil, J. R., & Goens, G. A. (1979). Collective bargaining and supervision: A matter of climate. Educational Leadership, 37, 175–177.
Zurück zum Zitat Walker, S. O., Petrill, S. A., & Plomin, R. (2005). A genetically sensitive investigation of the effects of the school environment and socio-economic status on academic achievement in seven-year-olds. Educational Psychology, 25, 55–73. doi:10.1080/0144341042000294895.CrossRef Walker, S. O., Petrill, S. A., & Plomin, R. (2005). A genetically sensitive investigation of the effects of the school environment and socio-economic status on academic achievement in seven-year-olds. Educational Psychology, 25, 55–73. doi:10.​1080/​0144341042000294​895.CrossRef
Zurück zum Zitat Wilson, D. (2004). The interface of school climate and school connectedness and relationships with aggression and victimization. The Journal of School Health, 74, 293–299.CrossRef Wilson, D. (2004). The interface of school climate and school connectedness and relationships with aggression and victimization. The Journal of School Health, 74, 293–299.CrossRef
Zurück zum Zitat Wood, N. B., Lawrenz, F. P., Huffman, D., & Schultz, M. (2006). Viewing the school environment through multiple lenses: In search of school-level variables tied to student achievement. Journal of Research in Science Teaching, 43, 237–254. doi:10.1002/tea.20108.CrossRef Wood, N. B., Lawrenz, F. P., Huffman, D., & Schultz, M. (2006). Viewing the school environment through multiple lenses: In search of school-level variables tied to student achievement. Journal of Research in Science Teaching, 43, 237–254. doi:10.​1002/​tea.​20108.CrossRef
Zurück zum Zitat Wubbels, T., & Levy, J. (Eds.). (1993). Do you know what you look like? Interpersonal relationships in education. London: Falmer Press. Wubbels, T., & Levy, J. (Eds.). (1993). Do you know what you look like? Interpersonal relationships in education. London: Falmer Press.
Metadaten
Titel
Statistical tests conducted with school environment data: The effect of teachers being clustered in schools
verfasst von
Jeffrey P. Dorman
Publikationsdatum
01.07.2009
Verlag
Springer Netherlands
Erschienen in
Learning Environments Research / Ausgabe 2/2009
Print ISSN: 1387-1579
Elektronische ISSN: 1573-1855
DOI
https://doi.org/10.1007/s10984-009-9054-y

Weitere Artikel der Ausgabe 2/2009

Learning Environments Research 2/2009 Zur Ausgabe