nach oben

Educational Assessment, Evaluation and Accountability

Erschienen in:

01.12.2011

Taking the grading leniency story to the edge. The influence of student, teacher, and course characteristics on student evaluations of teaching in higher education

verfasst von: Bert Brockx, Pieter Spooren, Dimitri Mortelmans

Erschienen in: Educational Assessment, Evaluation and Accountability | Ausgabe 4/2011

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

The use of student evaluation of teaching (SET) to evaluate and improve teaching is widespread amongst institutions of higher education. Many authors have searched for a conclusive understanding about the influence of student, course, and teacher characteristics on SET. One hotly debated discussion concerns the interpretation of the positive and statistically significant relationship that has been found between course grades and SET scores. In addition to reviewing the literature, the main purpose of the present study is to examine the influence of course grades and other characteristics of students, courses, and teachers on SET. Data from 1244 evaluations were collected using the SET-37 instrument and analyzed by means of cross-classified multilevel models. The results show positive significant relationships between course grades, class attendance, the examination period in which students receive their highest course grades, and the SET score. These relationships, however, are subject to different interpretations. Future research should focus on providing a definitive and empirically supported interpretation for these relationships. In the absence of such an interpretation, it will remain unclear whether these relationships offer proof of the validity of SET or whether they are a biasing factor.

Vorheriger Artikel The principal’s role in the post-evaluation process.—How does the principal engage in the work carried out after the schools self-evaluation?

Nächster Artikel Teacher factors and perceived assessment practices needs of social studies teachers in Cross River State, Nigeria

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Flanders is the Dutch-speaking part of Belgium.

Nederlands-Vlaamse Accreditatie Organisatie, Dutch-Flemish Accreditation Organisation.

In order to pass an examination, a student must achieve a score equivalent to at least 10 out of 20.

The 12 scales are as follows: clarity of objectives, value of subject matter, build-up of the subject matter, presentation skills, harmonious course organization – learning, contribution (of course materials) to understanding of the subject matter, course difficulty, help provided by the teacher during the learning process, authenticity of the examination, connection to advanced knowledge, content validity of the examinations, and formative examination.

These seven underlying dimensions are as follows: clarity of objectives, value of subject matter, build-up of subject matter, presentation skills, contribution (course materials) to understanding of the subject, course difficulty, and help provided by the teacher during the learning process.

The correlation matrix is available upon request from the authors.

The explained variance was calculated as the proportional reduction in the sum of the variance parameters (Snijders and Bosker 1999)

Apodaca, P., & Grad, H. (2005). The dimensionality of student ratings of teaching: integration of uni- and multidimensional models. Studies in Higher Education, 30(6), 723–748.CrossRef

Arnold, I. J. M. (2009). Do examinations influence student evaluations? International Journal of Educational Research, 48(4), 215–224.CrossRef

Bedard, K., & Kuhn, P. (2008). Where class size really matters: class size and student ratings of instructor effectiveness. Economics of Education Review, 27(3), 253–265.CrossRef

Beran, T., & Violato, C. (2005). Ratings of university teacher instruction: how much do student and course characteristics really matter? Assessment and Evaluation in Higher Education, 30(6), 593–601.CrossRef

Centra, J. A. (2003). Will teachers receive higher student evaluations by giving higher grades and less course work? Research in Higher Education, 44(5), 495–518.CrossRef

Centra, J. A., & Gaubatz, N. B. (2000). Is there gender bias in student evaluations of teaching? Journal of Higher Education, 71(1), 17–33.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences.: Lawrence Erlbaum associates.

Cohen, L., Manion, L., & Morrison, K. (2007). Research methods in education (6th ed.). London, New York: Routledge.

Darby, J. (2007). Are course evaluations subject to a halo effect? Research in Education, 77, 46–55.

Davidovitch, N., & Soen, D. (2006). Class attendance and students’ evaluation of their college instructors. College Student Journal, 40, 691–703.

Feldman, K. A. (1983). Seniority and experience of college-teachers as related to evaluations they receive from students. Research in Higher Education, 18(1), 3–124.CrossRef

Feldman, K. A. (1984). Class size and college students’ evaluations of teachers and courses: a closer look. Research in Higher Education, 21(1), 45–116.CrossRef

Feldman, K. A. (1997). Identifying exemplary teachers and teaching. Evidence from student ratings. In R. Perry, & J. Smart (Eds.), Effective teaching in higher education. Research and Practice (pp. 368–395).

Fernández, J., & Mateo, M. (1997). Student and faculty gender in ratings of university teaching quality. Sex Roles, 37(11), 997–1003.CrossRef

Gall, M. D., Gall, J. P., & Borg, W. R. (2007). Educational research: An introduction (8th ed.). Boston: Pearson/Allyn & Bacon.

Goldstein, H. (1995). Multilevel statistical models (2nd ed., Kendall’s library of statistics 3). London, New York: E. Arnold; Oxford University Press.

Gump, S. E. (2007). Student evaluations of teaching effectiveness and the leniency hypothesis: a literature review. Educational Research Quarterly, 30(3), 56–69.

Haladyna, T., & Amrein-Beardsley, A. (2009). Validation of a research-based student survey of instruction in a college of education. Educational Assessment, Evaluation and Accountability, 21(3), 255–276. doi:10.1007/s11092-008-9065-8.CrossRef

Heckert, T. M., Latier, A., Ringwald, A., & Silvey, B. (2006). Relation of course, instructor, and student characteristics to dimensions of student ratings of teaching effectiveness. College Student Journal, 40(1), 195–203.

Hofmann, D. A., & Gavin, M. B. (1998). Centering decisions in hierarchical linear models: implications for research in organizations. Journal of Management, 24(5), 623–641.

Honey, M. A. (1987). The interview as text: hermeneutics considered as a model for analyzing the clinically informed research interview. Human Development, 30(2), 69–82.CrossRef

Hox, J. (2002). Multilevel analysis techniques and applications.

Isely, P., & Singh, H. (2005). Do higher grades lead to favorable student evaluations? Journal of Economic Education(Winter), 29–42.

Krautmann, A. C., & Sander, W. (1999). Grades and student evaluations of teachers. Economics of Education Review, 18(1), 59–63.CrossRef

Kreft, I. G. G., Deleeuw, J., & Aiken, L. S. (1995). The effect of different forms of centering in hierarchical linear-models. Multivariate Behavioral Research, 30(1), 1–21.CrossRef

Kulik, J. A. (2001). Student ratings: validity, utility, and controversy. New Directions for institutional research(109), 9–25.

Lahey, J. L., & Griffith, J. C. (2002). Recent trends in higher education: accountability, efficiency, technology, and governance. Journal of Legal education, 52(4), 528–539.

Larsen, M. A. (2005). A critical analysis of teacher evaluation policy trends. Australian Journal of Education, 49(3), 292–305.

Marsh, H. W. (1984). Students’ evaluations of university teaching: dimensionality, reliability, validity, potential biases, and utility. Journal of Educational Psychology, 76(5), 707–754.CrossRef

Marsh, H. W. (1987). Students’ evaluations of university teaching: Research findings, methodological issues, and directions for future research.

Marsh, H. W. (1991a). A multidimensional perspective on students’ evaluations of teaching effectiveness: reply to Abrami and d’Apollonia (1991). Journal of Educational Psychology, 83(3), 416.CrossRef

Marsh, H. W. (1991b). Multidimensional students’ evaluations of teaching effectiveness: a test of alternative higher-order structures. Journal of Educational Psychology, 83(2), 285.CrossRef

Marsh, H. (2007a). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness. In The scholarship of teaching and learning in higher education: An evidence-based perspective (pp. 319–383).

Marsh, H. W. (2007b). Do university teachers become more effective with experience? A multilevel growth model of students’ evaluations of teaching over 13 years. Journal of Educational Psychology, 99(4), 775–790. doi:10.1037/0022-0663.99.4.775.CrossRef

Marsh, H. W., & Overall, J. U. (1981). The relative influence of course level, course type and instructor on students' evaluations of college teaching. American Educational Research Journal, 18(1), 103–112.

Marsh, H. W., & Roche, L. A. (2000). Effects of grading leniency and low workload on students’ evaluations of teaching: popular myth, bias, validity, or innocent bystanders? Journal of Educational Psychology, 92(1), 202–228.CrossRef

Marsh, H. W., Hau, K. T., Chung, C. M., & Siu, T. L. P. (1997). Students’ evaluations of university teaching: Chinese version of the students' evaluations of educational quality instrument. Journal of Educational Psychology, 89(3), 568–572.CrossRef

McPherson, M. A. (2006). Determinants of how students evaluate teachers. [Article]. Journal of Economic Education, 37, 3–20.CrossRef

McPherson, M. A., & Jewell, R. T. (2007). Leveling the playing field: should student evaluation scores be adjusted? Social Science Quarterly (Blackwell Publishing Limited), 88, 868–881.

McPherson, M. A., Jewell, R. T., & Myungsup, K. (2009). What determines student evaluation scores? A random effects analysis of undergraduate economics classes. Eastern Economic Journal, 35(1), 37–51.CrossRef

Mortelmans, D. & Spooren, P. (2009). A revalidation of the SET37-questionnaire for student evaluations of teaching. Educational Studies, 35, 547-552. doi:10.1080/03055690902880299.

Muthén, B. (2002). Beyond SEM: general latent variable modelling. Behaviormetrika, 29(1), 81–117.CrossRef

Nasser, F., & Hagtvet, K. A. (2006). Multilevel analysis of the effects of student and instructor/course characteristics on student ratings. Research in Higher Education, 47(5), 559–590.CrossRef

Norvilitis, J. M., & Zhang, J. (2009). The effect of perceived class mean on the evaluation of instruction. Educational Assessment, Evaluation and Accountability, 21(4), 299–311.CrossRef

Nowell, C. (2007). The impact of relative grade expectations on student evaluation of teaching. International Review of Economic Education, 6(2), 42–56.

Ory, J. C. (2001). Faculty thoughts and concerns about student ratings. New Directions for Teaching & Learning, (87), 3.

Pagani, L., & Seghieri, C. (2002). A statistical analysis of teaching effectiveness from students’ point of view. Metodološki zvezki - Advances in Methodology and Statistics, 17

Penny, A. R. (2003). Changing the agenda for research into students’ views about university teaching: four shortcomings of SRT research. Teaching in Higher Education, 8(3), 399–411.CrossRef

Shevlin, M., Banyard, P., Davies, M., & Griffiths, M. (2000). The validity of student evaluation of teaching in higher education: love Me, love my lectures? Assessment & Evaluation in Higher Education, 25(4), 397–405.CrossRef

Singer, J. D. (1998). Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. Journal of Educational and Behavioral Statistics, 23(4), 323–355. doi:10.3102/10769986023004323.

Snijders, T. A. B., & Bosker, R. J. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London; Thousand Oaks, Calif: Sage Publications.

Spooren, P. (2011). On the credibility of the judge : a cross-classified multilevel analysis on student evaluations of teaching? Studies in educational evaluation, 36(4), 121–131.

Spooren, P., & Mortelmans, D. (2006). Teacher professionalism and student evaluation of teaching: will better teachers receive higher ratings and will better students give higher ratings? Educational Studies, 32, 201–214. doi:10.1080/03055690600631101.

Spooren, P., Mortelmans, D., & Denekens, J. (2007). Student evaluation of teaching quality in higher education. Development of an instrument based on 10 Likert scales. Assessment and Evaluation in Higher Education, 32(6), 667–679. doi:10.1080/02602930601117191.

Stensaker, B., & Harvey, L. (2006). Old wine in new bottles? A comparison of Public and Private Accreditation Schemes in Higher education. Higher education Policy, 19(1), 65–85.CrossRef

Theall, M., & Franklin, J. (2001). Looking for bias in all the wrong places: a search for truth or a witch hunt in student ratings of instruction? New Directions for institutional research, 2001(109), 45–56.CrossRef

Ting, K-f. (2000). A multilevel perspective on student ratings of instruction: lessons from the chinese experience. Research in Higher Education, 41(5), 637–661.CrossRef

West, B. T., Welch, K. B., & Galecki, A. T. (2007). Linear mixed models: A practical guide using statistical software: Taylor & Francis Group.

Yao, Y., & Grady, M. (2005). How do faculty make formative use of student evaluation feedback?: a multiple case study. Journal of Personnel Evaluation in Education, 18(2), 107–126. doi:10.1007/s11092-006-9000-9.CrossRef

Titel: Taking the grading leniency story to the edge. The influence of student, teacher, and course characteristics on student evaluations of teaching in higher education
verfasst von: Bert Brockx
Pieter Spooren
Dimitri Mortelmans
Publikationsdatum: 01.12.2011
Verlag: Springer Netherlands
Erschienen in: Educational Assessment, Evaluation and Accountability / Ausgabe 4/2011
Print ISSN: 1874-8597
Elektronische ISSN: 1874-8600
DOI: https://doi.org/10.1007/s11092-011-9126-2

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 4/2011

The principal’s role in the post-evaluation process.—How does the principal engage in the work carried out after the schools self-evaluation?

The effects of civic education on political knowledge. A two year panel survey among Belgian adolescents

Editorial

Teacher factors and perceived assessment practices needs of social studies teachers in Cross River State, Nigeria