Skip to main content
SpringerLink
Log in

MULTILEVEL ANALYSIS OF THE EFFECTS OF STUDENT AND INSTRUCTOR/COURSE CHARACTERISTICS ON STUDENT RATINGS

  • Published:
Research in Higher Education Aims and scope Submit manuscript

Abstract

Multilevel SEM was used to examine the extent to which student, instructor, and course characteristics affect student ratings. Data were gathered from 1867 students enrolled in 117 courses at a large teacher training college in Israel. Four alternative two-level models that differ in only the nature of the relationship among interest in the course subject, expected grade, and student ratings were tested. Two of the models were judged as less appropriate, one because it failed to support the spurious relationship assumed between expected grade and student ratings, and the other on grounds of poor model-data fit. The other two models were equally good both in terms of the model-data fit and the amount of variance in student ratings that is accounted for by each of them. Both models supported the mediation effect of expected grade in the relationship between interest in the course subject and student ratings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

FIG. 1.
FIG. 2.
FIG. 3.

Similar content being viewed by others

References

  • Aleamoni, M. L. (November, 1978). Are there differences in perceived teaching effectiveness between males and females in anthropology? Paper Presented at the Annual Meeting of the American Anthropological Association, Los Angeles, California

  • Avi-Itzhak T., Kremer L. (1986) An investigation into the relationship between university faculty attitudes toward student rating and organizational and background factors. Educational Research Quarterly 10(2): 31–38

    Google Scholar 

  • Bachen M. C., McLoughlin M. M., Garcia S. S. (1999) Assessing the role of gender in college students’ evaluations of faculty. Communication Education 48(3): 193–210

    Article  Google Scholar 

  • Baron R. M., Kenny D. A. (1986) The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology 51(6): 1173–1182

    Article  Google Scholar 

  • Basow S. A., Howe K. G. (1987) Evaluation of college professors: Effects of professors’ sex type and sex, and student’s sex. Psychological Reports 60(2): 671–678

    Google Scholar 

  • Bonetti S. (1994) On the use of student questionnaires. Higher Education Review 26(3): 57–64

    Google Scholar 

  • Centra J. A. (1979) Determining Faculty Effectiveness. Jossey-Bass, San Francisco

    Google Scholar 

  • Centra J. A. (1993) Reflective Faculty Evaluation. Jossey-Bass, San Francisco

    Google Scholar 

  • Centra A. J., and Gaubatz B. N. (2000) Is there gender bias in student evaluations of teaching? Journal of Higher Education 71(1): 17–33

    Article  Google Scholar 

  • Chau C.-T. (1997) A Bootstrap experiment on the statistical properties of students’ ratings of teaching effectiveness. Research in Higher Education 38(4): 497–517

    Article  Google Scholar 

  • Delaney, L. Jr. E. (April, 1976). The relationships of student ratings of instruction to student, instructor and course characteristics. Paper Presented at the Annual Meeting of the American Educational Research Association, San Francisco, California

  • Eiszler C. F. (2002) College student evaluations of teaching and grade inflation. Research in Higher Education 43(4): 483–501

    Article  Google Scholar 

  • Feldman K. A. (1976a) Grades and college students’ evaluation of their courses and teachers. Research in Higher Education 4(1): 69–111

    Article  Google Scholar 

  • Feldman K. A. (1976b) The superior college teacher from the student’s view. Research in Higher Education 5(3): 243–288

    Article  Google Scholar 

  • Feldman K. A. (1983) Seniority and experience of college teachers as related to evaluations they receive from students. Research in Higher Education 18(1): 3–124

    Article  Google Scholar 

  • Feldman K. A. (1984) Class size and college students’ evaluations of teachers and courses: A closer look. Research in Higher Education 21(1): 45–116

    Article  Google Scholar 

  • Feldman K. A. (1993) College students’ views of male and female college teachers: Part II–evidence from students’ evaluations of their classroom teachers. Research in Higher Education 33(3): 317–375

    Article  Google Scholar 

  • Fernandez J., Mateo A. A., Muniz J. (1998) Is there a relationship between class size and student ratings of teacher quality? Educational and Psychological Measurement 58(4): 596–604

    Article  Google Scholar 

  • Fourie C. M. (2000) Class size and student ratings of teaching presentation. South African Journal of Higher Education 14(3): 132–138

    Google Scholar 

  • Goldberg G., Callahan J. (1991) Objectivity of student evaluations of instructors. Journal of Education for Business 66(6): 377–378

    Google Scholar 

  • Greenwald A. G., Gillmore G. M. (1997) Grading leniency is a removable contaminant of student ratings. American Psychologist 52(11): 1209–1217

    Article  Google Scholar 

  • Hancock R. G., Shannon M. D., Trentham L. L. (1992) Student and teacher gender in ratings of university faculty: results from five colleges of study. Journal of Personnel Evaluation in Education 6(3): 235–248

    Article  Google Scholar 

  • Haugen E. R. (1984) Educationists and academics: Ratings and community college instructors. Community/Junior College Quarterly of Research and Practice 8(1–4): 103–113

    Article  Google Scholar 

  • Howard G. C., Schmeck R. R. (1979) Relationship of changes in student motivation to student evaluations of instruction. Research in Higher Education 10: 305–315

    Article  Google Scholar 

  • Hu L. T., Bentler P. M. (1999) Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling 6(1): 1–55

    Google Scholar 

  • Jackson D. L., Teal C. R., Raines S. J., Nansel T. R., Force R. C., Burdsal C. A. (1999) The dimensions of students’ perceptions of teaching effectiveness. Educational and Psychological Measurement 59(4): 580–596

    Article  Google Scholar 

  • Jöreskog K. G. (1977) Structural equation modeling in the social sciences: Specification, estimation, and testing. In: Krishnaiah P. R. (eds.), Applications of Statistics, North-Holland, Amsterdam, pp. 265–287

    Google Scholar 

  • Jöreskog K.G. (1993) Testing structural equation model. In: Bollen K. A., Scott Long J. (eds) Testing Structural Equation Models. Sage, Newbury Park, CA, pp. 294–316

    Google Scholar 

  • Jöreskog K. G., Sörbom D. (1996) LISREL 8: User’s Reference Guide. Scientific Software International, Chicago, IL

    Google Scholar 

  • Kaplan D., Elliot R. P. (1997) A didactic example of multilevel structural equation modeling applicable to study of organizations. Structural Equation Modeling 4(1): 1–4

    Google Scholar 

  • Keller L.M. (1983) Motivational design instruction. In: Reigeluth C.M. (eds) Instructional Design Theories and Models: An Overview of Their Current Status. Erlbaum, Hillsdale, NJ

    Google Scholar 

  • Koon J. (1986) Teaching evaluation: Theory and practice for a student position. National Issues in Higher Education 22: 265–274

    Google Scholar 

  • Koon J., Murray H. G. (1995) Using multiple outcomes to validate student ratings of overall teacher effectiveness. Journal of Higher Education 66(1): 61–81

    Article  Google Scholar 

  • Kwan K.-P. (1999) How fair are student ratings in assessing the teaching performance of university teachers? Assessment & Evaluation in Higher Education 24(2): 181–195

    Article  Google Scholar 

  • Krapp A. (2002) Structural and dynamic aspects of interest development: Theoretical considerations from an ontogenetic perspective. Learning and Instruction 12(4): 383–409

    Article  Google Scholar 

  • Lin W. (1992) Is class size a bias to student evaluations of faculty? A review. Chinese University Education Journal 20(1): 49–53

    Google Scholar 

  • Marsh H. W. (1983) Multidimensional ratings of teaching effectiveness by students from different academic settings and their relation to student/course/instructor characteristics. Journal of Educational Psychology 75(1): 150–166

    Article  Google Scholar 

  • Marsh H. W. (1984) Student’s evaluations of university teaching: Dimensionality, reliability, validity, potential biases, and utility. Journal of Educational Psychology 76(5): 707–754

    Article  Google Scholar 

  • Marsh H. W. (1987) Students’ evaluations of university teaching: Research findings, methodological issues, and directions for future research [Entire issue]. International Journal of Educational Research 11(3): 263–388

    Article  Google Scholar 

  • Marsh H. W. (1989) Responses to review of “students’ evaluations of university teaching: Research findings, methodological issues, and directions for future research”. Instructional Evaluation 10: 5–9

    Google Scholar 

  • Marsh H. W., Cooper L. T. (1981) Prior subject interest, students’ evaluations, and instructional effectiveness. Multivariate Behavioral Research 16(1): 83–104

    Article  Google Scholar 

  • Marsh H. W., Dunkin M. (1997) Student evaluation of university teaching: A multidimensional perspective. In: Perry P. R., Smart J. C. (eds) Effective Teaching in Higher Education: Research and Practice. Agathon, New York, pp. 241–320

    Google Scholar 

  • Marsh H. W., Rochi L. A. (1997) Making students’ evaluations of teaching effectiveness effective. American Psychologist 52(11): 1187–1197

    Article  Google Scholar 

  • Marsh H. W., Rochi L. A. (2000) Effects of grading leniency and low workload on students’ evaluations of popular myth, bias, validity, or innocent bystanders? Journal of Educational Psychology 92(1): 202–228

    Article  Google Scholar 

  • Marsh H. W., Overall U. J., Kesler P. S. (1979) Validity of student evaluation of instructional effectiveness: A comparison of faculty self-evaluations and evaluation by their students. Journal of Educational Psychology 71(2): 149–160

    Article  Google Scholar 

  • Mason P. M., Steagall J. W., Fabritius M. M. (1995) Student evaluations of faculty: A new procedure for aggregate measures of performance. Economics of Education Review 14(4): 403–416

    Article  Google Scholar 

  • Mateo A., Fernandez J. (1996) Incidence of class size on the evaluation of university teaching quality. Educational and Psychological Measurement 56(5): 771–778

    Article  Google Scholar 

  • McKee G. B., Dowalibly J. F. (1985) The relationship between student, course and instructor characteristics and hearing impaired students’ ratings of instruction. Volta Review 87(2): 77–86

    Google Scholar 

  • Menges R. J. (1973) The new reporters: Students rate instruction. New Directions for Higher Education 1(4): 59–76

    Article  Google Scholar 

  • Murphy P. K., Alexander A. P. (2002) What counts? The predictive powers of subject-matter knowledge, strategic processing, and interest in domain-specific performance. Journal of Experimental Education 70(3): 197–214

    Article  Google Scholar 

  • Murray G. H., Rushton J. P., Paunonen V. S. (1990) Teacher personality traits and student instructional ratings in six types of university courses. Journal of Educational Psychology 82(2): 250–261

    Article  Google Scholar 

  • Muthén B. (1989) Latent variable modeling in heterogeneous populations. Psychometrika 54(4): 557–585

    Article  Google Scholar 

  • Muthén, B. (1991) Multilevel factor analysis of class and student achievement components. Journal of Educational Measurement 28(4): 338–354

    Article  Google Scholar 

  • Muthén, B. (1994). Multilevel covariance structure analysis. In: Hox, J., and Kreft, I. (eds.), Multilevel Modeling, a special issue of Sociological Methods & Research 22: 376–398

  • Muthén L. K., Muthén B. O. (1998). Mplus User’s Guide. Muthén& Muthén, Los Angeles, CA

    Google Scholar 

  • Muthén, L. K., and Muthén, B. O. (2003). Mplus Version 2.13. Addendum to the Mplus User’s Guide. Available at: http://www.statmodel.com/version2.html

  • Muthén B., Sattora A. (1989) Multilevel aspects of varying parameters in structural models. In: Bock R. D. (eds) Multilevel Analysis of Educational Data. Academic, San Diego, CA, pp. 87–99

    Google Scholar 

  • Nasser F., Fresko B. (2002) Faculty views of student evaluation of college teaching. Assessment & Evaluation in Higher Education 27(2): 187–198

    Article  Google Scholar 

  • Olivares J. O. (2002) A conceptual and analytical critique of student ratings of teachers in the USA with implications for teacher effectiveness and student teaching. Teaching in Higher Education 8(2): 233–245

    Article  Google Scholar 

  • Petchers K. M., Chow C. J. (1988) Sources of variation in students’ evaluations of instruction in a graduate social work Program. Journal of Social Work Education 24(1): 35–42

    Google Scholar 

  • Rabalais, J. M. (1977, July). The relationship between number of graduate semester hours, years of teaching experience and student evaluation of overall rating of instructors for a selected sample of junior college faculty. Paper Presented at the Annual Conference of the Southeast Region AERA Special Interest Group in Community College Research, New Orleans

  • Roundenbush S. W., Willms J. D. (eds) (1991) Schools, Classrooms, and Pupils: International Studies of Schooling from a Multilevel Perspective. Academic Press, Orlando, Fl

    Google Scholar 

  • Sailor P., Worthen B., Shin E. H. (1997) Class level as a possible mediator of the relationship between grades and student ratings of teaching. Assessment & Evaluation in Higher Education 22(3): 261–269

    Article  Google Scholar 

  • Schmelkin-Padhazur L., Spencer K. J., Gellman E. S. (1997) Faculty perspectives on course and teacher evaluation. Research in Higher Education 38(5): 575–592

    Article  Google Scholar 

  • Stapleton J. R., Murkison G. (2001) Optimizing the fairness of student evaluations: A study of correlations between instructor excellence, study production, learning production, and expected grades. Journal of Management Education 25(3): 269–291

    Article  Google Scholar 

  • Tatro N. C. (1995) Gender effects on student evaluations of faculty. Journal of Research and Development in Education 28(3): 169–173

    Google Scholar 

  • Tenko R., Marcoulides G. (2000) First Course in Structural Equation Modeling. Erlbaum, Mahwah, NJ

    Google Scholar 

  • Theall M., Franklin J. (1990) Student Ratings in the Context of Complex Evaluation Systems. Jossey Bass, San Francisco, pp. 17–34

    Google Scholar 

  • Tieman R. C., Ranklin-Ullock B. (1985) Student evaluations of teachers: An examination of the effect of sex and field study. Teaching Sociology 12(2): 1977–191

    Article  Google Scholar 

  • Young S., Shaw G. D. (1999) Profiles of effective college and university teachers. Journal of Higher Education 70(6): 670–686

    Article  Google Scholar 

  • Wachtel K. H. (1998) Student evaluation of college teaching effectiveness: A brief review. Assessment & Evaluation in Higher Education 23(2): 191–211

    Article  Google Scholar 

  • Wigington H., Tollefson N., Rodriguez E. (1989) Students’ ratings of instructors revisited: Interactions among class and instructor variables. Research in Higher Education 30(3): 331–344

    Article  Google Scholar 

  • Willet H. L. (1980) Comparison of institutional effectiveness of full-and part-time faculty. Community/Junior College Research Quarterly 5(1): 23–30

    Article  Google Scholar 

  • Worrell C. F., Kuterbach D. L. (2001) The use of student ratings of teacher behaviors with academically talented high school students. Journal of Secondary Gifted Education 12(4): 236–247

    Google Scholar 

  • Worthington A. C. (2002) The impact of student perceptions and characteristics on teaching evaluations: A case study in finance education. Assessment & Evaluation in Higher Education 27(1): 49–64

    Article  Google Scholar 

  • Zahn K. D., Schramm M. R. (1992) Student perception of teacher effectiveness based on teacher employment and course skill level. Business Education Forum 46(3): 16–18

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Education, Tel Aviv University, Ramat Aviv, Israel

    Fadia Nasser

  2. Department of Psychology, University of Oslo, Oslo, Norway

    Knut A. Hagtvet

  3. School of Education, Tel Aviv University, Ramat Aviv, 69978, Israel

    Fadia Nasser

Authors
  1. Fadia Nasser
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Knut A. Hagtvet
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fadia Nasser.

Appendices

Appendix A

Items on the Evaluation Questionnaire

  1. 1.

    The instructor teaches the course according to the syllabus.

  2. 2.

    This course can contribute to my professional future.

  3. 3.

    In this course, I am given the opportunity to demonstrate my knowledge and competence.

  4. 4.

    The instructor creates a pleasant learning environment.

  5. 5.

    The instructor defined course requirements clearly.

  6. 6.

    Course content is stimulating.

  7. 7.

    The instructor is tolerant of students’ views that differ from his/her own.

  8. 8.

    In this course there are many opportunities for students to be active.

  9. 9.

    The instructor provides clear instructions for all assignments.

  10. 10.

    The instructor encourages the exchange of ideas in class.

  11. 11.

    The content of the course is presented in an interesting way.

  12. 12.

    Course assignments contribute to my understanding of the course materials.

  13. 13.

    Class time is efficiently used.

  14. 14.

    Discussions in class contribute to my understanding of course materials.

  15. 15.

    Course assignments are too difficult.

  16. 16.

    The instructor is not considerate of individual differences among students.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Nasser, F., Hagtvet, K. MULTILEVEL ANALYSIS OF THE EFFECTS OF STUDENT AND INSTRUCTOR/COURSE CHARACTERISTICS ON STUDENT RATINGS. Res High Educ 47, 559–590 (2006). https://doi.org/10.1007/s11162-005-9007-y

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11162-005-9007-y

Keywords

Search

Navigation