Skip to main content

2016 | OriginalPaper | Buchkapitel

Analysing RateMyProfessors Evaluations Across Institutions, Disciplines, and Cultures: The Tell-Tale Signs of a Good Professor

verfasst von : Mahmoud Azab, Rada Mihalcea, Jacob Abernethy

Erschienen in: Social Informatics

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Can we tell a good professor from their students’ comments? And are there differences between what is considered to be a good professor by different student groups? We use a large corpus of student evaluations collected from the RateMyProfessors website, covering different institutions, disciplines, and cultures, and perform several comparative experiments and analyses aimed to answer these two questions. Our results indicate that (1) we can reliably classify good professors from poor professors with an accuracy of over 90 %, and (2) we can separate the evaluations made for good professors by different groups with accuracies in the range of 71–89 %. Furthermore, a qualitative analysis performed using topic modeling highlights the aspects of interest for different student groups.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
2
The feature selection methods and the machine learning algorithms used in this study have been implemented in Python using the Sci-kit Learn machine learning library [16]. We use a maximum document frequency of 0.5 and lowercased text. We also experimented with stemming but it was found to degrade performance.
 
3
In each of these figures, the topic distributions for a group add up to 100 % (e.g., the blue/dark and yellow/light columns in Fig. 3 each add up to 100 %).
 
Literatur
1.
Zurück zum Zitat Agarwal, A., Biadsy, F., Mckeown, K.R.: Contextual phrase-level polarity analysis using lexical affect scoring and syntactic n-grams. In: Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics, pp. 24–32. Association for Computational Linguistics (2009) Agarwal, A., Biadsy, F., Mckeown, K.R.: Contextual phrase-level polarity analysis using lexical affect scoring and syntactic n-grams. In: Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics, pp. 24–32. Association for Computational Linguistics (2009)
2.
Zurück zum Zitat Bermingham, A., Smeaton, A.F.: Classifying sentiment in microblogs: is brevity an advantage? In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management, pp. 1833–1836. ACM (2010) Bermingham, A., Smeaton, A.F.: Classifying sentiment in microblogs: is brevity an advantage? In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management, pp. 1833–1836. ACM (2010)
3.
Zurück zum Zitat Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)MATH Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)MATH
4.
Zurück zum Zitat Boyd, R.L., Wilson, S.R., Pennebaker, J.W., Kosinski, M., Stillwell, D.J., Mihalcea, R.: Values in words: using language to evaluate and understand personal values. In: Ninth International AAAI Conference on Web and Social Media (2015) Boyd, R.L., Wilson, S.R., Pennebaker, J.W., Kosinski, M., Stillwell, D.J., Mihalcea, R.: Values in words: using language to evaluate and understand personal values. In: Ninth International AAAI Conference on Web and Social Media (2015)
5.
Zurück zum Zitat Brown, M.J., Baillie, M., Fraser, S.: Rating ratemyprofessors.com: a comparison of online and official student evaluations of teaching. Coll. Teach. 57(2), 89–92 (2009)CrossRef Brown, M.J., Baillie, M., Fraser, S.: Rating ratemyprofessors.com: a comparison of online and official student evaluations of teaching. Coll. Teach. 57(2), 89–92 (2009)CrossRef
6.
Zurück zum Zitat Coladarci, T., Kornfield, I.: Ratemyprofessors.com versus formal in-class student evaluations of teaching. Pract. Assess. Res. Eval. 12(6), 1–15 (2007) Coladarci, T., Kornfield, I.: Ratemyprofessors.com versus formal in-class student evaluations of teaching. Pract. Assess. Res. Eval. 12(6), 1–15 (2007)
7.
Zurück zum Zitat Felton, J., Koper, P.T., Mitchell, J., Stinson, M.: Attractiveness, easiness and other issues: student evaluations of professors on ratemyprofessors.com. Assess. Eval. High. Educ. 33(1), 45–61 (2008)CrossRef Felton, J., Koper, P.T., Mitchell, J., Stinson, M.: Attractiveness, easiness and other issues: student evaluations of professors on ratemyprofessors.com. Assess. Eval. High. Educ. 33(1), 45–61 (2008)CrossRef
8.
Zurück zum Zitat Felton, J., Mitchell, J., Stinson, M.: Web-based student evaluations of professors: the relations between perceived quality, easiness and sexiness. Assess. Eval. High. Educ. 29(1), 91–108 (2004)CrossRef Felton, J., Mitchell, J., Stinson, M.: Web-based student evaluations of professors: the relations between perceived quality, easiness and sexiness. Assess. Eval. High. Educ. 29(1), 91–108 (2004)CrossRef
9.
Zurück zum Zitat Freng, S., Webber, D.: Turning up the heat on online teaching evaluations: does hotness matter? Teach. Psychol. 36(3), 189–193 (2009)CrossRef Freng, S., Webber, D.: Turning up the heat on online teaching evaluations: does hotness matter? Teach. Psychol. 36(3), 189–193 (2009)CrossRef
10.
Zurück zum Zitat Helterbran, V.R.: The ideal professor: student perceptions of effective instructor practices, attitudes, and skills. Education 129(1), 125 (2008) Helterbran, V.R.: The ideal professor: student perceptions of effective instructor practices, attitudes, and skills. Education 129(1), 125 (2008)
11.
Zurück zum Zitat Kim, S.M., Hovy, E.: Determining the sentiment of opinions. In: Proceedings of the 20th International Conference on Computational Linguistics, p. 1367. Association for Computational Linguistics (2004) Kim, S.M., Hovy, E.: Determining the sentiment of opinions. In: Proceedings of the 20th International Conference on Computational Linguistics, p. 1367. Association for Computational Linguistics (2004)
12.
Zurück zum Zitat Lu, Y., Zhai, C.: Opinion integration through semi-supervised topic modeling. In: Proceedings of the 17th International Conference on World Wide Web, pp. 121–130. ACM (2008) Lu, Y., Zhai, C.: Opinion integration through semi-supervised topic modeling. In: Proceedings of the 17th International Conference on World Wide Web, pp. 121–130. ACM (2008)
14.
Zurück zum Zitat Otto, J., Sanford, D.A., Ross, D.N.: Does ratemyprofessor.com really rate my professor? Assess. Eval. High. Educ. 33(4), 355–368 (2008)CrossRef Otto, J., Sanford, D.A., Ross, D.N.: Does ratemyprofessor.com really rate my professor? Assess. Eval. High. Educ. 33(4), 355–368 (2008)CrossRef
15.
Zurück zum Zitat Pang, B., Lee, L.: A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts. In: Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, p. 271. Association for Computational Linguistics (2004) Pang, B., Lee, L.: A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts. In: Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, p. 271. Association for Computational Linguistics (2004)
16.
Zurück zum Zitat Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetMATH Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetMATH
17.
Zurück zum Zitat Stone, P., Earl, B.: A computer approach to content analysis: studies using the general inquirer system. In: Proceedings of the Spring Joint Computer Conference, ACM (1963) Stone, P., Earl, B.: A computer approach to content analysis: studies using the general inquirer system. In: Proceedings of the Spring Joint Computer Conference, ACM (1963)
18.
Zurück zum Zitat Strapparava, C., Valitutti, A.: Wordnet affect: an affective extension of wordnet. In: Proceedings of the 4th International Conference on Language Resources and Evaluation, pp. 1083–1086 (2004) Strapparava, C., Valitutti, A.: Wordnet affect: an affective extension of wordnet. In: Proceedings of the 4th International Conference on Language Resources and Evaluation, pp. 1083–1086 (2004)
19.
Zurück zum Zitat Timmerman, T.: On the validity of ratemyprofessors.com. J. Educ. Bus. 84(1), 55–61 (2008)CrossRef Timmerman, T.: On the validity of ratemyprofessors.com. J. Educ. Bus. 84(1), 55–61 (2008)CrossRef
20.
Zurück zum Zitat Turney, P.D.: Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 417–424. Association for Computational Linguistics (2002) Turney, P.D.: Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 417–424. Association for Computational Linguistics (2002)
21.
Zurück zum Zitat Wilson, T., Wiebe, J., Hoffmann, P.: Recognizing contextual polarity in phrase-level sentiment analysis. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, pp. 347–354. Association for Computational Linguistics (2005) Wilson, T., Wiebe, J., Hoffmann, P.: Recognizing contextual polarity in phrase-level sentiment analysis. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, pp. 347–354. Association for Computational Linguistics (2005)
Metadaten
Titel
Analysing RateMyProfessors Evaluations Across Institutions, Disciplines, and Cultures: The Tell-Tale Signs of a Good Professor
verfasst von
Mahmoud Azab
Rada Mihalcea
Jacob Abernethy
Copyright-Jahr
2016
DOI
https://doi.org/10.1007/978-3-319-47880-7_27

Neuer Inhalt