Skip to main content
Erschienen in: International Journal of Technology and Design Education 1/2017

23.09.2015

Towards effective group work assessment: even what you don’t see can bias you

verfasst von: Gahgene Gweon, Soojin Jun, Susan Finger, Carolyn Penstein Rosé

Erschienen in: International Journal of Technology and Design Education | Ausgabe 1/2017

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In project-based learning (PBL) courses, which are common in design and technology education, instructors regard both the process and the final product to be important. However, conducting an accurate assessment for process feedback is not an easy task because instructors of PBL courses often have to make judgments based on a limited view of group work. In this paper, we provide explanations about how in practice instructors actually exhibit cognitive biases and judgments made using incomplete information in the context of an engineering design education classroom. More specifically, we hypothesize that instructors would be susceptible to human errors that are well known in social psychology, the halo effect and the fundamental attribution error, because they have a limited view of group work when they facilitate distributed and remote groups. Through this study, we present two main contributions, namely (1) insights based on classroom data about limitations of current instructor assessment practices, (2) an illustration of using principles from social psychology as a lens for exploring important design questions for designing tools that monitor support oversight of group work. In addition to the study, we illustrate how the findings from our classroom study can be used for effective group assessments.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Balzer, W. K., & Sulsky, L. M. (1992). Halo and performance appraisal research: A critical examination. Journal of Applied Psychology, 77(6), 975–985.CrossRef Balzer, W. K., & Sulsky, L. M. (1992). Halo and performance appraisal research: A critical examination. Journal of Applied Psychology, 77(6), 975–985.CrossRef
Zurück zum Zitat Becker, B. E., & Cardy, R. L. (1986). Influence of halo error on appraisal effectiveness: A conceptual and empirical reconsideration. Journal of Applied Psychology, 71(4), 662–671.CrossRef Becker, B. E., & Cardy, R. L. (1986). Influence of halo error on appraisal effectiveness: A conceptual and empirical reconsideration. Journal of Applied Psychology, 71(4), 662–671.CrossRef
Zurück zum Zitat Beehr, T. A., Ivanitskaya, L., Hansen, C. P., Erofeev, D., & Gudanowski, D. M. (2001). Evaluation of 360 degree feedback ratings: Relationships with each other and with performance and selection predictors. Journal of Organizational Behavior, 22(7), 775–788.CrossRef Beehr, T. A., Ivanitskaya, L., Hansen, C. P., Erofeev, D., & Gudanowski, D. M. (2001). Evaluation of 360 degree feedback ratings: Relationships with each other and with performance and selection predictors. Journal of Organizational Behavior, 22(7), 775–788.CrossRef
Zurück zum Zitat Bober, M., Sullivan, H., Lowther, D., & Harrison, P. (1998). Instructional practices of teachers enrolled in educational technology and general educational programs. Educational Technology Research and Development, 46(3), 81–97.CrossRef Bober, M., Sullivan, H., Lowther, D., & Harrison, P. (1998). Instructional practices of teachers enrolled in educational technology and general educational programs. Educational Technology Research and Development, 46(3), 81–97.CrossRef
Zurück zum Zitat Burger, J. (1991). Changes in attributions over time: The ephemeral fundamental attribution error. Social Cognition, 9(2), 182–193.CrossRef Burger, J. (1991). Changes in attributions over time: The ephemeral fundamental attribution error. Social Cognition, 9(2), 182–193.CrossRef
Zurück zum Zitat Carroll, J. S. (1978). Causal attributions in expert parole decisions. Journal of Personality and Social Psychology, 36(12), 1501–1511.CrossRef Carroll, J. S. (1978). Causal attributions in expert parole decisions. Journal of Personality and Social Psychology, 36(12), 1501–1511.CrossRef
Zurück zum Zitat Chen, M. (2003). Visualizing the pulse of a classroom. In Proceedings of international multimedia conference, pp. 555–561, Berkeley, CA: ACM Press. Chen, M. (2003). Visualizing the pulse of a classroom. In Proceedings of international multimedia conference, pp. 555–561, Berkeley, CA: ACM Press.
Zurück zum Zitat Cook, M., & Klumper, D. (1999). Metacognitive, social and interpersonal skills and aptitudes in officer performance with distributed teams. In Paper presented at RTO HFM workshop on Officer Selection, in Monterey, USA. Cook, M., & Klumper, D. (1999). Metacognitive, social and interpersonal skills and aptitudes in officer performance with distributed teams. In Paper presented at RTO HFM workshop on Officer Selection, in Monterey, USA.
Zurück zum Zitat Cooper, W. (1981). Ubiquitous halo. Psychological Bulletin, 90(2), 218–244.CrossRef Cooper, W. (1981). Ubiquitous halo. Psychological Bulletin, 90(2), 218–244.CrossRef
Zurück zum Zitat DiMicco, J., Hoolenbach, K., & Bender, W. (2006). Using visualizations to review a group’s interaction dynamics. In Proceedings of the SIGCHI conference on human factors in computing systems, pp 706–711, New York: ACM Press. DiMicco, J., Hoolenbach, K., & Bender, W. (2006). Using visualizations to review a group’s interaction dynamics. In Proceedings of the SIGCHI conference on human factors in computing systems, pp 706–711, New York: ACM Press.
Zurück zum Zitat Dimitracopoulou, A., Hoppe, U., & Dillenbourg, P. (2004). Interaction analysis supporting participants during technology based collaborative activities. In Paper presented at CSCL symposium, October 7–9, in Kaleidoscope Noe, Lausanne. Dimitracopoulou, A., Hoppe, U., & Dillenbourg, P. (2004). Interaction analysis supporting participants during technology based collaborative activities. In Paper presented at CSCL symposium, October 7–9, in Kaleidoscope Noe, Lausanne.
Zurück zum Zitat Dutson, A. J., Todd, R. H., Magleby, S. P., & Sorensen, C. D. (1997). A review of literature on teaching design through project-oriented capstone courses. Journal of Engineering Education, 76(1), 17–28.CrossRef Dutson, A. J., Todd, R. H., Magleby, S. P., & Sorensen, C. D. (1997). A review of literature on teaching design through project-oriented capstone courses. Journal of Engineering Education, 76(1), 17–28.CrossRef
Zurück zum Zitat Feeley, T. (2002). Evidence of halo effects in student evaluations of communication instruction. Communication Education, 51(3), 225–236.CrossRef Feeley, T. (2002). Evidence of halo effects in student evaluations of communication instruction. Communication Education, 51(3), 225–236.CrossRef
Zurück zum Zitat Gipps, C. (2005). What is the role for ICT-based assessment in universities? Studies in Higher Education, 30(2), 171–180.CrossRef Gipps, C. (2005). What is the role for ICT-based assessment in universities? Studies in Higher Education, 30(2), 171–180.CrossRef
Zurück zum Zitat Gómez Puente, S. M., van Eijck, M., & Jochems, W. (2013). Empirical validation of characteristics of design-based learning in higher education. International Journal of Engineering Education, 29(2), 491–503. Gómez Puente, S. M., van Eijck, M., & Jochems, W. (2013). Empirical validation of characteristics of design-based learning in higher education. International Journal of Engineering Education, 29(2), 491–503.
Zurück zum Zitat Gopinath, C. (1999). Alternatives to instructor assessment of class participation. Journal of Education for Business, 75(1), 10–14.CrossRef Gopinath, C. (1999). Alternatives to instructor assessment of class participation. Journal of Education for Business, 75(1), 10–14.CrossRef
Zurück zum Zitat Harvey, J., Town, J. P., & Yarkim, K. (1981). How fundamental is the fundamental attribution error? Journal of Personality and Social Psychology, 40(2), 346–349.CrossRef Harvey, J., Town, J. P., & Yarkim, K. (1981). How fundamental is the fundamental attribution error? Journal of Personality and Social Psychology, 40(2), 346–349.CrossRef
Zurück zum Zitat Jochems, W., & Kreijns, K. (2006). Measuring social aspects of distributed learning groups. European Educational Research Journal, 5(2), 110–121.CrossRef Jochems, W., & Kreijns, K. (2006). Measuring social aspects of distributed learning groups. European Educational Research Journal, 5(2), 110–121.CrossRef
Zurück zum Zitat Johari, A., & Bradshaw, A. (2008). Project-based learning in an internship program: A qualitative study of related roles and their motivational attributes. Education Technology Research and Development, 56, 329–359.CrossRef Johari, A., & Bradshaw, A. (2008). Project-based learning in an internship program: A qualitative study of related roles and their motivational attributes. Education Technology Research and Development, 56, 329–359.CrossRef
Zurück zum Zitat Kay, J., Maisonneuve, N., Yacef, K., & Reimann, P. (2006). Wattle tree: What’ll it tell us? University of Sydney Technical Report. Kay, J., Maisonneuve, N., Yacef, K., & Reimann, P. (2006). Wattle tree: What’ll it tell us? University of Sydney Technical Report.
Zurück zum Zitat Kelsey, D. M., Kearney, P., Plax, T. G., Allen, T. H., & Ritter, K. J. (2004). College students’ attributions of teacher misbehaviors. Communication Education, 53(1), 40–55.CrossRef Kelsey, D. M., Kearney, P., Plax, T. G., Allen, T. H., & Ritter, K. J. (2004). College students’ attributions of teacher misbehaviors. Communication Education, 53(1), 40–55.CrossRef
Zurück zum Zitat Kimbell, R. (2007). E-assessment in project e-scape. Design and Technology Education: An International Journal, 12(2), 66–76. Kimbell, R. (2007). E-assessment in project e-scape. Design and Technology Education: An International Journal, 12(2), 66–76.
Zurück zum Zitat Lambart, E., Sharma, A., & Levy, M. (1997). What information can relationship marketers obtain from customer evaluations of salespeople? Industrial Marketing Management, 26(2), 177–187.CrossRef Lambart, E., Sharma, A., & Levy, M. (1997). What information can relationship marketers obtain from customer evaluations of salespeople? Industrial Marketing Management, 26(2), 177–187.CrossRef
Zurück zum Zitat Madan, A., Caneel, R., and Pentland, A. (2004). GroupMedia: Distributed multimodal interfaces. In Proceedings of sixth international conference on multimodal interfaces ICMI04. Madan, A., Caneel, R., and Pentland, A. (2004). GroupMedia: Distributed multimodal interfaces. In Proceedings of sixth international conference on multimodal interfaces ICMI04.
Zurück zum Zitat McPherson, M., & Young, S. L. (2004). What students think when teachers get upset: Fundamental attribution error and student generated reasons for teacher anger. Communication Quarterly, 52(4), 357–369.CrossRef McPherson, M., & Young, S. L. (2004). What students think when teachers get upset: Fundamental attribution error and student generated reasons for teacher anger. Communication Quarterly, 52(4), 357–369.CrossRef
Zurück zum Zitat Meier, A., Spada, H., & Rummel, N. (2007). Evaluating collaboration: A rating scheme for assessing the quality of collaborative process. International Journal of Computer-Supported Collaborative Learning, 2, 63–86.CrossRef Meier, A., Spada, H., & Rummel, N. (2007). Evaluating collaboration: A rating scheme for assessing the quality of collaborative process. International Journal of Computer-Supported Collaborative Learning, 2, 63–86.CrossRef
Zurück zum Zitat Nicol, D., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218.CrossRef Nicol, D., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218.CrossRef
Zurück zum Zitat Phielix, C., Prins, F., & Kirschner, P. (2010). Awareness of group performance in a CSCL-environment: Effects of peer feedback and reflection. Computers and Human Behavior, 26(2), 151–161.CrossRef Phielix, C., Prins, F., & Kirschner, P. (2010). Awareness of group performance in a CSCL-environment: Effects of peer feedback and reflection. Computers and Human Behavior, 26(2), 151–161.CrossRef
Zurück zum Zitat Pianesi, F., Zancarnaro, M., Not, E., Leonardi, C., Falcon, V., & Lepri, B. (2008). Multimodal support to group dynamics. Personal and Ubiquitous Computing, 12(3), 181–195.CrossRef Pianesi, F., Zancarnaro, M., Not, E., Leonardi, C., Falcon, V., & Lepri, B. (2008). Multimodal support to group dynamics. Personal and Ubiquitous Computing, 12(3), 181–195.CrossRef
Zurück zum Zitat Price, M., Handley, K., & Millar, J. (2011). Feedback: Focusing attention on engagement. Studies in Higher Education, 36(8), 879–896.CrossRef Price, M., Handley, K., & Millar, J. (2011). Feedback: Focusing attention on engagement. Studies in Higher Education, 36(8), 879–896.CrossRef
Zurück zum Zitat Reimann, P., Yacef, K., & Kay, J. (2011). Analyzing collaborative interactions with data mining methods for the benefit of learning. In S. Puntambekar, G. Erkens, & C. Hmelo-Silver (Eds.), Analyzing interactions in CSCL: Methods, approaches and issue (pp. 161–186). New York: Springer.CrossRef Reimann, P., Yacef, K., & Kay, J. (2011). Analyzing collaborative interactions with data mining methods for the benefit of learning. In S. Puntambekar, G. Erkens, & C. Hmelo-Silver (Eds.), Analyzing interactions in CSCL: Methods, approaches and issue (pp. 161–186). New York: Springer.CrossRef
Zurück zum Zitat Ren, Y., Kiesler, S., & Fussell, S. R. (2008). Multiple group coordination in complex and dynamic task environments: Interruptions, coping mechanisms, and technology recommendations. Journal of Management Information Systems, 25(1), 107–133.CrossRef Ren, Y., Kiesler, S., & Fussell, S. R. (2008). Multiple group coordination in complex and dynamic task environments: Interruptions, coping mechanisms, and technology recommendations. Journal of Management Information Systems, 25(1), 107–133.CrossRef
Zurück zum Zitat Rienks, R.J., Zhang, D., Gatica-Perez, D., & Post, W. (2006) Detection and application of influence rankings in small group meetings. In Proceedings of eighth international conference on multimodal interfaces ICMI’06. Rienks, R.J., Zhang, D., Gatica-Perez, D., & Post, W. (2006) Detection and application of influence rankings in small group meetings. In Proceedings of eighth international conference on multimodal interfaces ICMI’06.
Zurück zum Zitat Rohde, M., Klamma, R., Jarke, M., & Wulf, V. (2007). Reality is our laboratory: Communities of practice in applied computer science. Behavior and Information Technology, 26(1), 81–94.CrossRef Rohde, M., Klamma, R., Jarke, M., & Wulf, V. (2007). Reality is our laboratory: Communities of practice in applied computer science. Behavior and Information Technology, 26(1), 81–94.CrossRef
Zurück zum Zitat Ross, L. (1977). The intuitive psychologist and his shortcomings: Distortions in the attribution process. In L. Berkowitz (Ed.), Advances in experimental social psychology (pp. 173–220). New York: Academic Press. Ross, L. (1977). The intuitive psychologist and his shortcomings: Distortions in the attribution process. In L. Berkowitz (Ed.), Advances in experimental social psychology (pp. 173–220). New York: Academic Press.
Zurück zum Zitat Tetlock, P. (1985). Accountability: A social check on the fundamental attribution error. Social Psychology Quarterly, 48(3), 227–236.CrossRef Tetlock, P. (1985). Accountability: A social check on the fundamental attribution error. Social Psychology Quarterly, 48(3), 227–236.CrossRef
Zurück zum Zitat Thorndike, E. L. (1920). A constant error on psychological rating. Journal of Applied Psychology, 4(1), 25–29.CrossRef Thorndike, E. L. (1920). A constant error on psychological rating. Journal of Applied Psychology, 4(1), 25–29.CrossRef
Zurück zum Zitat Weinberger, A., & Fischer, F. (2005). A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers and Education, 46(1), 71–95.CrossRef Weinberger, A., & Fischer, F. (2005). A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers and Education, 46(1), 71–95.CrossRef
Zurück zum Zitat Wong, J., Oh, L. M., Ou, J., Rosé, C. P., Yang, J., & Fussell, S.R. (2007). Sharing a single expert among multiple partners. In Proceedings of the SIGCHI conference on human factors in computing systems, pp. 261–270. New York: ACM Press. Wong, J., Oh, L. M., Ou, J., Rosé, C. P., Yang, J., & Fussell, S.R. (2007). Sharing a single expert among multiple partners. In Proceedings of the SIGCHI conference on human factors in computing systems, pp. 261–270. New York: ACM Press.
Metadaten
Titel
Towards effective group work assessment: even what you don’t see can bias you
verfasst von
Gahgene Gweon
Soojin Jun
Susan Finger
Carolyn Penstein Rosé
Publikationsdatum
23.09.2015
Verlag
Springer Netherlands
Erschienen in
International Journal of Technology and Design Education / Ausgabe 1/2017
Print ISSN: 0957-7572
Elektronische ISSN: 1573-1804
DOI
https://doi.org/10.1007/s10798-015-9332-1

Weitere Artikel der Ausgabe 1/2017

International Journal of Technology and Design Education 1/2017 Zur Ausgabe

    Premium Partner