nach oben

Information Systems Frontiers

Erschienen in:

29.11.2021

Paradoxical Tensions Related to AI-Powered Evaluation Systems in Competitive Sports

verfasst von: Elena Mazurova, Willem Standaert, Esko Penttinen, Felix Ter Chian Tan

Erschienen in: Information Systems Frontiers | Ausgabe 3/2022

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Judging in competitive sports is prone to errors arising from the inherent limitations to humans’ cognitive and sensorial capabilities and from various potential sources of bias that influence judges. Artistic gymnastics offers a case in point: given the complexity of scoring and the ever-increasing speed of athletes’ performance, systems powered by artificial intelligence (AI) seem to promise benefits for the judging process and its outcomes. To characterize today’s human judging process for artistic gymnastics and examine contrasts against an AI-powered system currently being introduced in this context, an in-depth case study analyzed interview data from various stakeholder groups (judges, gymnasts, coaches, federations, technology providers, and fans). This exploratory study unearthed several paradoxical tensions accompanying AI-based evaluations in this setting. The paper identifies and illustrates tensions of this nature related to AI-powered systems’ accuracy, objectivity, explainability, relationship with artistry, interaction with humans, and consistency.

Vorheriger Artikel Designing Transparency for Effective Human-AI Collaboration

Nächster Artikel Designing Personality-Adaptive Conversational Agents for Mental Health Care

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nur mit Berechtigung zugänglich

An example is provided by the Reuters piece “Judging scandal still haunts Korean Yang,” available at https://www.reuters.com/article/us-olympics-rio-gymnastics-yang/judging-scandal-still-haunts-korean-yang-idUSKCN10205W.

The introduction of AI in soccer is particularly high-profile; see https://www.espn.com.au/football/fifa-world-cup/story/4237186/fifas-semi-automated-offside-var-tech-to-go-into-development-in-2021.

These two pieces characterize media coverage: P. Logothetis (2017, November 4). “Cracking the vault: Artificial intelligence judging comes to gymnastics.” The Guardian. Retrieved from https://www.theguardian.com/sport/blog/2017/nov/04/ai-judges-gymnastics-olympics; L. Radnofsky (2019, August 23). “The robots are coming (to judge gymnastics).” The Wall Street Journal, at https://www.wsj.com/articles/the-robots-are-coming-to-judge-gymnastics-11566471601.

The CoP rulebook, covering these point values, can be downloaded from http://www.fig-gymnastics.com/publicdir/rules/files/en_WAG%20CoP%202017-2020.pdf.

Details are provided in Fujitsu’s announcement at https://www.fujitsu.com/global/about/resources/news/press-releases/2018/1120-01.html.

For the FIG announcement, see https://www.gymnastics.sport/site/pages/judges-support.php.

These materials are available at https://blog.global.fujitsu.com/fgb/wp-content/uploads/sites/3/2021/01/Zz03NjljMGY5YTY5ODkxMGM1ZGRjNWI0ZTIwNjkyMTE4Nw.jpg & https://blog.global.fujitsu.com/fgb/wp-content/uploads/sites/3/2021/01/Zz1kM2QyZDE1M2E0OTYwMmVlNTc3MjViYTU0MzE3ZjZjZA.jpg.

Information about the system can be found at https://medium.com/syncedreview/meet-fujitsus-ai-gymnastics-judges-8cb52613b2a.

M. Sarazen highlighted this aspect in the Synced piece “Meet Fujitsu’s AI gymnastics judges,” published on January 26, 2019; see https://syncedreview.com/2019/01/26/meet-fujitsus-ai-gymnastics-judges/.

This statement, made by a Fujitsu official, was quoted by L. Clarck in the October 9, 2019, Washington Post article “Computer-assisted judging is being tested at gymnastics world championships,” available at https://www.washingtonpost.com/sports/olympics/computer-assisted-judging-is-being-tested-at-gymnastics-world-championships/2019/10/09/214a515e-eab9-11e9-9306-47cb0324fd44_story.html (accessed on October 10, 2019).

Asatiani, A., Malo, P., Nagbol, P., Penttinen, E., Rinta-Kahila, T., & Salovaara, A. (2021). Sociotechnical envelopment of artificial intelligence: An approach to organizational deployment of inscrutable artificial intelligence systems. Journal of the Association for Information Systems, 22(2), 325–352.CrossRef

Aysolmaz, B., Iren, D., & Dau, N. (2020). Preventing algorithmic Bias in the development of algorithmic decision-making systems: A Delphi study. Proceedings of the 53rd Hawaii international conference on system sciences, 3, 5267–5276. https://doi.org/10.24251/hicss.2020.648.

Baskerville, R. L., Myers, M. D., & Yoo, Y. (2020). Digital first: The ontological reversal and new challenges for information systems research. MIS Ouarterly, 44(2), 509–523.

Benbya, H., Pachidi, S., Davenport, T., & Jarvenpaa, S. (2019). Call for papers JAIS-MISQE joint SI on artificial intelligence in organizations – JAIS-MISQE SI on artificial intelligence.

Benbya, H., Pachidi, S., & Jarvenpaa, S. (2021). Special issue editorial: Artificial intelligence in organizations: Implications for information systems research. Journal of the Association for Information Systems, 22(2), 281–303.CrossRef

Brynjolfsson, E., & Mcafee, A. (2017). The business of artificial intelligence. What it can — And cannot — Do for your organization. Harvard Business Review, 4–11.

Calabretta, G., Gemser, G., & Wijnberg, N. M. (2017). The interplay between intuition and rationality in strategic decision making: A paradox perspective. Organization Studies, 38(3–4), 365–401. https://doi.org/10.1177/0170840616655483CrossRef

Can, H., Lu, M., & Gan, L. (2011). The research on application of information technology in sports stadiums. Physics Procedia, 604–609. https://doi.org/10.1016/j.phpro.2011.11.093

Cortsen, K., & Rascher, D. A. (2018). The application of sports technology and sports data for commercial purposes. In: The use of technology in sports - Emerging challenges. https://doi.org/10.1016/j.colsurfa.2011.12.014.

Dellermann, D., Ebel, P., Söllner, M., & Leimeister, J. M. (2019). Hybrid Intelligence. Business and Information Systems Engineering. https://doi.org/10.1007/s12599-019-00595-2

Du, W. (Derek), Pan, S. L., Leidner, D. E., & Ying, W. (2019). Affordances, experimentation and actualization of FinTech: A blockchain implementation study. Journal of Strategic Information Systems, 28(1), 50–65. https://doi.org/10.1016/j.jsis.2018.10.002.

Dubé, L., & Robey, D. (2009). Surviving the paradoxes of virtual teamwork. Information Systems Journal, 19(1), 3–30. https://doi.org/10.1111/j.1365-2575.2008.00313.xCrossRef

Dumoulin. (2020). Accuracy and National Bias of figure skating judges: The good, the bad and the ugly. MIT Sloan Sports Analytics Conference.

Duong, A. (2008). Bias in the 2008 Beijing Olympics (gymnastics).

Faraj, S., Pachidi, S., & Sayegh, K. (2018). Working and organizing in the age of the learning algorithm. Information and Organization, 28(1), 62–70. https://doi.org/10.1016/j.infoandorg.2018.02.005CrossRef

Ferger, K., & Hackbarth, M. (2017). New way of determining horizontal displacement in competitive trampolining. Science of Gymnastics Journal, 9(3), 303–310.

Flessas, K., Mylonas, D., Panagiotaropoulou, G., Tsopani, D., Korda, A., Siettos, C., … Smyrnis, N. (2014). Judging the judges’ performance in rhythmic gymnastics. Medicine and Science in Sports and Exercise, 47(3), 640–648. https://doi.org/10.1249/MSS.0000000000000425.

Fujiwara, H., & Ito, K. (2018). ICT-based judging support system for artistic gymnastics and intended new world created through 3D sensing technology. Fujitsu Scientific & Technical Journal, 54(4), 66–72.

Gioia, D. A., Corley, K. G., & Hamilton, A. L. (2012). Seeking qualitative rigor in inductive research: Notes on the Gioia methodology. Organizational Research Methods, 16(1), 15–31. https://doi.org/10.1177/1094428112452151CrossRef

Goebeler, L., Standaert, W., & Xiao, X. (2021). Hybrid sport configurations: The intertwining of the physical and the digital. Proceedings of the 54th Hawaii international conference on system sciences.

Harding, J., & James, D. (2010). Performance assessment innovations for elite snowboarding. Procedia Engineering, 2(2), 2919–2924. https://doi.org/10.1016/j.proeng.2010.04.088CrossRef

Harding, J., Toohey, K., Martin, D., Hahn, A., & James, D. (2008). Technology and half-pipe snowboard competition — Insight from elite- level judges. 7th ISEA conference. https://doi.org/10.1007/978-2-287-99056-4.

Heiniger, S., & Mercier, H. (2018). National Bias of international gymnastics judges during the 2013-2016 Olympic cycle. arXiv preprint arXiv:1807.10033.

Jarvenpaa, S. L., & Lang, K. R. (2005). Managing the paradoxes of Mobile technology. Information Systems Management, 22(4), 7–23. https://doi.org/10.1201/1078.10580530/45520.22.4.20050901/90026.2CrossRef

Kallinikos, J., Aaltonen, A., & Marton, A. (2013). The ambivalent ontology of digital artifacts. MIS Quarterly, 37(2), 357–370.CrossRef

Keller, R., Stohr, A., Fridgen, G., Lockl, J., & Rieger, A. (2019). Affordance-experimentation-actualization theory in artificial intelligence research - a predictive maintenance story. 40th international conference on information systems, 1–17.

Kellogg, K. C., Valentine, M. A., & Christin, A. (2020). Algorithms at work: The new contested terrain of control. Academy of Management Annals, 14(1), 366–410. https://doi.org/10.5465/annals.2018.0174CrossRef

Kerr, R. (2018). Technologies for judging, umpiring and refereeing. In: Sport and technology (pp. 114–134). https://doi.org/10.7765/9781784995201.00010.

Klein, H., & Myers, M. (1999). A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Quarterly, 23(1), 67–93.CrossRef

Kolbinger, O., & Lames, M. (2017). Scientific approaches to technological officiating aids in game sports. Current Issues in Sport Science (CISS), 2, 1–10. https://doi.org/10.15203/ciss_2017.001CrossRef

Konovsky, M. a. (2000). Understanding procedural justice and its impact on business organizations. Journal of Management, 26(3), 489–511. https://doi.org/10.1177/014920630002600306CrossRef

Lee, A. S., & Baskerville, R. L. (2003). Generalizing generalizability in information systems research. Information Systems Research, 14(3). https://doi.org/10.1287/isre.14.3.221.16560

Leveaux, R. (2010). Facilitating Referee’s decision making in sport via the application of technology. Communications of the IBIMA. https://doi.org/10.5171/2010.545333

Leveaux, R. (2012). 2012 Olympic games decision making Technologies for Taekwondo Competition. Communications of the IBIMA. https://doi.org/10.5171/2012.834755

Lewis, M. W. (2000). Exploring paradox: Toward a more comprehensive guide. Academy of Management Review, 25(4), 760–776. https://doi.org/10.1080/15358593.2015.1116592CrossRef

Majchrzak, A., Faraj, S., Kane, G. C., & Azad, B. (2013). The contradictory influence of social media affordances on online communal knowledge sharing. Journal of Computer-Mediated Communication, 19(1), 38–55. https://doi.org/10.1111/jcc4.12030CrossRef

Mazurova, E., Penttinen, E., & Salovaara, A. (2021). Stakeholder-dependent views on biases of human- and machine-based judging systems. Hawaii International Conference on System Sciences.

Mercier, H., & Heiniger, S. (2018). Judging the judges: Evaluating the performance of international gymnastics judges. ArXiv, 1–12.

Meske, C., & Bunde, E. (2020). Transparency and trust in human-AI-interaction: The role of model-agnostic explanations in computer vision-based decision support. Proceedings of the international conference of AI in HCI 2020, 12217 LNCS, 54–69. https://doi.org/10.1007/978-3-030-50334-5_4.

Meske, C., Bunde, E., Schneider, J., & Gersch, M. (Forthcoming). Explainable Artificial Intelligence: Objectives, Stakeholders, and Future Research Opportunities. Information Systems Management, 1–11. https://doi.org/10.1080/10580530.2020.1849465

Nadeem, A., Abedin, B., & Marjanovic, O. (2020). Gender Bias in AI: A review of contributing factors and mitigating strategies. ACIS 2020 proceedings.

Omorczyk, J., Nosiadek, L., Ambroży, T., & Nosiadek, A. (2015). High-frequency video capture and a computer program with frame-by-frame angle determination functionality as tools that support judging in artistic gymnastics. Acta of Bioengineering and Biomechanics, 17(3), 85–93.

Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Sage Publications. https://doi.org/10.1177/1035719X0300300213CrossRef

Perederij, V. V. (2013). The problem of the quality of judging in rhythmic gymnastics. Pedagogics, Psychology, Medical-Biological Problems of Physical Training and Sports, 3, 43–46. https://doi.org/10.6084/m9.figshare.647325CrossRef

Plessner, H., & Haar, T. (2006). Sports performance judgments from a social cognitive perspective. Psychology of Sport and Exercise, 7(6), 555–575. https://doi.org/10.1016/j.psychsport.2006.03.007CrossRef

Plessner, H., & Schallies, E. (2005). Judging the cross on rings: A matter of achieving shape constancy. Applied Cognitive Psychology, 19(9), 1145–1156. https://doi.org/10.1002/acp.1136CrossRef

Poole, M., & van de Ven, A. (1989). Using paradox to build management and organization theories. The Academy of Management Review, 14(4), 562–578.CrossRef

Rai, A. (2020). Explainable AI: From black box to glass box. Journal of the Academy of Marketing Science, 48(1), 137–141. https://doi.org/10.1007/s11747-019-00710-5CrossRef

Rai, A., Constantinides, P., & Sarker, S. (2019). Next-generation digital platforms: Toward human–AI hybrids. MIS Quarterly, 43(1), iii–ix.

Raisch, S., & Krakowski, S. (2021). Artificial intelligence and management: The automation–augmentation paradox. Academy of Management Review, 46(1), 192–210. https://doi.org/10.5465/AMR.2018.0072CrossRef

Robey, D., & Boudreau, M. (1999). Accounting for the contradictory organizational consequences of information technology: Theoretical directions and methodological implications. Information Systems Research, 10(June), 167–185. https://doi.org/10.1287/isre.10.2.167CrossRef

Rzepka, C., & Berger, B. (2018). User interaction with AI-enabled systems: A systematic review of IS research. International conference on information systems 2018, ICIS 2018, (December).

Salovaara, A., Lyytinen, K., & Penttinen, E. (2019). High reliability in digital organizing: Mindlessness, the frame problem, and digital operations. MIS Quarterly, 43(2), 555–578. https://doi.org/10.25300/MISQ/2019/14577CrossRef

Schad, J., Lewis, M. W., Raisch, S., & Smith, W. K. (2016). Paradox research in management science: Looking Back to move forward. Academy of Management Annals, 10(1), 5–64. https://doi.org/10.1080/19416520.2016.1162422CrossRef

Schön, D. (1983). The reflective practitioner: How professionals think in action. Basic Books.

Shrestha, Y. R., Ben-Menahem, S. M., & von Krogh, G. (2019). Organizational decision-making structures in the age of artificial intelligence. California Management Review, 61(4), 66–84. https://doi.org/10.1177/0008125619862257CrossRef

Siggelkow, N. (2007). Persuasion with case studies. Academy of Management Journal, 50(1), 20–24. https://doi.org/10.5465/AMJ.2007.24160882CrossRef

Smith, W. K., & Lewis, M. W. (2011). Toward a theory of paradox: A dynamic equilibrium model of organizing. Academy of Management Review, 36(2), 381–403. https://doi.org/10.5465/amr.2011.59330958CrossRef

Stefani, R. (1998). Predicting outcomes. In J. Bennett (Ed.), Statistics in sport (pp. 249–275). Hodder Education Publishers.

Tan, F. T. C., Tan, B., & Pan, S. L. (2016). Developing a leading digital multi-sided platform: Examining IT affordances and competitive actions in alibaba.com. Communications of the Association for Information Systems, 38(1), 738–760. https://doi.org/10.17705/1CAIS.03836CrossRef

Taymazov, V., Bakulev, S., Pavlenko, A., Simakov, A., & Chistyakov, V. (2013). To a question of electronic refereeing systems application in taekwondo (VTF). Uchenye Zapiski Universiteta Imeni P.F. Lesgafta, 155–160. https://doi.org/10.5930/issn.1994-4683.2013.04.98.p155-160

van den Broek, E., Sergeeva, A., & Huysman, M. (2019). Hiring algorithms: An ethnography of fairness in practice. 40th international conference on information systems, ICIS 2019.

van den Broek, E., Sergeeva, A., & Huysman, M. (2021). When the machine meets the expert: An ethnography of developing AI for hiring. MIS Quarterly, 45(3b), 1557–1580. https://doi.org/10.25300/MISQ/2021/16559

Walsham, G. (2006). Doing interpretive research. European Journal of Information Systems, 15(3), 320–330. https://doi.org/10.1057/palgrave.ejis.3000589CrossRef

Wimelius, H., Mathiassen, L., Holmström, J., & Keil, M. (2021). A paradoxical perspective on technology renewal in digital transformation. Information Systems Journal, 31(1), 198–225. https://doi.org/10.1111/isj.12307CrossRef

Xiao, X., Chian Tan, F. Ter, Lim, E. T. K., Henningsson, S., Vatrapu, R., Hedman, J., … Van Hillegersberg, J. (2017). Sports digitalization: An overview and a research agenda. 38th international conference on information systems, 1–21.

Xu, J., Benbasat, I., & Cenfetelli, R. T. (2014). The nature and consequences of trade-off transparency in the context of recommendation agents. MIS Quarterly, 38(2), 379–406.CrossRef

Yin, R. K. (2013). Case study research: Design and methods. Sage Publications.

Titel: Paradoxical Tensions Related to AI-Powered Evaluation Systems in Competitive Sports
verfasst von: Elena Mazurova
Willem Standaert
Esko Penttinen
Felix Ter Chian Tan
Publikationsdatum: 29.11.2021
Verlag: Springer US
Erschienen in: Information Systems Frontiers / Ausgabe 3/2022
Print ISSN: 1387-3326
Elektronische ISSN: 1572-9419
DOI: https://doi.org/10.1007/s10796-021-10215-8

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 3/2022

Understanding the Design Elements Affecting User Acceptance of Intelligent Agents: Past, Present and Future

Designing Personality-Adaptive Conversational Agents for Mental Health Care

Voice Assistant vs. Chatbot – Examining the Fit Between Conversational Agents’ Interaction Modalities and Information Search Tasks

Re-Examining the Impact of Multidimensional Trust on Patients’ Online Medical Consultation Service Continuance Decision

“Don’t Neglect the User!” – Identifying Types of Human-Chatbot Interactions and their Associated Characteristics

Assessing Consumers’ Co‐production and Future Participation On Value Co‐creation and Business Benefit: an F-P-C-B Model Perspective

Premium Partner