skip to main content
research-article
Public Access

Understanding Women's Remote Collaborative Programming Experiences: The Relationship between Dialogue Features and Reported Perceptions

Published:05 January 2021Publication History
Skip Abstract Section

Abstract

In recent years, remote collaboration has become increasingly common both in the workplace and in the classroom. It is imperative that we understand and support remote collaborative problem solving, particularly understanding the experiences of people from historically marginalized groups whose intellectual contributions are essential for addressing the pressing needs society faces. This paper reports on a study in which 58 introductory computer science students constructed code remotely with a partner following either predefined structured roles (driver and navigator in pair programming) or without predefined structured roles. Between the structured-role and unstructured-role conditions, participants? normalized learning gain, Intrinsic Motivation Inventory scores, and system usability scores were not significantly different. However, regardless of the collaboration condition, women reported significantly higher levels of stress, lower levels of perceived competence, and less perceived choice compared to men. Because computer science is a context in which women have been historically marginalized, we next examined the relationship between student gender and collaborative dialogues by extracting lexical and sentiment features from the textual messages partners exchanged. Results reveal that dialogue features, such as number of utterances, utterance length, and partner sentiment, significantly correlated with women's reports of stress, perceived competence, or perceived choice. These findings provide insight on women's experiences in remote programming, suggest that dialogue features can predict their collaborative experiences, and hold implications for designing systems that help provide collaborative experiences in which everyone can thrive.

References

  1. Marjolijn L. Antheunis, Patti M. Valkenburg, and Jochen Peter. 2012. The Quality of Online, Offline, and Mixed-Mode Friendships Among Users of a Social Networking Site. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, Vol. 6, 3 (2012).Google ScholarGoogle Scholar
  2. National Education Association. 2012. Preparing 21st Century Students for a Global Society: An Educator's Guide to the 'Four Cs'. Alexandria, VA: National Education Association (2012).Google ScholarGoogle Scholar
  3. Prashant Baheti, Edward Gehringer, and David Stotts. 2002. Exploring the Efficacy of Distributed Pair Programming. In Conference on Extreme Programming and Agile Methods. 208--220.Google ScholarGoogle Scholar
  4. Kristy Elizabeth Boyer, Robert Phillips, Michael D. Wallis, Mladen A. Vouk, and James C. Lester. 2008. Balancing Cognitive and Motivational Scaffolding in Tutorial Dialogue. In Proceedings of the International Conference on Intelligent Tutoring Systems (ITS). 239--249.Google ScholarGoogle Scholar
  5. Danielle Bragg, Kyle Rector, and Richard E. Ladner. 2015. A User-Powered American Sign Language Dictionary. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW). 1837--1848.Google ScholarGoogle Scholar
  6. John Brooke. 1996. SUS - A quick and dirty usability scale. In Usability Evaluation in Industry. 4--7.Google ScholarGoogle Scholar
  7. Andrzej Buda and Andrzej Jarynowski. 2010. Life Time of Correlations and Its Applications. Wydawnictwo Nie. zalezne.Google ScholarGoogle Scholar
  8. Mehmet Celepkolu, Joseph B. Wiggins, Kristy Elizabeth Boyer, and Kyla McMullen. 2017. Think First: Fostering Substantive Contributions in Collaborative Problem-Solving Dialogues. In Proceedings of the 12th International Conference on Computer Supported Collaborative Learning (CSCL). 295--302.Google ScholarGoogle Scholar
  9. Jan Chong and Rosanne Siino. 2006. Interruptions on Software Teams: A Comparison of Paired and Solo Programmers. In Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work (CSCW). 29--38.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Jacob Cohen. 2013. Statistical Power Analysis for the Behavioral Sciences. Academic Press.Google ScholarGoogle Scholar
  11. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated Hate Speech Detection and the Problem of Offensive Language. In Proceedings of the 11th International AAAI Conference on Web and Social Media. 512--515.Google ScholarGoogle ScholarCross RefCross Ref
  12. Mona Emara, Ramkumar Rajendran, Gautam Biswas, Mahmod Okasha, and Adel Alsaeid Elbanna. 2018. Do Students' Learning Behaviors Differ when they Collaborate in Open-Ended Learning Environments? Proceedings of the ACM on Human-Computer Interaction, Vol. 2, CSCW (2018), 1--19.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Stephen M. Fiore, Art Graesser, Samuel Greiff, Patrick Griffin, Brian Gong, Patrick Kyllonen, Christine Massey, Harry O'Neil, Jim Pellegrino, Robert Rothman, Helen Soulé, and Alina von Davier. 2017. Collaborative Problem Solving: Considerations for the National Assessment of Educational Progress. (2017).Google ScholarGoogle Scholar
  14. Alastair J. Gill, Robert M. French, Darren Gergle, and Jon Oberlander. 2008. The Language of Emotion in Short Blog Texts. In Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work (CSCW). 299--302.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Jeffrey T. Hancock, Christopher Landrigan, and Courtney Silver. 2007. Expressing Emotion in Text-Based Communication. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). 929--932.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Brian Hanks. 2008. Empirical Evaluation of Distributed Pair Programming. International Journal of Human-Computer Studies, Vol. 66, 7 (2008), 530--544.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Emily M. Hastings, Farnaz Jahanbakhsh, Karrie Karahalios, Darko Marinov, and Brian P. Bailey. 2018. Structure or Nurture? The Effects of Team-Building Activities and Team Composition on Team Outcomes. Proceedings of the ACM on Human-Computer Interaction (CHI), Vol. 2, CSCW (2018).Google ScholarGoogle Scholar
  18. Clayton J. Hutto and Eric Gilbert. 2014. VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text. In Proceedings of the Eighth International AAAI Conference on Weblogs and Social Media. 216--225.Google ScholarGoogle Scholar
  19. Henry Jenkins. 2009. Confronting the Challenges of Participatory Culture: Media Education for the 21st Century.Google ScholarGoogle Scholar
  20. Patrick Jermann and Marc-Antoine Nüssli. 2012. Effects of Sharing Text Selections on Gaze Cross-Recurrence and Interaction Quality in a Pair Programming Task. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (CSCW). 1125--1134.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. L. Crystal Jiang, Natalie N. Bazarova, and Jeffrey T. Hancock. 2011. The Disclosure--Intimacy Link in Computer-Mediated Communication: An Attributional Extension of the Hyperpersonal Model. Human Communication Research, Vol. 37, 1 (2011), 58--77.Google ScholarGoogle ScholarCross RefCross Ref
  22. Malte Jung, Jan Chong, and Larry Leifer. 2012. Group Hedonic Balance and Pair Programming Performance: Affective Interaction Dynamics as Indicators of Performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). 829--838.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Beaumie Kim. 2001. Social Constructivism. In Emerging Perspectives on Learning, Teaching, and Technology, Michael Orey (Ed.). 55--61.Google ScholarGoogle Scholar
  24. Young Bin Kim, Jun Gi Kim, Wook Kim, Jae Ho Im, Tae Hyeong Kim, Shin Jin Kang, and Chang Hun Kim. 2016. Predicting Fluctuations in Cryptocurrency Transactions Based on User Comments and Replies. PLOS ONE, Vol. 11, 8 (2016).Google ScholarGoogle Scholar
  25. Morgan Klaus Scheuerman, Katta Spiel, Oliver L. Haimson, Foad Hamidi, and Stacy M. Branham. 2019. HCI Guidelines for Gender Equity and Inclusivity. Retrieved May 13, 2020 from https://www.morgan-klaus.com/sigchi-gender-guidelines.Google ScholarGoogle Scholar
  26. Johannes Konert, Henrik Bellh"auser, René Röpke, Eduard Gallwas, and Ahmed Zucik. 2016. MoodlePeers: Factors Relevant in Learning Group Formation for Improved Learning Outcomes, Satisfaction and Commitment in E-learning Scenarios using GroupAL. In Adaptive and Adaptable Learning: Proceedings of the 11th European Conference on Technology-Enhanced Learning (EC-TEL). 390--396.Google ScholarGoogle ScholarCross RefCross Ref
  27. Brian Kooiman, Wenling Li, Michael Wesolek, and Heeja Kim. 2015. Validation of the Relatedness Scale of the Intrinsic Motivation Inventory through Factor Analysis. International Journal of Multidisciplinary Research and Modern Education, Vol. 1, 2 (2015), 302--311.Google ScholarGoogle Scholar
  28. Theodora Koulouri, Stanislao Lauria, and Robert D. Macredie. 2017. The Influence of Visual Feedback and Gender Dynamics on Performance, Perception and Communication Strategies in CSCW. International Journal of Human-Computer Studies, Vol. 97 (2017), 162--181.Google ScholarGoogle ScholarCross RefCross Ref
  29. Amit Kramer, Devasheesh P. Bhave, and Tiffany D. Johnson. 2014. Personality and Group Performance: The Importance of Personality Composition and Work Tasks. Personality and Individual Differences, Vol. 58 (2014), 132--137.Google ScholarGoogle ScholarCross RefCross Ref
  30. Adam D. I. Kramer, Susan R. Fussell, and Leslie D. Setlock. 2004. Text Analysis as a Tool for Analyzing Conversation in Online Support Groups. In CHI '04 Extended Abstracts on Human Factors in Computing Systems (CHI EA). 1485--1488.Google ScholarGoogle Scholar
  31. Sandeep Kaur Kuttal, Kevin Gerstner, and Alexandra Bejarano. 2019. Remote Pair Programming in Online CS Education: Investigating through a Gender Lens. In 2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 75--85.Google ScholarGoogle ScholarCross RefCross Ref
  32. Marjan Laal and Seyed Mohammad Ghodsi. 2012. Benefits of Collaborative Learning. Procedia-Social and Behavioral Sciences, Vol. 31 (2012), 486--490.Google ScholarGoogle ScholarCross RefCross Ref
  33. Cliff Lampe, Rebecca Gray, Andrew T. Fiore, and Nicole Ellison. 2014. Help is on the Way: Patterns of Responses to Resource Requests on Facebook. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW). 3--15.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Colleen M. Lewis and Niral Shah. 2015. How Equity and Inequity Can Emerge in Pair Programming. In Proceedings of the 11th Annual Conference on International Computing Education Research (ICER). 41--50.Google ScholarGoogle Scholar
  35. James R. Lewis and Jeff Sauro. 2018. Item Benchmarks for the System Usability Scale. Journal of Usability Studies, Vol. 13, 3 (2018), 158--167.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Xing Liu and Hari Koirala. 2012. Ordinal Regression Analysis: Using Generalized Ordinal Logistic Regression Models to Estimate Educational Data. Journal of Modern Applied Statistical Methods, Vol. 11, 1 (2012), 242--254.Google ScholarGoogle ScholarCross RefCross Ref
  37. Michael Madaio, Kun Peng, Amy Ogan, and Justine Cassell. 2018. A Climate of Support: A Process-Oriented Analysis of the Impact of Rapport on Peer Tutoring. In Proceedings of the 13th International Conference of the Learning Sciences (ICLS). 600--607.Google ScholarGoogle Scholar
  38. Tara Matthews, Jalal U. Mahmud, Jilin Chen, Michael Muller, Eben Haber, and Hernan Badenes. 2015. They Said What?: Exploring the Relationship Between Language Use and Member Satisfaction in Communities. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW). 819--825.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Hanna Maurin, Diane H. Sonnenwald, Bruce Cairns, James E. Manning, Eugene B. Freid, and Henry Fuchs. 2006. Exploring Gender Differences in Perceptions of 3D Telepresence Collaboration Technology: An Example from Emergency Medical Care. In Proceedings of the 4th Nordic Conference on Human-Computer Interaction: Changing Roles (NordiCHI). 381--384.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Edward McAuley, Terry Duncan, and Vance V. Tammen. 1989. Psychometric Properties of the Intrinsic Motivation Inventory in a Competitive Sport Setting: A Confirmatory Factor Analysis. Research Quarterly for Exercise and Sport, Vol. 60, 1 (1989), 48--58.Google ScholarGoogle ScholarCross RefCross Ref
  41. Peter McCullagh. 1980. Regression Models for Ordinal Data. Journal of the Royal Statistical Society: Series B (Methodological), Vol. 42, 2 (1980), 109--127.Google ScholarGoogle ScholarCross RefCross Ref
  42. Charlie McDowell, Linda Werner, Heather Bullock, and Julian Fernald. 2002. The Effects of Pair-Programming on Performance in an Introductory Programming Course. In Proceedings of the 33rd ACM Technical Symposium on Computer Science Education (SIGCSE). 38--42.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Meike Osinski and Nikol Rummel. 2019. Towards Successful Knowledge Integration in Online Collaboration: An Experiment on the Role of Meta-Knowledge. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019).Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Jennifer Robison, Scott McQuiggan, and James C. Lester. 2009. Evaluating the Consequences of Affective Feedback in Intelligent Tutoring Systems. In 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. 1--6.Google ScholarGoogle Scholar
  45. Fernando J. Rodríguez, Kimberly Michelle Price, and Kristy Elizabeth Boyer. 2017. Expressing and Addressing Uncertainty: A Study of Collaborative Problem-Solving Dialogues. In Proceedings of the 12th International Conference on Computer Supported Collaborative Learning (CSCL). 207--214.Google ScholarGoogle Scholar
  46. Jeff Sauro and James R. Lewis. 2016. Quantifying the User Experience: Practical Statistics for User Research.Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Linda J. Sax, Kathleen J. Lehman, Jerry A. Jacobs, M. Allison Kanny, Gloria Lim, Laura Monje-Paulson, and Hilary B. Zimmerman. 2017. Anatomy of an Enduring Gender Gap: The Evolution of Women?s Participation in Computer Science. The Journal of Higher Education, Vol. 88, 2 (2017), 258--293.Google ScholarGoogle ScholarCross RefCross Ref
  48. Toni Schmader. 2012. Stereotype Threat: Theory, Process, and Application. Oxford University Press.Google ScholarGoogle Scholar
  49. Andrew J. Scholand, Yla R. Tausczik, and James W. Pennebaker. 2010. Social Language Network Analysis. In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work (CSCW). 23--26.Google ScholarGoogle Scholar
  50. Joseph Seering, Felicia Ng, Zheng Yao, and Geoff Kaufman. 2018. Applications of Social Identity Theory to Research and Design in Computer-Supported Cooperative Work. Proceedings of the ACM on Human-Computer Interaction, Vol. 2, CSCW (2018).Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Lauren E. Sherman, Minas Michikyan, and Patricia M. Greenfield. 2013. The Effects of Text, Audio, Video, and In-Person Communication on Bonding Between Friends. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, Vol. 7, 2 (2013).Google ScholarGoogle Scholar
  52. Angela E.B. Stewart, Hana Vrzakova, Chen Sun, Jade Yonehiro, Cathlyn Adele Stone, Nicholas D. Duran, Valerie Shute, and Sidney K. D'Mello. 2019. I Say, You Say, We Say: Using Spoken Language to Model Socio-Cognitive Processes during Computer-Supported Collaborative Problem Solving. Proceedings of the ACM on Human-Computer Interaction (CHI), Vol. 3, CSCW (2019), 1--19.Google ScholarGoogle Scholar
  53. Henri Tajfel. 1974. Social Identity and Intergroup Behaviour. Information (International Social Science Council), Vol. 13, 2 (1974), 65--93.Google ScholarGoogle ScholarCross RefCross Ref
  54. Chiew Seng Sean Tan, Johannes Schöning, Kris Luyten, and Karin Coninx. 2014. Investigating the Effects of Using Biofeedback as Visual Stress Indicator during Video-Mediated Collaboration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). 71--80.Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Yla R. Tausczik and James W. Pennebaker. 2013. Improving Teamwork Using Real-Time Language Feedback. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). 459--468.Google ScholarGoogle Scholar
  56. Divy Thakkar, Nithya Sambasivan, Purva Kulkarni, Pratap Kalenahalli Sudarshan, and Kentaro Toyama. 2018. The Unexpected Entry and Exodus of Women in Computing and HCI in India. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI). 1--12.Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. David Thissen, Lynne Steinberg, and Daniel Kuang. 2002. Quick and Easy Implementation of the Benjamini-Hochberg Procedure for Controlling the False Positive Rate in Multiple Comparisons. Journal of Educational and Behavioral Statistics, Vol. 27, 1 (2002), 77--83.Google ScholarGoogle ScholarCross RefCross Ref
  58. Patti M. Valkenburg and Jochen Peter. 2009. Social Consequences of the Internet for adolescents: A Decade of Research. Current Directions in Psychological Science, Vol. 18, 1 (2009), 1--5.Google ScholarGoogle ScholarCross RefCross Ref
  59. Jan M. van Bruggen, Paul A. Kirschner, and Wim Jochems. 2002. External Representation of Argumentation in CSCL and the Management of Cognitive Load. Learning and Instruction, Vol. 12, 1 (2002), 121--138.Google ScholarGoogle ScholarCross RefCross Ref
  60. April Yi Wang, Anant Mittal, Christopher Brooks, and Steve Oney. 2019. How Data Scientists Use Computational Notebooks for Real-Time Collaboration. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019), 1--30.Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Armin Weinberger, Karsten Stegmann, and Frank Fischer. 2007. Knowledge Convergence in Collaborative Learning: Concepts and Assessment. Learning and Instruction, Vol. 17, 4 (2007), 416--426.Google ScholarGoogle ScholarCross RefCross Ref
  62. Linda L. Werner, Brian Hanks, and Charlie McDowell. 2004. Pair-Programming Helps Female Computer Science Students. ACM Journal of Educational Resources in Computing, Vol. 4, 1 (2004), 1--8.Google ScholarGoogle Scholar
  63. Laurie Williams and Richard L. Upchurch. 2001. In Support of Student Pair-Programming. In Proceedings of the 32nd ACM Technical Symposium on Computer Science Education (SIGCSE). 327--331.Google ScholarGoogle Scholar
  64. Svante Wold, Kim Esbensen, and Paul Geladi. 1987. Principal Component Analysis. Chemometrics and Intelligent Laboratory Systems, Vol. 2, 1--3 (1987), 37--52.Google ScholarGoogle ScholarCross RefCross Ref
  65. Kimberly Michelle Ying, Lydia G. Pezzullo, Mohona Ahmed, Kassandra Crompton, Jeremiah Blanchard, and Kristy Elizabeth Boyer. 2019. In Their Own Words: Gender Differences in Student Perceptions of Pair Programming. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE). 1053--1059.Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Jaebong Yoo and Jihie Kim. 2014. Can Online Discussion Participation Predict Group Project Performance? Investigating the Roles of Linguistic Features and Participation Patterns. International Journal of Artificial Intelligence in Education, Vol. 24, 1 (2014), 8--32.Google ScholarGoogle ScholarCross RefCross Ref
  67. Chien Wen (Tina) Yuan, Yu-Hsuan Liu, Hao-Chuan Wang, and Yuan-Chi Tseng. 2019. Gender Effects on Collaborative Online Brainstorming Teamwork. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA). 1--6.Google ScholarGoogle Scholar
  68. Mark Zarb, Janet Hughes, and John Richards. 2013. Industry-Inspired Guidelines Improve Students' Pair Programming Communication. In Proceedings of the 18th ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE). 135--140.Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Kristina M. Zosuls, Cindy Faith Miller, Diane N. Ruble, Carol Lynn Martin, and Richard A. Fabes. 2011. Gender Development Research in Sex Roles: Historical Trends and Future Directions. Sex Roles, Vol. 64, 11--12 (2011), 826--842.Google ScholarGoogle ScholarCross RefCross Ref
  70. Stuart Zweben and Betsy Bizot. 2019. 2018 Taulbee Survey: Undergrad Enrollment Continues Upward; Doctoral Degree Production Declines but Doctoral Enrollment Rises. Computing Research News, Vol. 31, 5 (2019), 3--74.Google ScholarGoogle Scholar
  71. Stuart Zweben, Jodi Tims, and Yan Timanovsky. 2019. ACM-NDC Study 2018--2019: Seventh Annual Study of Non-Doctoral-Granting Departments in Computing. ACM Inroads, Vol. 10, 3 (2019), 40--54.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Understanding Women's Remote Collaborative Programming Experiences: The Relationship between Dialogue Features and Reported Perceptions

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image Proceedings of the ACM on Human-Computer Interaction
        Proceedings of the ACM on Human-Computer Interaction  Volume 4, Issue CSCW3
        CSCW
        December 2020
        1825 pages
        EISSN:2573-0142
        DOI:10.1145/3446568
        Issue’s Table of Contents

        Copyright © 2021 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 5 January 2021
        Published in pacmhci Volume 4, Issue CSCW3

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader