Abstract
In recent years, remote collaboration has become increasingly common both in the workplace and in the classroom. It is imperative that we understand and support remote collaborative problem solving, particularly understanding the experiences of people from historically marginalized groups whose intellectual contributions are essential for addressing the pressing needs society faces. This paper reports on a study in which 58 introductory computer science students constructed code remotely with a partner following either predefined structured roles (driver and navigator in pair programming) or without predefined structured roles. Between the structured-role and unstructured-role conditions, participants? normalized learning gain, Intrinsic Motivation Inventory scores, and system usability scores were not significantly different. However, regardless of the collaboration condition, women reported significantly higher levels of stress, lower levels of perceived competence, and less perceived choice compared to men. Because computer science is a context in which women have been historically marginalized, we next examined the relationship between student gender and collaborative dialogues by extracting lexical and sentiment features from the textual messages partners exchanged. Results reveal that dialogue features, such as number of utterances, utterance length, and partner sentiment, significantly correlated with women's reports of stress, perceived competence, or perceived choice. These findings provide insight on women's experiences in remote programming, suggest that dialogue features can predict their collaborative experiences, and hold implications for designing systems that help provide collaborative experiences in which everyone can thrive.
- Marjolijn L. Antheunis, Patti M. Valkenburg, and Jochen Peter. 2012. The Quality of Online, Offline, and Mixed-Mode Friendships Among Users of a Social Networking Site. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, Vol. 6, 3 (2012).Google Scholar
- National Education Association. 2012. Preparing 21st Century Students for a Global Society: An Educator's Guide to the 'Four Cs'. Alexandria, VA: National Education Association (2012).Google Scholar
- Prashant Baheti, Edward Gehringer, and David Stotts. 2002. Exploring the Efficacy of Distributed Pair Programming. In Conference on Extreme Programming and Agile Methods. 208--220.Google Scholar
- Kristy Elizabeth Boyer, Robert Phillips, Michael D. Wallis, Mladen A. Vouk, and James C. Lester. 2008. Balancing Cognitive and Motivational Scaffolding in Tutorial Dialogue. In Proceedings of the International Conference on Intelligent Tutoring Systems (ITS). 239--249.Google Scholar
- Danielle Bragg, Kyle Rector, and Richard E. Ladner. 2015. A User-Powered American Sign Language Dictionary. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW). 1837--1848.Google Scholar
- John Brooke. 1996. SUS - A quick and dirty usability scale. In Usability Evaluation in Industry. 4--7.Google Scholar
- Andrzej Buda and Andrzej Jarynowski. 2010. Life Time of Correlations and Its Applications. Wydawnictwo Nie. zalezne.Google Scholar
- Mehmet Celepkolu, Joseph B. Wiggins, Kristy Elizabeth Boyer, and Kyla McMullen. 2017. Think First: Fostering Substantive Contributions in Collaborative Problem-Solving Dialogues. In Proceedings of the 12th International Conference on Computer Supported Collaborative Learning (CSCL). 295--302.Google Scholar
- Jan Chong and Rosanne Siino. 2006. Interruptions on Software Teams: A Comparison of Paired and Solo Programmers. In Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work (CSCW). 29--38.Google ScholarDigital Library
- Jacob Cohen. 2013. Statistical Power Analysis for the Behavioral Sciences. Academic Press.Google Scholar
- Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated Hate Speech Detection and the Problem of Offensive Language. In Proceedings of the 11th International AAAI Conference on Web and Social Media. 512--515.Google ScholarCross Ref
- Mona Emara, Ramkumar Rajendran, Gautam Biswas, Mahmod Okasha, and Adel Alsaeid Elbanna. 2018. Do Students' Learning Behaviors Differ when they Collaborate in Open-Ended Learning Environments? Proceedings of the ACM on Human-Computer Interaction, Vol. 2, CSCW (2018), 1--19.Google ScholarDigital Library
- Stephen M. Fiore, Art Graesser, Samuel Greiff, Patrick Griffin, Brian Gong, Patrick Kyllonen, Christine Massey, Harry O'Neil, Jim Pellegrino, Robert Rothman, Helen Soulé, and Alina von Davier. 2017. Collaborative Problem Solving: Considerations for the National Assessment of Educational Progress. (2017).Google Scholar
- Alastair J. Gill, Robert M. French, Darren Gergle, and Jon Oberlander. 2008. The Language of Emotion in Short Blog Texts. In Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work (CSCW). 299--302.Google ScholarDigital Library
- Jeffrey T. Hancock, Christopher Landrigan, and Courtney Silver. 2007. Expressing Emotion in Text-Based Communication. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). 929--932.Google ScholarDigital Library
- Brian Hanks. 2008. Empirical Evaluation of Distributed Pair Programming. International Journal of Human-Computer Studies, Vol. 66, 7 (2008), 530--544.Google ScholarDigital Library
- Emily M. Hastings, Farnaz Jahanbakhsh, Karrie Karahalios, Darko Marinov, and Brian P. Bailey. 2018. Structure or Nurture? The Effects of Team-Building Activities and Team Composition on Team Outcomes. Proceedings of the ACM on Human-Computer Interaction (CHI), Vol. 2, CSCW (2018).Google Scholar
- Clayton J. Hutto and Eric Gilbert. 2014. VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text. In Proceedings of the Eighth International AAAI Conference on Weblogs and Social Media. 216--225.Google Scholar
- Henry Jenkins. 2009. Confronting the Challenges of Participatory Culture: Media Education for the 21st Century.Google Scholar
- Patrick Jermann and Marc-Antoine Nüssli. 2012. Effects of Sharing Text Selections on Gaze Cross-Recurrence and Interaction Quality in a Pair Programming Task. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (CSCW). 1125--1134.Google ScholarDigital Library
- L. Crystal Jiang, Natalie N. Bazarova, and Jeffrey T. Hancock. 2011. The Disclosure--Intimacy Link in Computer-Mediated Communication: An Attributional Extension of the Hyperpersonal Model. Human Communication Research, Vol. 37, 1 (2011), 58--77.Google ScholarCross Ref
- Malte Jung, Jan Chong, and Larry Leifer. 2012. Group Hedonic Balance and Pair Programming Performance: Affective Interaction Dynamics as Indicators of Performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). 829--838.Google ScholarDigital Library
- Beaumie Kim. 2001. Social Constructivism. In Emerging Perspectives on Learning, Teaching, and Technology, Michael Orey (Ed.). 55--61.Google Scholar
- Young Bin Kim, Jun Gi Kim, Wook Kim, Jae Ho Im, Tae Hyeong Kim, Shin Jin Kang, and Chang Hun Kim. 2016. Predicting Fluctuations in Cryptocurrency Transactions Based on User Comments and Replies. PLOS ONE, Vol. 11, 8 (2016).Google Scholar
- Morgan Klaus Scheuerman, Katta Spiel, Oliver L. Haimson, Foad Hamidi, and Stacy M. Branham. 2019. HCI Guidelines for Gender Equity and Inclusivity. Retrieved May 13, 2020 from https://www.morgan-klaus.com/sigchi-gender-guidelines.Google Scholar
- Johannes Konert, Henrik Bellh"auser, René Röpke, Eduard Gallwas, and Ahmed Zucik. 2016. MoodlePeers: Factors Relevant in Learning Group Formation for Improved Learning Outcomes, Satisfaction and Commitment in E-learning Scenarios using GroupAL. In Adaptive and Adaptable Learning: Proceedings of the 11th European Conference on Technology-Enhanced Learning (EC-TEL). 390--396.Google ScholarCross Ref
- Brian Kooiman, Wenling Li, Michael Wesolek, and Heeja Kim. 2015. Validation of the Relatedness Scale of the Intrinsic Motivation Inventory through Factor Analysis. International Journal of Multidisciplinary Research and Modern Education, Vol. 1, 2 (2015), 302--311.Google Scholar
- Theodora Koulouri, Stanislao Lauria, and Robert D. Macredie. 2017. The Influence of Visual Feedback and Gender Dynamics on Performance, Perception and Communication Strategies in CSCW. International Journal of Human-Computer Studies, Vol. 97 (2017), 162--181.Google ScholarCross Ref
- Amit Kramer, Devasheesh P. Bhave, and Tiffany D. Johnson. 2014. Personality and Group Performance: The Importance of Personality Composition and Work Tasks. Personality and Individual Differences, Vol. 58 (2014), 132--137.Google ScholarCross Ref
- Adam D. I. Kramer, Susan R. Fussell, and Leslie D. Setlock. 2004. Text Analysis as a Tool for Analyzing Conversation in Online Support Groups. In CHI '04 Extended Abstracts on Human Factors in Computing Systems (CHI EA). 1485--1488.Google Scholar
- Sandeep Kaur Kuttal, Kevin Gerstner, and Alexandra Bejarano. 2019. Remote Pair Programming in Online CS Education: Investigating through a Gender Lens. In 2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 75--85.Google ScholarCross Ref
- Marjan Laal and Seyed Mohammad Ghodsi. 2012. Benefits of Collaborative Learning. Procedia-Social and Behavioral Sciences, Vol. 31 (2012), 486--490.Google ScholarCross Ref
- Cliff Lampe, Rebecca Gray, Andrew T. Fiore, and Nicole Ellison. 2014. Help is on the Way: Patterns of Responses to Resource Requests on Facebook. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW). 3--15.Google ScholarDigital Library
- Colleen M. Lewis and Niral Shah. 2015. How Equity and Inequity Can Emerge in Pair Programming. In Proceedings of the 11th Annual Conference on International Computing Education Research (ICER). 41--50.Google Scholar
- James R. Lewis and Jeff Sauro. 2018. Item Benchmarks for the System Usability Scale. Journal of Usability Studies, Vol. 13, 3 (2018), 158--167.Google ScholarDigital Library
- Xing Liu and Hari Koirala. 2012. Ordinal Regression Analysis: Using Generalized Ordinal Logistic Regression Models to Estimate Educational Data. Journal of Modern Applied Statistical Methods, Vol. 11, 1 (2012), 242--254.Google ScholarCross Ref
- Michael Madaio, Kun Peng, Amy Ogan, and Justine Cassell. 2018. A Climate of Support: A Process-Oriented Analysis of the Impact of Rapport on Peer Tutoring. In Proceedings of the 13th International Conference of the Learning Sciences (ICLS). 600--607.Google Scholar
- Tara Matthews, Jalal U. Mahmud, Jilin Chen, Michael Muller, Eben Haber, and Hernan Badenes. 2015. They Said What?: Exploring the Relationship Between Language Use and Member Satisfaction in Communities. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW). 819--825.Google ScholarDigital Library
- Hanna Maurin, Diane H. Sonnenwald, Bruce Cairns, James E. Manning, Eugene B. Freid, and Henry Fuchs. 2006. Exploring Gender Differences in Perceptions of 3D Telepresence Collaboration Technology: An Example from Emergency Medical Care. In Proceedings of the 4th Nordic Conference on Human-Computer Interaction: Changing Roles (NordiCHI). 381--384.Google ScholarDigital Library
- Edward McAuley, Terry Duncan, and Vance V. Tammen. 1989. Psychometric Properties of the Intrinsic Motivation Inventory in a Competitive Sport Setting: A Confirmatory Factor Analysis. Research Quarterly for Exercise and Sport, Vol. 60, 1 (1989), 48--58.Google ScholarCross Ref
- Peter McCullagh. 1980. Regression Models for Ordinal Data. Journal of the Royal Statistical Society: Series B (Methodological), Vol. 42, 2 (1980), 109--127.Google ScholarCross Ref
- Charlie McDowell, Linda Werner, Heather Bullock, and Julian Fernald. 2002. The Effects of Pair-Programming on Performance in an Introductory Programming Course. In Proceedings of the 33rd ACM Technical Symposium on Computer Science Education (SIGCSE). 38--42.Google ScholarDigital Library
- Meike Osinski and Nikol Rummel. 2019. Towards Successful Knowledge Integration in Online Collaboration: An Experiment on the Role of Meta-Knowledge. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019).Google ScholarDigital Library
- Jennifer Robison, Scott McQuiggan, and James C. Lester. 2009. Evaluating the Consequences of Affective Feedback in Intelligent Tutoring Systems. In 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. 1--6.Google Scholar
- Fernando J. Rodríguez, Kimberly Michelle Price, and Kristy Elizabeth Boyer. 2017. Expressing and Addressing Uncertainty: A Study of Collaborative Problem-Solving Dialogues. In Proceedings of the 12th International Conference on Computer Supported Collaborative Learning (CSCL). 207--214.Google Scholar
- Jeff Sauro and James R. Lewis. 2016. Quantifying the User Experience: Practical Statistics for User Research.Google ScholarDigital Library
- Linda J. Sax, Kathleen J. Lehman, Jerry A. Jacobs, M. Allison Kanny, Gloria Lim, Laura Monje-Paulson, and Hilary B. Zimmerman. 2017. Anatomy of an Enduring Gender Gap: The Evolution of Women?s Participation in Computer Science. The Journal of Higher Education, Vol. 88, 2 (2017), 258--293.Google ScholarCross Ref
- Toni Schmader. 2012. Stereotype Threat: Theory, Process, and Application. Oxford University Press.Google Scholar
- Andrew J. Scholand, Yla R. Tausczik, and James W. Pennebaker. 2010. Social Language Network Analysis. In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work (CSCW). 23--26.Google Scholar
- Joseph Seering, Felicia Ng, Zheng Yao, and Geoff Kaufman. 2018. Applications of Social Identity Theory to Research and Design in Computer-Supported Cooperative Work. Proceedings of the ACM on Human-Computer Interaction, Vol. 2, CSCW (2018).Google ScholarDigital Library
- Lauren E. Sherman, Minas Michikyan, and Patricia M. Greenfield. 2013. The Effects of Text, Audio, Video, and In-Person Communication on Bonding Between Friends. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, Vol. 7, 2 (2013).Google Scholar
- Angela E.B. Stewart, Hana Vrzakova, Chen Sun, Jade Yonehiro, Cathlyn Adele Stone, Nicholas D. Duran, Valerie Shute, and Sidney K. D'Mello. 2019. I Say, You Say, We Say: Using Spoken Language to Model Socio-Cognitive Processes during Computer-Supported Collaborative Problem Solving. Proceedings of the ACM on Human-Computer Interaction (CHI), Vol. 3, CSCW (2019), 1--19.Google Scholar
- Henri Tajfel. 1974. Social Identity and Intergroup Behaviour. Information (International Social Science Council), Vol. 13, 2 (1974), 65--93.Google ScholarCross Ref
- Chiew Seng Sean Tan, Johannes Schöning, Kris Luyten, and Karin Coninx. 2014. Investigating the Effects of Using Biofeedback as Visual Stress Indicator during Video-Mediated Collaboration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). 71--80.Google ScholarDigital Library
- Yla R. Tausczik and James W. Pennebaker. 2013. Improving Teamwork Using Real-Time Language Feedback. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). 459--468.Google Scholar
- Divy Thakkar, Nithya Sambasivan, Purva Kulkarni, Pratap Kalenahalli Sudarshan, and Kentaro Toyama. 2018. The Unexpected Entry and Exodus of Women in Computing and HCI in India. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI). 1--12.Google ScholarDigital Library
- David Thissen, Lynne Steinberg, and Daniel Kuang. 2002. Quick and Easy Implementation of the Benjamini-Hochberg Procedure for Controlling the False Positive Rate in Multiple Comparisons. Journal of Educational and Behavioral Statistics, Vol. 27, 1 (2002), 77--83.Google ScholarCross Ref
- Patti M. Valkenburg and Jochen Peter. 2009. Social Consequences of the Internet for adolescents: A Decade of Research. Current Directions in Psychological Science, Vol. 18, 1 (2009), 1--5.Google ScholarCross Ref
- Jan M. van Bruggen, Paul A. Kirschner, and Wim Jochems. 2002. External Representation of Argumentation in CSCL and the Management of Cognitive Load. Learning and Instruction, Vol. 12, 1 (2002), 121--138.Google ScholarCross Ref
- April Yi Wang, Anant Mittal, Christopher Brooks, and Steve Oney. 2019. How Data Scientists Use Computational Notebooks for Real-Time Collaboration. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019), 1--30.Google ScholarDigital Library
- Armin Weinberger, Karsten Stegmann, and Frank Fischer. 2007. Knowledge Convergence in Collaborative Learning: Concepts and Assessment. Learning and Instruction, Vol. 17, 4 (2007), 416--426.Google ScholarCross Ref
- Linda L. Werner, Brian Hanks, and Charlie McDowell. 2004. Pair-Programming Helps Female Computer Science Students. ACM Journal of Educational Resources in Computing, Vol. 4, 1 (2004), 1--8.Google Scholar
- Laurie Williams and Richard L. Upchurch. 2001. In Support of Student Pair-Programming. In Proceedings of the 32nd ACM Technical Symposium on Computer Science Education (SIGCSE). 327--331.Google Scholar
- Svante Wold, Kim Esbensen, and Paul Geladi. 1987. Principal Component Analysis. Chemometrics and Intelligent Laboratory Systems, Vol. 2, 1--3 (1987), 37--52.Google ScholarCross Ref
- Kimberly Michelle Ying, Lydia G. Pezzullo, Mohona Ahmed, Kassandra Crompton, Jeremiah Blanchard, and Kristy Elizabeth Boyer. 2019. In Their Own Words: Gender Differences in Student Perceptions of Pair Programming. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE). 1053--1059.Google ScholarDigital Library
- Jaebong Yoo and Jihie Kim. 2014. Can Online Discussion Participation Predict Group Project Performance? Investigating the Roles of Linguistic Features and Participation Patterns. International Journal of Artificial Intelligence in Education, Vol. 24, 1 (2014), 8--32.Google ScholarCross Ref
- Chien Wen (Tina) Yuan, Yu-Hsuan Liu, Hao-Chuan Wang, and Yuan-Chi Tseng. 2019. Gender Effects on Collaborative Online Brainstorming Teamwork. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA). 1--6.Google Scholar
- Mark Zarb, Janet Hughes, and John Richards. 2013. Industry-Inspired Guidelines Improve Students' Pair Programming Communication. In Proceedings of the 18th ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE). 135--140.Google ScholarDigital Library
- Kristina M. Zosuls, Cindy Faith Miller, Diane N. Ruble, Carol Lynn Martin, and Richard A. Fabes. 2011. Gender Development Research in Sex Roles: Historical Trends and Future Directions. Sex Roles, Vol. 64, 11--12 (2011), 826--842.Google ScholarCross Ref
- Stuart Zweben and Betsy Bizot. 2019. 2018 Taulbee Survey: Undergrad Enrollment Continues Upward; Doctoral Degree Production Declines but Doctoral Enrollment Rises. Computing Research News, Vol. 31, 5 (2019), 3--74.Google Scholar
- Stuart Zweben, Jodi Tims, and Yan Timanovsky. 2019. ACM-NDC Study 2018--2019: Seventh Annual Study of Non-Doctoral-Granting Departments in Computing. ACM Inroads, Vol. 10, 3 (2019), 40--54.Google ScholarDigital Library
Index Terms
- Understanding Women's Remote Collaborative Programming Experiences: The Relationship between Dialogue Features and Reported Perceptions
Recommendations
A groupware system to support collaborative programming: Design and experiences
The advances in network and collaboration technologies enable the creation of powerful environments for collaborative programming. One such environment is COLLECE, a groupware system to support collaborative edition, compilation and execution of ...
Remote collaborative multi-user informal learning experiences: design and evaluation
EC-TEL'11: Proceedings of the 6th European conference on Technology enhanced learning: towards ubiquitous learningThis paper presents a customizable system used to develop a collaborative multi-user problem solving game. It addresses the increasing demand for appealing informal learning experiences in museum-like settings. The system facilitates remote ...
Collaboard: a remote collaboration groupware device featuring an embodiment-enriched shared workspace
GROUP '10: Proceedings of the 2010 ACM International Conference on Supporting Group WorkIn this paper we present a mixed presence groupware device called "CollaBoard". The device improves collaboration between co-located and remote partners by providing a high level of workspace awareness. This is achieved by superimposing a life-size ...
Comments