Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
MTZ-Fachkongress

Antriebe und Energiesysteme von morgen


Austausch von Automobil- und Energiewirtschaft

Am 14./15. Mai geht es in Chemnitz um die Mobilitätswende durch Elektro- und Wasserstofffahrzeuge.
Erweiterte Suche
Anmelden
nach oben
Empirical Software Engineering
Erschienen in: Empirical Software Engineering 1/2023

01.01.2023

Automatic prediction of rejected edits in Stack Overflow

verfasst von: Saikat Mondal, Gias Uddin, Chanchal Roy

Erschienen in: Empirical Software Engineering | Ausgabe 1/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The content quality of shared knowledge in Stack Overflow (SO) is crucial in supporting software developers with their programming problems. Thus, SO allows its users to suggest edits to improve the quality of a post (i.e., question and answer). However, existing research shows that many suggested edits in SO are rejected due to undesired contents/formats or violating edit guidelines. Such a scenario frustrates or demotivates users who would like to conduct good-quality edits. Therefore, our research focuses on assisting SO users by offering them suggestions on how to improve their editing of posts. First, we manually investigate 764 (382 questions + 382 answers) rejected edits by rollbacks and produce a catalog of 19 rejection reasons. Second, we extract 15 texts and user-based features to capture those rejection reasons. Third, we develop four machine learning models using those features. Our best-performing model can predict rejected edits with 69.1% precision, 71.2% recall, 70.1% F1-score, and 69.8% overall accuracy. Fourth, we introduce an online tool named EditEx that works with the SO edit system. EditEx can assist users while editing posts by suggesting the potential causes of rejections. We recruit 20 participants to assess the effectiveness of EditEx. Half of the participants (i.e., treatment group) use EditEx and another half (i.e., control group) use the SO standard edit system to edit posts. According to our experiment, EditEx can support SO standard edit system to prevent 49% of rejected edits, including the commonly rejected ones. However, it can prevent 12% rejections even in free-form regular edits. The treatment group finds the potential rejection reasons identified by EditEx influential. Furthermore, the median workload suggesting edits using EditEx is half compared to the SO edit system.
Vorheriger Artikel A systematic literature review on trust in the software ecosystem
Nächster Artikel A controlled experiment on the impact of microtasking on programming

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren
Fußnoten
Literatur
Abdellatif A, Costa D, Badran K, Abdalkareem R, Shihab E (2020) Challenges in chatbot development: a study of stack overflow posts. In: 17th International conference on mining software repositories, October 5–6, 2020, Seoul, Republic of Korea. ACM, New York
Agichtein E, Castillo C, Donato D, Gionis A, Mishne G (2008) Finding high-quality content in social media. In: Proceedings of the 2008 international conference on web search and data mining, pp 183–194
Ahmed S, Bagherzadeh M (2018) What do concurrency developers ask about?: a large-scale study using stack overflow. In: Proceedings of the 12th ACM/IEEE international symposium on empirical software engineering and measurement, article no. 30
Alkadhi R, Lata T, Guzmany E, Bruegge B (2017) Rationale in development chat messages: an exploratory study. In: 2017 IEEE/ACM 14th international conference on mining software repositories (MSR), IEEE, pp 436–446
Alkadhi R, Nonnenmacher M, Guzman E, Bruegge B (2018) How do developers discuss rationale?. In: 2018 IEEE 25th international conference on software analysis, evolution and reengineering (SANER). IEEE, pp 357–369
Anderson A, Huttenlocher D, Kleinberg J, Leskovec J (2012) Discovering value from community activity on focused question answering sites: a case study of stack overflow. In: Proceedings of the 18th ACM SIGKDD international conference on knowledge discovery and data mining, pp 850–858
Asaduzzaman M, Mashiyat A S, Roy C K, Schneider K A (2013) Answering questions about unanswered questions of stack overflow. In: Proceedings of the 10th working conference on mining software repositories, pp 87–100
Bagherzadeh M, Khatchadourian R (2019) Going big: a large-scale study on what big data developers ask. In: Proceedings of the 2019 27th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, ESEC/FSE 2019. ACM, New York, pp 432–442
Bagozzi R P, Dholakia U M (2006) Open source software user communities: a study of participation in linux user groups. J Manag Sci 52(7):1099–1115
Baltes S, Dumani L, Treude C, Diehl S (2018) Sotorrent: reconstructing and analyzing the evolution of stack overflow posts. In: Proceedings of the 15th international conference on mining software repositories, pp 319–330
Barua A, Thomas S W, Hassan A E (2012) What are developers talking about? An analysis of topics and trends in stack overflow. Empir Softw Eng 1–31
Beyer S, Macho C, Di Penta M, Pinzger M (2018) Automatically classifying posts into question categories on stack overflow. In: 2018 IEEE/ACM 26th international conference on program comprehension (ICPC). IEEE, pp 211–21110
Bi T, Xia X, Lo D, Grundy J, Zimmermann T, Ford D (2021) Accessibility in software practice: a practitioner’s perspective. arXiv:210308778
Boslaugh S (2012) Statistics in a nutshell: a desktop quick reference. O’Reilly Media, Inc
Calefato F, Lanubile F, Novielli N (2017) Emotxt: a toolkit for emotion recognition from text. In: 2017 Seventh international conference on affective computing and intelligent interaction workshops and demos (ACIIW), pp 79–80
Calefato F, Lanubile F, Novielli N (2018) How to ask for technical help? Evidence-based guidelines for writing questions on stack overflow. J Inf Softw Technol 94:186–207CrossRef
Calefato F, Lanubile F, Novielli N (2019) An empirical assessment of best-answer prediction models in technical q&a sites. ESE 1–48
Calvo R A, O’Rourke S T, Jones J, Yacef K, Reimann P (2005) Collaborative writing support tools on the cloud. IEEE Trans Learn Technol 41:66–99
Cao A, Chintamani K K, Pandya A K, Ellis R D (2009) Nasa tlx: software for assessing subjective mental workload. Behav Res Methods 41(1):113–117CrossRef
Chakraborty P, Shahriyar R, Iqbal A, Uddin G (2021) How do developers discuss and support new programming languages in technical q&a site? An empirical study of go, swift, and rust in stack overflow. Information and Software Technology (IST) 19
Chatterjee P, Damevski K, Pollock L, Augustine V, Kraft N A (2019) Exploratory study of slack q&a chats as a mining source for software engineering tools. In: 2019 IEEE/ACM 16th international conference on mining software repositories (MSR). IEEE, pp 490–501
Chen T, Guestrin C (2016) Xgboost: a scalable tree boosting system. In: Proceedings of the 22nd ACM sigkdd international conference on knowledge discovery and data mining, pp 785–794
Chen C, Xing Z, Liu Y (2017a) By the community & for the community: a deep learning approach to assist collaborative editing in q&a sites. In: Proceedings of the ACM on human-computer interaction, Article 32
Chen C, Xing Z, Liu Y (2017b) By the community & for the community: a deep learning approach to assist collaborative editing in q&a sites. In: Proceedings of the ACM on human-computer interaction, Article No. 32
Chen C, Chen X, Sun J, Xing Z, Li G (2018) Data-driven proactive policy assurance of post quality in community q&a sites. In: Proceedings of the ACM on human-computer interaction, Article 33
Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37–46CrossRef
Cohen J (1968) Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychol Bull 70(4):213CrossRef
Dabbish L, Stuart C, Tsay J, Herbsleb J (2012) Social coding in github: transparency and collaboration in an open software repository. In: Proceedings of the ACM conference on computer supported cooperative work, pp 37–46
Davis J, Goadrich M (2006) The relationship between precision-recall and roc curves. In: Proceedings of ICML, pp 233–240
Duijn M, Kucera A, Bacchelli A (2015) Quality questions need quality code: classifying code fragments on stack overflow. In: Proceedings of the IEEE/ACM 12th working conference on mining software repositories, pp 410–413
Garcia D, Zanetti M S, Schweitzer F (2013) The role of emotions in contributors activity: a case study on the gentoo community. In: 2013 International conference on cloud and green computing. IEEE, pp 410–417
Goldberger J, Hinton G E, Roweis S T, Salakhutdinov R R (2005) Neighbourhood components analysis. In: Advances in neural information processing systems, pp 513–520
Gottipati S, Lo D, Jiang J (2011) Finding relevant answers in software forums. In: 2011 26th IEEE/ACM international conference on automated software engineering (ASE 2011). IEEE, pp 323–332
Groves R M, Fowler J F J, Couper M P, Lepkowski J M, Singer E, Tourangeau R (2011) Survey methodology
Guzman E, Bruegge B (2013) Towards emotional awareness in software development teams. In: Proceedings of the 2013 9th joint meeting on foundations of software engineering, pp 671–674
Guzman E, Azócar D, Li Y (2014) Sentiment analysis of commit comments in github: an empirical study. In: Proceedings of the 11th working conference on mining software repositories, pp 352–355
Harper F M, Raban D, Rafaeli S, Konstan J A (2008) Predictors of answer quality in online q&a sites. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 865–874
Hart S G (1986) Nasa task load index (tlx)
Hart S G, Staveland L E (1988) Development of nasa-tlx (task load index): results of empirical and theoretical research. In: Advances in psychology, vol 52. Elsevier, pp 139–183
He H, Garcia E A (2008) Learning from imbalanced data. IEEE Trans Knowl Data Eng (9):1263–1284
Hudson N, Chilana P K, Guo X, Day J, Liu E (2015) Understanding triggers for clarification requests in community-based software help forums. In: Proceedings of the IEEE symposium on visual languages and human-centric computing, pp 189–193
Islam M R, Zibran M F (2016) Towards understanding and exploiting developers’ emotional variations in software engineering. In: 2016 IEEE 14th international conference on software engineering research, management and applications (SERA). IEEE, pp 185–192
Jongeling R, Sarkar P, Datta S, Serebrenik A (2017) On negative results when using sentiment analysis tools for software engineering research. Empir Softw Eng 22(5):2543–2584CrossRef
Joshi A, Kale S, Chandel S, Pal D K (2015) Likert scale: explored and explained. CJAST
Khan J Y, Khondaker M T I, Uddin G, Iqbal A (2021) Automatic detection of five api documentation smells: practitioners’ perspectives. In: IEEE International conference on software analysis, evolution and reengineering (SANER), p 12
Kitchenham B A, Pfleeger S L (2008) Personal opinion surveys. In: Guide to advanced empirical software engineering
Kittur A, Kraut R E (2008) Harnessing the wisdom of crowds in wikipedia: quality through coordination. In: Proceedings of the ACM conference on Computer supported cooperative work, pp 37–46
Lakhani K R, von Hippel E (2003) How open source software works: free user-to-user assistance. J Res Policy 32(6):923–943CrossRef
Li G, Zhu H, Lu T, Ding X, Gu N (2015a) Is it good to be like wikipedia?: exploring the trade-offs of introducing collaborative editing model to q&a sites. In: Proceedings of the 18th ACM conference on computer supported cooperative work & social computing, pp 1080–1091
Li L, He D, Jeng W, Goodwin S, Zhang C (2015b) Answer quality characteristics and prediction on an academic q&a site: a case study on researchgate. In: Proceedings of the 24th international conference on World Wide Web, pp 1453–1458
Lowry P B, Curtis A M, Lowry M R (2005) A taxonomy of collaborative writing to improve empirical research, writing practice, and tool development. J Bus Commun 41:66–99CrossRef
Lundberg S M, Lee S I (2017) A unified approach to interpreting model predictions. In: Proceedings of the 31st international conference on neural information processing systems
Lundberg S M, Erion G, Chen H, DeGrave A, Prutkin J M, Nair B, Katz R, Himmelfarb J, Bansal N, Lee S I (2020) From local explanations to global understanding with explainable ai for trees. Nat Mach Intell 2 (1):56–67CrossRef
Macbeth G, Razumiejczyk E, Ledesma R D (2011) Cliff’s delta calculator: a non-parametric effect size program for two groups of observations. Univ Psychol 10(2):545–555CrossRef
McKnight P E, Najab J (2010) Mann-whitney u test. The Corsini Encyclopedia of Psychology 1–1
Memarian B, Mitropoulos P (2011) Work factors affecting task demands of masonry work. In: Proceedings of the 47th annual international conference of associated schools of construction
Molnar C (2020) Interpretable machine learning. Lulu.com
Mondal S, Rahman M M, Roy C K (2019) Can issues reported at stack overflow questions be reproduced?: an exploratory study. In: Proceedings of the 16th international conference on mining software repositories, pp 479–489
Mondal S, Uddin G, Roy C K (2020) Automatic identification of rollback edit with reasons in stack overflow q&a site. In: 36th IEEE international conference on software maintenance and evolution (ICSME)—registered protocol report, pp 856–856
Mondal S, Uddin G, Roy C K (2021a) Rollback edit inconsistencies in developer forum. In: 2021 IEEE/ACM 18th international conference on mining software repositories (MSR), pp 380–391
Munteanu C, Baecker R, Penn G (2008) Collaborative editing for improved usefulness and usability of transcript-enhanced webcasts. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 373–382
Murgia A, Tourani P, Adams B, Ortu M (2014) Do developers feel emotions? An exploratory analysis of emotions in software artifacts. In: Proceedings of the 11th working conference on mining software repositories, pp 262–271
Novielli N, Calefato F, Lanubile F (2014) Towards discovering the role of emotions in stack overflow. In: Proceedings of the 6th international workshop on social software engineering, pp 33–36
Novielli N, Calefato F, Lanubile F (2015) The challenges of sentiment detection in the social programmer ecosystem. In: Proceedings of the 7th international workshop on social software engineering, pp 33–40
Noyes J M, Bruneau D P (2007) A self-analysis of the nasa-tlx workload measure. Ergonomics 50(4):514–519CrossRef
Ortu M, Adams B, Destefanis G, Tourani P, Marchesi M, Tonelli R (2015) Are bullies more productive? Empirical study of affectiveness vs. issue fixing time. In: 2015 IEEE/ACM 12th working conference on mining software repositories. IEEE, pp 303–313
Parnin C, Treude C, Grammel L, Storey M A (2012) Crowd documentation: exploring the coverage and the dynamics of api discussions on stack overflow. Tech. rep. Georgia Tech.
Polikar R (2006) Ensemble based systems in decision making. IEEE Circ Syst Mag 6(3):21–45CrossRef
Ponzanelli L, Mocci A, Bacchelli A, Lanza M (2014a) Improving low quality stack overflow post detection. In: Proceedings of the 30th international conference on software maintenance and evolution, pp 541–544
Ponzanelli L, Mocci A, Bacchelli A, Lanza M (2014b) Understanding and classifying the quality of technical forum questions. In: 2014 14th International conference on quality software. IEEE, pp 343–352
Ponzanelli L, Mocci A, Bacchelli A, Lanza M (2014c) Understanding and classifying the quality of technical forum questions. In: Proceedings of the 14th international conference on quality software, pp 343–352
Ponzanelli L, Mocci A, Bacchelli A, Lanza M, Fullerton D (2014d) Improving low quality stack overflow post detection. In: Proceedings of the IEEE international conference on software maintenance and evolution, pp 541–544
Prasetyo P K, Lo D, Achananuparp P, Tian Y, Lim E P (2012) Automatic classification of software related microblogs. In: 2012 28th IEEE international conference on software maintenance (ICSM). IEEE, pp 596–599
Rahman M M, Roy C K (2015a) An insight into the unresolved questions at stack overflow. In: Proceedings of the 12th working conference on mining software repositories. IEEE Press, pp 426–429
Rahman M M, Roy C K (2015b) An insight into the unresolved questions at stack overflow. In: Proceedings of the 12th working conference on mining software repositories, pp 426–429
Rainer A, Hall T, Baddoo N (2003) Persuading developers to “buy into” software process improvement: a local opinion and empirical evidence. In: 2003 International symposium on empirical software engineering, 2003. ISESE 2003. Proceedings. IEEE, pp 326–335
Saha R K, Saha A K, Perry D E (2013) Toward understanding the causes of unanswered questions in software information sites: a case study of stack overflow. In: Proceedings of FSE, pp 663–666
Seaman C B, Basili V R (1998) Communication and organization: an empirical study of discussion in inspection meetings. IEEE Trans Softw Eng 24 (7):559–572CrossRef
Sharek D (2011) A useable, online nasa-tlx tool. In: Proceedings of the human factors and ergonomics society annual meeting, vol 55. SAGE Publications Sage, Los Angeles, pp 1375–1379
Shihab E, Jiang Z M, Hassan A E (2009) Studying the use of developer irc meetings in open source projects. In: 2009 IEEE International conference on software maintenance. IEEE, pp 147–156
Singer J, Vinson N G (2002) Ethical issues in empirical studies of software engineering. TSE
Stratton S J (2021) Population research: convenience sampling strategies. Prehosp Disaster Med 36(4):373–374CrossRef
Tian Y, Achananuparp P, Lubis I N, Lo D, Lim E P (2012) What does software engineering community microblog about?. In: 2012 9th IEEE working conference on mining software repositories (MSR). IEEE, pp 247–250
Tian Y, Lo D, Lawall J (2014) Automated construction of a software-specific word similarity database. In: Proceedings of CSMR-WCRE, pp 44–53
Uddin G, Khomh F (2017a) Automatic summarization of API reviews. In: Proceedings of 32nd IEEE/ACM international conference on automated software engineering, p 12
Uddin G, Khomh F (2017c) Opiner: a search and summarization engine for API reviews. In: Proceedings of 32nd IEEE/ACM international conference on automated software engineering, p 6
Uddin G, Khomh F (2019) Automatic opinion mining from API reviews from stack overflow. IEEE Trans Softw Eng 35
Uddin G, Robillard M P (2015) How api documentation fails. IEEE Softw 32(4):76–83CrossRef
Uddin G, Baysal O, Guerroj L, Khomh F (2019) Understanding how and why developers seek and analyze api related opinions. IEEE Trans Softw Eng 40
Uddin G, Khomh F, Roy C K (2020a) Automatic api usage scenario documentation from technical q&a sites. ACM Trans Softw Eng Methodol 43
Uddin G, Khomh F, Roy C K (2020b) Automatic mining of api usage scenarios from stack overflow. Inf Softw Technol (IST) 16
Vagias W M (2006) Likert-type scale response anchors. Clemson International Institute for Tourism & Research Development, Department of Parks, Recreation and Tourism Management Clemson University
Wang J, Xu M, Wang H, Zhang J (2006) Classification of imbalanced data by using the smote algorithm and locally linear embedding. In: 2006 8th International conference on signal processing, vol 3. IEEE
Wang X, Kuzmickaja I, Stol K J, Abrahamsson P, Fitzgerald B (2013) Microblogging in open source software development: the case of drupal and twitter. IEEE Softw 31(4):72–80CrossRef
Wang S, Chen T H P, Hassan A E (2018) How do users revise answers on technical Q&A websites? A case study on stack overflow. IEEE Trans Softw Eng 19
Ya Y, Tong H, Xie T, Akoglu L, Xu F, Lu J (2013) Want a good answer? Ask a good question first! Tech. rep. arXiv:1311.​6876
Ya Y, Tong H, Xie T, Akoglu L, Xu F, Lu J (2015) Detecting high-quality posts in community question answering sites. J Inf Sci 302 (1):70–82CrossRef
Yujian L, Bo L (2007) A normalized levenshtein distance metric. IEEE Trans Pattern Anal Mach Intell 29(6):1091–1095CrossRef
Zhang T, Upadhyaya G, Reinhardt A, Rajan H, Kim M (2018) Are code examples on an online q&a forum reliable?: a study of api misuse on stack overflow. In: Proceedings of the 40th international conference on software engineering, pp 886–896
Metadaten
Titel
Automatic prediction of rejected edits in Stack Overflow
verfasst von
Saikat Mondal
Gias Uddin
Chanchal Roy
Publikationsdatum
01.01.2023
Verlag
Springer US
Erschienen in
Empirical Software Engineering / Ausgabe 1/2023
Print ISSN: 1382-3256
Elektronische ISSN: 1573-7616
DOI
https://doi.org/10.1007/s10664-022-10242-2

Weitere Artikel der Ausgabe 1/2023

Empirical Software Engineering 1/2023 Zur Ausgabe

OriginalPaper

Towards effective assessment of steady state performance in Java software: are we there yet?

OriginalPaper

Variational satisfiability solving: efficiently solving lots of related SAT problems

OriginalPaper

On the privacy of mental health apps

OriginalPaper

Smells in system user interactive tests

OriginalPaper

Evaluating classifiers in SE research: the ECSER pipeline and two replication studies

OriginalPaper

Can static analysis tools find more defects?

Premium Partner

    Bildnachweise