Skip to main content
SpringerLink
Log in

GARUSO: a gamification approach for involving stakeholders outside organizational reach in requirements engineering

  • Original Article
  • Published:
Requirements Engineering Aims and scope Submit manuscript

Abstract

Stakeholder participation is a key success factor of Requirements Engineering (RE). Typically, the techniques used for identifying and involving stakeholders in RE assume that stakeholders can be identified among the members of the organizations involved when a software system is ordered, developed or maintained—and that these stakeholders can be told or even mandated to contribute. However, these assumptions no longer hold for many of today’s software systems where significant stakeholders (in particular, end-users and people affected by a system) are outside organizational reach: They are neither known nor can they easily be identified in the involved organizations nor can they be told to participate in RE activities. We have developed the GARUSO approach to address this problem. It uses a strategy for identifying stakeholders outside organizational reach and a social media platform that applies gamification for motivating these stakeholders to participate in RE activities. In this article, we describe the GARUSO approach and report on its empirical evaluation. We found that the identification strategy attracted a crowd of stakeholders outside organizational reach to the GARUSO platform and motivated them to participate voluntarily in collaborative RE activities. From our findings, we derived a first set of design principles on how to involve stakeholders outside organizational reach in RE. Our work expands the body of knowledge on crowd RE regarding stakeholders outside organizational reach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. The questionnaire is available under: https://figshare.com/s/4da2e2c4469bc590a97c.

  2. https://www.unipark.com/en/.

  3. The data is available under: https://figshare.com/s/d3abc24c965395abb6fd.

  4. An example of the onboarding of novice users is available here: https://figshare.com/s/e231f59d1fc0d828ba7f.

  5. https://translate.google.com/.

  6. https://www.djangoproject.com/.

  7. https://getbootstrap.com/.

  8. http://fontawesome.io/.

  9. https://www.sharethis.com/.

  10. https://analytics.google.com.

  11. The data used for the quantitative analysis is available at https://figshare.com/s/00cd571cf8cd67a207fb.

  12. https://support.google.com/analytics/answer/6205762.

  13. https://support.google.com/adwords/answer/2404190.

References

  1. Bano M, Zowghi D (2015) A systematic review on the relationship between user involvement and system success. Inf Softw Technol 58:148–169

    Article  Google Scholar 

  2. Bartle R (1996) Hearts, clubs, diamonds, spades: players who suit MUDs. J MUD Res 1(1):19

    Google Scholar 

  3. Bartle R (2004) Designing virtual worlds. New Riders

  4. Bartle R (2005) Virtual worlds: why people play. Massively Multiplayer Game Dev 2(1):3–18

    Google Scholar 

  5. Brody N (1994) Heritability of traits. Psychol Inquiry 5(2):117–119

    Article  Google Scholar 

  6. Cox EP III (1980) The optimal number of response alternatives for a scale: a review. J Mark Res 17(4):407–422

    Article  Google Scholar 

  7. Dalpiaz F, Snijders R, Brinkkemper S, Hosseini M, Shahri A, Ali R (2017) Engaging the crowd of stakeholders in requirements engineering via gamification. In: Gamification: using game elements in serious contexts. Springer, Berlin, pp 123–135

  8. Deterding S, Dixon D, Khaled R, Nacke L (2011) From game design elements to gamefulness: defining gamification. In: 15th International academic MindTrek conference: envisioning future media environments, pp 9–15. ACM

  9. Dreyfus SE, Dreyfus HL (1980) A five-stage model of the mental activities involved in directed skill acquisition. TR ORC 80-2, University of California, Berkeley

  10. Empa: Nest—exploring the future of buildings. https://www.empa.ch/web/nest/ Accessed 25 Aug 2016

  11. Farzan R, Brusilovsky P (2011) Encouraging user participation in a course recommender system: an impact on user behavior. Comput Hum Behav 27(1):276–284

    Article  Google Scholar 

  12. Fernandes J, Duarte D, Ribeiro C, Farinha C, Pereira JM, da Silva MM (2012) iThink: a game-based approach towards improving collaboration and participation in requirement elicitation. Procedia Comput Sci 15:66–77

    Article  Google Scholar 

  13. Ferro LS, Walz SP, Greuter S (2013) Towards personalised, gamified systems: an investigation into game design, personality and player typologies. In: 9th Australasian conference on interactive entertainment: matters of life and death. ACM, pp 1–6

  14. Glinz M (2014) A glossary of requirements engineering terminology, version 1.6. International Requirements Engineering Board, IREB

  15. Glinz M, Wieringa RJ (2007) Stakeholders in requirements engineering. IEEE Softw 24(2):18–20

    Article  Google Scholar 

  16. Goguen J (1994) Requirements engineering as the reconciliation of technical and social issues. In: Requirements Engineering: Social and Technical Issues. Academic Press, London, pp 165–199

  17. Goodman LA (1961) Snowball sampling. Ann Math Stat 32(1):148–170

    MathSciNet  MATH  Google Scholar 

  18. Gosling SD, Vazire S, Srivastava S, John OP (2004) Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires. Am Psychol 59(2):93–104

    Article  Google Scholar 

  19. Groen EC, Seyff N, Ali R, Dalpiaz F, Doerr J, Guzman E, Hosseini M, Marco J, Oriol M, Perini A, Stade M (2017) The crowd in requirements engineering: the landscape and challenges. IEEE Softw 34(2):44–52

    Article  Google Scholar 

  20. Guzman E, Ibrahim M, Glinz M (2017) A little bird told me: Mining tweets for requirements and software evolution. In: 25th IEEE international requirements engineering conference (RE’17), pp 11–20. IEEE

  21. Hawlitschek A, Köppen V (2014) Analyzing player behavior in digital game-based learning: advantages and challenges. In: 8th European conference on games based learning. Academic Conferences and Publishing International, pp 199–206

  22. Hosseini M, Shahri A, Phalp K, Taylor J, Ali R, Dalpiaz F (2015) Configuring crowdsourcing for requirements elicitation. In: 9th International conference on research challenges in information science (RCIS), pp 133–138

  23. Huber MZ, Hilty LM (2015) Gamification and sustainable consumption: overcoming the limitations of persuasive technologies. In: ICT innovations for sustainability. Springer, Berlin, pp 367–385

  24. Huber MZ, Hilty LM, Glinz M (2015) Uncovering sustainability requirements: an exploratory case study in canteens. In: 5th International workshop on requirements engineering for sustainable systems. CEUR

  25. Huber Kolpondinos MZ, Glinz M (2017) Behind points and levels—the influence of gamification algorithms on requirements prioritization. In: 25th IEEE International requirements engineering conference (RE’17), pp 332–341

  26. Huber Kolpondinos MZ, Glinz M (2017) Tailoring gamification to requirements elicitation: a stakeholder centric motivation concept. In: 10th International workshop on cooperative and human aspects of software engineering (CHASE)@ICSE2017. IEEE, pp 9–15

  27. Ipeirotis PG (2010) Demographics of mechanical turk. NYU working paper (CEDER-10-01)

  28. Ipeirotis PG, Gabrilovich E (2014) Quizz: targeted crowdsourcing with a billion (potential) users. In: 23rd International conference on world wide web. ACM, pp 143–154

  29. Johann T, Maalej W (2015) Democratic mass participation of users in requirements engineering? In: 23rd IEEE International requirements engineering conference (RE’15), pp 256–261

  30. Kankanhalli A, Taher M, Cavusoglu H, Kim SH (2012) Gamification: a new paradigm for online user engagement. In: International conference on information systems (ICIS 2012), vol. 4. pp 3573–3582

  31. Kim Y, Sohn D, Choi SM (2011) Cultural difference in motivations for using social network sites: a comparative study of American and Korean college students. Comput Hum Behav 27(1):365–372

    Article  Google Scholar 

  32. Kolb DA (1984) Experiential learning: experience as the source of learning and development. Prentice-Hall, Englewood Cliffs

    Google Scholar 

  33. Konaté J, Sahraoui AEK, Kolfschoten GL (2014) Collaborative requirements elicitation: a process-centred approach. Group Decis Negot 23(4):847–877

    Article  Google Scholar 

  34. Lago P, Penzenstadler B (2017) Reality check for software engineering for sustainability–pragmatism required. J Softw: Evol Process 29(2)

  35. Lee JJ, Hammer J (2011) Gamification in education: what, how, why bother? Acad Exch Q 15(2):146

    Google Scholar 

  36. Lim SL, Damian D, Ishikawa F, Finkelstein A (2013) Using web 2.0 for stakeholder analysis: StakeSource and its application in ten industrial projects. Springer, Berlin, pp 221–242

  37. Lim SL, Quercia D, Finkelstein A (2010) StakeNet: using social networks to analyse the stakeholders of large-scale software projects. In: 32nd International conference on software engineering. ACM, pp 295–304

  38. Lim SL, Quercia D, Finkelstein A (2010) StakeSource: harnessing the power of crowdsourcing and social networks in stakeholder analysis. In: 32nd International conference on software engineering—vol 2. ACM, pp 239–242

  39. Lohmann S, Dietzold S, Heim P, Heino N (2009) A web platform for social requirements engineering. In: Software engineering, workshop volume, Lecture Notes in Informatics, vol P-150, pp 309–315

  40. Lombriser P, Dalpiaz F, Lucassen G, Brinkkemper S (2016) Gamified requirements engineering: model and experimentation. In: 22nd International working conference on requirements engineering: foundation for software quality (REFSQ 2016). Springer, Berlin, pp 171–187

  41. Lucassen G, Dalpiaz F, van der Werf JM, Brinkkemper S (2015) Forging high-quality user stories: towards a discipline for agile requirements. In: 23rd IEEE International requirements engineering conference (RE’15), pp 126–135

  42. Maalej W, Pagano D (2011) On the socialness of software. In: 9th International conference on dependable, autonomic and secure computing. IEEE, pp 864–871

  43. Marczewski A (2015) User types. In: Even Ninja monkeys like to play: gamification, game thinking and motivational design (1st ed.). CreateSpace Independent Publishing Platform, pp 65–80

  44. Maslow AH (1943) A theory of human motivation. Psychol Rev 50(4):370–396

    Article  Google Scholar 

  45. Nielsen J (2006) The 90-9-1 rule for participation inequality in social media and online communities. https://www.nngroup.com/articles/participation-inequality/, Accessed 23 Apr 2019

  46. Nonnecke B, Preece J (2000) Lurker demographics: counting the silent. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’00, pp 73–80. ACM

  47. Nuseibeh B, Easterbrook S (2000) Requirements engineering: a roadmap. In: The future of software engineering. ACM, pp 35–46

  48. Oriol M, Stade M, Fotrousi F, Nadal S, Varga J, Seyff N, Abellò A, Franch X, Marco J, Schmidt O (2018) Fame: supporting continuous requirements elicitation by combining user feedback and monitoring. In: 26th IEEE international requirements engineering conference (RE’18). IEEE, pp 217–227

  49. Osgood CE, Suci GJ, Tannenbaum PH (1964) The measurement of meaning. University of Illinois Press, Urbana

    Google Scholar 

  50. Paolacci G, Chandler J, Ipeirotis PG (2010) Running experiments on Amazon Mechanical Turk. Judgm Decis Mak 5(5):411–419

    Google Scholar 

  51. Poushter J (2016) Smartphone ownership and internet usage continues to climb in emerging economies. Pew Research Center, February 22, 2016

  52. Regnell B, Beremark P, Eklundh O (1998) A market-driven requirements engineering process: results from an industrial process improvement programme. Requir Eng 3(2):121–129

    Article  Google Scholar 

  53. Rosen DE, Purinton E (2004) Website design: viewing the web as a cognitive landscape. J Bus Res 57(7):787–794

    Article  Google Scholar 

  54. Ryan RM, Deci EL (2000) Intrinsic and extrinsic motivations: classic definitions and new directions. Contemp Educ Psychol 25(1):54–67

    Article  Google Scholar 

  55. Seyff N, Todoran I, Caluser K, Singer L, Glinz M (2015) Using popular social network sites to support requirements elicitation, prioritization and negotiation. J Internet Serv Appl 6(7):1–16

    Google Scholar 

  56. Shaheen R, Ahsan A, Anwar Z (2018) Requirements management for market driven software products—key issues. In: 2018 International conference on computing, mathematics and engineering technologies (iCoMET), pp 1–6

  57. Snijders R, Dalpiaz F, Brinkkemper S, Hosseini M, Ali R, Ozum A (2015) REfine: a gamified platform for participatory requirements engineering. In: 1st International workshop on crowd-based requirements engineering. IEEE, pp 1–6

  58. Tondello GF, Wehbe RR, Diamond L, Busch M, Marczewski A, Nacke LE (2016) The gamification user types hexad scale. In: Annual symposium on computer–human interaction in play. ACM, pp 229–243

  59. Valtysson B (2012) Facebook as a digital public sphere: processes of colonization and emancipation. tripleC: Commun Capital Crit Open Access J Glob Sustain Inf Soc 10(1):77–91

    Article  Google Scholar 

  60. Wiegers K (1999) First things first: prioritizing requirements. Softw Dev 7(9):48–53

    Google Scholar 

  61. Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A (2012) Experimentation in software engineering. Springer, Berlin

    Book  Google Scholar 

  62. Yang D, Wu D, Koolmanojwong S, Brown AW, Boehm BW (2008) WikiWinWin: a wiki based system for collaborative requirements negotiation. In: 41st Annual Hawaii international conference on system sciences. IEEE, pp 24–24

  63. Yang J, Adamic LA, Ackerman MS (2008) Crowdsourcing and knowledge sharing: strategic user behavior on Taskcn. In: Proceedings of the 9th ACM conference on electronic commerce. ACM, pp 246–255

  64. Zichermann G, Cunningham C (2011) Gamification by design: implementing game mechanics in web and mobile apps. O’Reilly Media Inc

  65. Zowghi D, Coulin C (2005) Requirements elicitation: a survey of techniques, approaches, and tools. In: Engineering and managing software requirements. Springer, Berlin, pp 19–46

Download references

Acknowledgements

We thank all participants of the two studies for their voluntary engagement, Empa for the opportunity to conduct these studies in the scope of one of their projects and Irina Koitz for her feedback and discussions.

Author information

Authors and Affiliations

  1. Department of Informatics, University of Zurich, Binzmühlestrasse 14, 8050, Zurich, Switzerland

    Martina Z. Kolpondinos & Martin Glinz

Authors
  1. Martina Z. Kolpondinos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Glinz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martina Z. Kolpondinos.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kolpondinos, M.Z., Glinz, M. GARUSO: a gamification approach for involving stakeholders outside organizational reach in requirements engineering. Requirements Eng 25, 185–212 (2020). https://doi.org/10.1007/s00766-019-00314-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00766-019-00314-z

Keywords

Search

Navigation