Top

Empirical Software Engineering

Published in:

01-12-2015

Software requirements prioritization and selection using linguistic tools and constraint solvers—a controlled experiment

Authors: Jason McZara, Shahryar Sarkani, Thomas Holzer, Timothy Eveleigh

Published in: Empirical Software Engineering | Issue 6/2015

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Implementing the entire set of requirements for a software system is often not feasible owing to time and resource limitations. A key driver for successful delivery of any software system is the ability to prioritize the large number of requirements. Prioritization of requirements is a key challenge because current methods are not scalable to handle a realistic number of requirements. Current methods for requirements prioritization in market-driven software development projects are neither sufficient nor proven. A prioritization technique that is more time-efficient, accurate, and easier to implement for large-scale projects than current practices is needed. We address these challenges with a prioritization method that incorporates the use of a linguistic tool and constraint solver. In this paper we propose a method, referred to as SNIPR, for requirements prioritization and selection based on natural language processing and satisfiability modulo theories solvers. We present a controlled experiment in which 40 systems engineers prioritized and selected 20 requirements from a list of 100 using SNIPR and the weighted sum model. Results show that the SNIPR method consumes less time, improves selection accuracy, and is easier to perform than the weighted sum model. These results motivate further research using linguistic tools and constraint solvers for the prioritization of large sets of requirements.

previous article Link analysis algorithms for static concept location: an empirical assessment

next article Optimising agile development practices for the maintenance operation: nine heuristics

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

Aasem, M., Ramzan, M., & Jaffar, A (2010) Analysis and optimization of software requirements prioritization techniques. International conference on information and emerging technologies (ICETET), IEEE Computer Society, 1–6

Abran, A., Moore, J., & Dupuis, R (2001) SWEBOK Guide to the software engineering body of knowledge, IEEE. Ding, W. and Marchionini, G. 1997. A study on video browsing strategies. Technical report. University of Maryland at College Park

Afshari A, Mojahed M, Yusuff RM (2010) Simple additive weighting approach to personnel selection problem. Int J Innov Manag Technol 1(5):511–515

Ahl, A (2005) An experimental comparison of five prioritization techniques - investigating ease of use, accuracy, and scalability. Master Thesis No. MSE-2005-11, School of Engineering, Blekinge Institute of Technology

Aho AV, Hopcroft JE, Ullman JD (1983) Data structures and algorithms. Addison-Wesley, ReadingMATH

Aurum A, Wohlin C (2003) The fundamental nature of requirements engineering activities as a decision-making process. Inf Softw Technol 45(14):945–954CrossRef

Avesani, P., Bazzanella, C., Perini, A., & Susi, A (2004) Supporting the requirements prioritization process. A machine learning approach, in proceedings of 16th international conference on software engineering and knowledge engineering (SEKE 2004), pages 306–311, Banff, Alberta, Canada, June 2004. KSI press

Avesani, P., Bazzanella, C., Perini, A., & Susi, A (2005) Facing scalability issues in requirements prioritization with machine learning techniques. In proceedings of the 13th IEEE international requirements engineering conference, IEEE computer society, Paris, 297–305

Babar, I. B., Ramzan, M., & Ghayyur, S. A. K (2011) Challenges and future trends in software requirements prioritization. International conference on computer networks and information technology 2011, 319–324

Bagchi P, Rao RP (1992) Decision making in mergers: an application of the analytic hierarchy process. Manag Decis Econ 13(2):91–99CrossRef

Barney S, Aurum A, Wohlin C (2008) A product management challenge: creating software product value through requirements selection. J Syst Archit 54:576–593CrossRef

Beg, M., Abbas, Q., & Verma, R (2008) An approach for requirement prioritization using B-Tree. First international conference on emerging trends in engineering and technology (ICITET), IEEE computer society, 1216–1221

Berander, P (2004) Using students as subjects in requirements prioritization. proceedings of the 2004 international symposium on empirical software engineering (ISESE’04). IEEE Computer Society, 167–176

Berander P, Andrews A (2005) Requirements prioritization. In: Aurum A, Wohlin C (eds) Engineering and managing software requirements. Springer Verlag, Berlin, pp 69–94CrossRef

Berander P, Svahnberg M (2009) Evaluating two ways of calculating priorities in requirements hierarchies - an experiment on hierarchical cumulative voting. J Syst Softw 82(5):836–850CrossRef

Berander, P., Khan, K. A., & Lehtola, L (2006) Towards a research framework on requirements prioritization. proceedings of the sixth conference on software engineering research and practice in Sweden, 39–48

Bjarnason E, Wnuk K, Regnell B (2012) Are you biting off more than you can chew? A case study on causes and effects of overscoping in large-scale software engineering. J Inf Softw Technol 54:1107–1124CrossRef

Carlshamre, P., Sandahl, K., Lindvall, M., Regnell, B., & Natt och Dag, J (2001) An industrial survey of requirements interdependencies in software product release planning. Proceedings of the fifth IEE international symposium on requirements engineering, 84–91

Danesh, A., Mortazavi, S., & Danesh, S (2009) Requirements prioritization in on-line banking systems: using value-oriented framework. International conference on computer technology and development (ICCT), IEEE computer society, 158–161

Daneva, M., & Herrmann, A (2008) Requirements prioritization based on benefit and cost prediction: a method classification framework. Press 34th Euromicro conference software engineering and advanced applications, IEEE computer society, 240–247

Davis AM (2003) The art of requirements triage. IEEE Compute 36(3):42–49CrossRef

Durillo, J., Zhang Y., Alba, E., & Nebro, A (2009) A study of the multi-objective next release problem. IEEE computer society, 49–58

Dutertre, B., & De Moura, L (2006) The Yices SMT solver. Web. http://yices.csl.sri.com/tool-paper.pdf. Accessed 5 May 2013

Falessi, D., Cantone, G., & Canfora, G (2010) A comprehensive characterization of NLP techniques for identifying equivalent requirements. Proceedings of the 2010 ACM-IEEE international symposium on empirical software engineering and measurement

Firesmith D (2004) Prioritizing requirements. J Object Technol 3(8):35–47CrossRef

Greer, D., & Ruhe, G. (2003). Software release planning: an evolutionary and iterative approach. Information and software technology, pp 243–253

Greer D, Bustard D, Sunazuka T (1999a) Effecting and measuring risk reduction in software development. NEC J Res Dev 40(3):378–438

Greer, D., Bustard, D. W., & Sunazuka, T (1999b) Prioritisation of system changes using cost-benefit and risk assessments. Proceedings IEEE international symposium on requirements engineering, 180–187

Hatton, S. (2008). Choosing the right prioritisation method. Proceedings of the 19th Australian conference on software engineering, 517–526

Herrmann A, Paech B (2009) Practical challenges of requirements prioritization based on risk estimation. Empir Softw Eng 14:644–674CrossRef

Holbrook EA, Hayes JH, Dekhtyar A, Li W (2013) A study of methods for textual satisfaction assessment. Empir Softw Eng 18:139–176CrossRef

Höst, M., Wohlin, C., & Thelin, T (2005) Experimental context classification: incentives and experience of subjects. Proceedings of the 27^th international conference on software engineering (ICSE), 470–478

Jantunen S, Lehtola L, Gause DC, Dumdum UR, Barnes RJ (2011) The challenge of release planning. In Software product management (IWSPM), 2011 Fifth International workshop. IEEE, pp 36–45

Jung, H. W (1998) Optimizing value and cost in requirements analysis. IEEE Software, July/August, 74–78

Karatzas K, Dioudi E, Moussiopoulos N (2003) Identification of major components for integrated urban air quality management and information systems via user requirements prioritisation. Environ Model Softw 18(2):173–178CrossRef

Karlsson, J (1996) Software requirements prioritizing. Proceedings of the 2nd IEEE international conference on requirements engineering, 110–116

Karlsson, J., & Ryan, K (1996) Supporting the selection of software requirements. Proceedings of the 8th international workshop on software specification and design, 146–149

Karlsson J, Ryan K (1997) A cost-value approach for prioritizing requirements. IEEE Softw 14(5):67–74CrossRef

Karlsson J, Olsson S, Ryan K (1997) Improved practical support for large-scale requirements prioritising. Requir Eng 2:51–60CrossRef

Karlsson J, Wohlin C, Regnell B (1998) An evaluation of methods for prioritizing software requirements. Inf Softw Technol 39:939–947CrossRef

Karlsson, L., Dahlstedt, A. G., Dag. J. N., Regnell, B., & Persson, A (2003) Challenges in market-driven requirements engineering - an industrial interview study. Proceedings of the eighth international workshop on requirements engineering: foundation for software quality, 101–112

Karlsson, L., Berander, P., Regnell, B., & Wohlin, C (2004) Requirements prioritisation: an experiment on exhaustive pair-wise comparisons versus planning game partitioning. Proceedings of the 8^th international conference on empirical assessment in software engineering, 145–154

Karlsson L, Thelin T, Regnell B, Berander P, Wohlin C (2007) Pair-wise comparisons versus planning game partitioning-experiments on requirements prioritisation techniques. Empir Softw Eng 12(1):3–33CrossRef

Khan, K. A (2006) A systematic review of software requirements prioritization (Unpublished MS Thesis). School of Engineering, Blekinge Institute of Technology

Khurum M, Gorschek T, Angelis L, Feldt R (2009) A controlled experiment of a method for early requirements triage utilizing product strategies. Requirements engineering: foundation for software quality. Lect Notes Comput Sci 5512:22–36CrossRef

Laurent, P., Cleland-Huang, J., & Duan, C (2007) Towards automated requirements triage. 15th IEEE international requirements engineering conference, 131–140

Leffingwell D, Widrig D (2000) Managing software requirements—a unified approach. Addison-Wesley, Upper Saddle River

Lehtola L, Kauppinen M (2006) Suitability of requirements prioritization methods for market-driven software product development. Softw Process Improv Pract 11(1):7–19CrossRef

Li C, Akker M, Brinkkemper S, Diepen G (2010) An integrated approach for requirement selection and scheduling in software release planning. Requir Eng 15:375–396CrossRef

Lubars, M., Potts, C., & Richter, C (1993) A review of the state of the practice in requirements modeling. Proceedings of the IEEE international symposium of requirements engineering, 2–14

Ma, Q (2009) The effectiveness of requirements prioritization techniques for a medium to large number of requirements: a systematic literature review. school of computing and mathematical sciences, Auckland University of Technology

Mack, N., Woodsong, C., MacQueen, K. M., Guest, G., & Namey, E (2005) Qualitative research methods: a data collector’s field guide. Family health international (pp. 6–9).

Natt och Dag J, Gervasi V, Brinkkemper S, Regnell B (2005) A linguistic approach to large-scale requirements management. IEEE Softw 22(1):32–39CrossRef

Natt och Dag J, Regnell B, Carlshamre P, Anderson M, Karlsson J (2002) A feasibility study of automated natural language requirements analysis in market-driven development. Requir Eng 7(1):20–33CrossRefMATH

Natt och Dag J, Thelin T, Regnell B (2006) An experiment on linguistic tool support for consolidation of requirements from multiple sources in market-driven product development. Empir Softw Eng 11:303–329CrossRef

Ngo-The A, Ruhe G (2005) Decision support in requirements engineering. In: Aurum A, Wohlin C (eds) Engineering and managing software requirements. Springer Verlag, Berlin, pp 267–286CrossRef

Palma, F., Susi, A., Tonella, P (2011) Using an SMT solver for interactive requirements prioritization. Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on foundations of software engineering ACM, 48–58

Perini, A., Ricca, F., Susi, A., & Bazzanella, C (2007) An empirical study to compare the accuracy of AHP and CBRanking techniques for requirements prioritization. Fifth international workshops on comparative evaluation in requirements engineering, 23–35

Port, D., Olkov, A., & Menzie, T (2008) Using simulation to investigate requirements prioritization strategies. Proceedings of the 23rd IEEE/ACM international conference on automated software engineering, 268–277

Racheva, Z., Daneva, M., & Buglione, L (2008) Supporting the dynamic reprioritization of requirements in agile development of software products. Second international workshop on software product management, 49–58

Racheva, Z., Daneva, M., Herrmann, A., & Wieringa, R.J. (2010). A conceptual model and process for client-driven agile requirements prioritization. Fourth international conference on research challenges in information science (RCIS), 287–298

Regnell B, Beremark P, Eklundh O (1998) A market-driven requirements engineering process: results from an industrial process improvement programme. Requir Eng 3(2):121–129CrossRef

Regnell B, Runeson P, Thelin T (2000) Are the perspectives really different? – Further experimentation on scenario-based reading of requirements. Empir Softw Eng 5:331–356CrossRefMATH

Regnell B, Höst M, NattochDag J, Beremark P, Hjelm T (2001) An industrial case study on distributed prioritisation in market-driven requirements engineering for packaged software. Requir Eng 6(1):51–62CrossRefMATH

Regnell, B., Karlsson, L., & Host, M (2003) An analytical model for requirements selection quality evaluation in product software development. Proceedings of the 11th IEEE international requirements engineering conference, 254–263

Regnell, B., Svensson, R., & Wnuk, K (2008a) Can we beat the complexity of very large-scale requirements engineering? Springer-Verlag Berlin Heidelberg, 123–128

Regnell B, Svensson RB, Wnuk K (2008b) Requirements engineering: foundation for software quality. Lect Notes Comput Sci 5025(2008):123–128CrossRef

Reifer DJ (2002) How good are agile methods? IEEE Softw 19(4):16–18CrossRef

Ruhe, G., & Greer, D (2003) Quantitative studies in software release planning under risk and resource constraints. Empirical software engineering, 262–270

Ruhe G, Saliu MO (2005) The art and science of software release planning. Softw IEEE 22(6):47–53CrossRef

Ruhe, G., Eberlein, A., & Pfahl, D (2002) Quantitative WinWin: a new method for decision support in requirements negotiation. Proceedings of the 14^th international conference on software engineering and knowledge engineering, 159–166

Saaty TL (1980) The analytic hierarchy process. McGraw-Hill, New YorkMATH

Saliu OM (2009) Release planning of software systems. VDM Verlag, Saarbrücken

Sjøberg DI, Hannay JE, Hansen O, Kampenes VB, Karahasanovic A, Liborg NK, Rekdal AC (2005) A survey of controlled experiments in software engineering. IEEE Trans Softw Eng 31(9):733–753CrossRef

Sommerville I, Sawyer P (1997) Requirements engineering - a good practice guide. Wiley, ChichesterMATH

Svahnberg M, Goreschek T, Feldt R, Torkar R, Saleem SB, Shafique MU (2010) A systematic review on strategic release planning models. Inf Softw Technol 52:237–248CrossRef

Svensson, R.B., Gorschek, T., Regnell, B., Torkar, R., Shahrokni, A., Feldt, R., & Aurum, A (2011) Prioritization of quality requirements: state of practice in eleven companies. proceedings of the requirements engineering conference (RE), 2011 19th IEEE International, 69–78

Tonella, P., Susi, A., & Palma, F (2010) Using Interactive GA for requirements prioritization. 2nd international symposium on search based software engineering, IEEE Computer society, 57–66

Tonella, P., Susi, A., & Palma, F (2012) Interactive requirements prioritization using a genetic algorithm. Information and software technology, 57–66

Tonella P, Susi A, Palma F (2013) Interactive requirements prioritization using a genetic algorithm. Inf Softw Technol 55:173–187CrossRef

Triantaphyllou E (2000) Multi-criteria decision making methods. Appl Optim 44:5–21CrossRef

Triantaphyllou, E., & Baig, K (2005) The impact of aggregating benefit and cost criteria in four MCDA methods. IEEE transactions on engineering management, 213–226

van den Akker, M., Brinkkemper, S., Diepen, G., & Versendaal, J (2005) Determination of the next release of a software product: an approach using integer linear programming. Proceedings from REFSQ’05: The 11th International workshop on requirements engineering: Foundation for software quality, 119–124

van den Akker M, Brinkkemper S, Diepen G, Versendaal J (2008) Software product release planning through optimization and what-if analysis. Inf Softw Technol 50:101–111CrossRef

Wnuk K, Regnell B, Schrewelius C (2009) Architecting and coordinating thousands of requirements – an industrial case study. Requirements engineering: foundation for software quality. Lect Notes Comput Sci 5512:118–123CrossRef

Wnuk K, Regnell B, Berenbach B (2011) Scaling up Requirements Engineering – Exploring the Challenges of Increasing Size and Complexity in Market-Driven Software Development. Requir Eng Found Softw Qual Lect Notes Comput Sci 6606:54–59CrossRef

Yap B, Sim H (2011) Comparisons of various types of normality tests. J Stat Comput Simul 81(12):2141–2155MathSciNetCrossRefMATH

Title: Software requirements prioritization and selection using linguistic tools and constraint solvers—a controlled experiment
Authors: Jason McZara
Shahryar Sarkani
Thomas Holzer
Timothy Eveleigh
Publication date: 01-12-2015
Publisher: Springer US
Published in: Empirical Software Engineering / Issue 6/2015
Print ISSN: 1382-3256
Electronic ISSN: 1573-7616
DOI: https://doi.org/10.1007/s10664-014-9334-8

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 6/2015

Towards a decision-making structure for selecting a research design in empirical software engineering

How product owner teams scale agile methods to large distributed enterprises

Optimising agile development practices for the maintenance operation: nine heuristics

Link analysis algorithms for static concept location: an empirical assessment

Semi-automatic selection of primary studies in systematic literature reviews: is it reasonable?

Recommending reference API documentation

Premium Partner