Agile Software Development (ASD) is becoming more popular in all fields of industry. For an agile transformation, organizations need to continuously improve their established approaches to Requirements Engineering (RE) as well as their approaches to software development. This is accompanied by some challenges in terms of agile RE. The main objective of this paper is to identify the most important challenges in agile RE industry has to face today. Therefore, we conducted an iterative expert judgement process with 26 experts in the field of ASD, comprising three complementary rounds.
In sum, we identified 20 challenges in three rounds. Six of these challenges are defined as key challenges. Based on the results, we provide options for dealing with those key challenges by means of agile techniques and tools. The results show that the identified challenges are often not limited to ASD, but they rather refer to software development in general. Therefore, we can conclude that organizations still struggle with agile transition and understanding agile values, in particular, in terms of stakeholder and user involvement.
1 Introduction
Agile Software development (ASD) gains in popularity in today’s business world due to enabling immediately changes in the direction of product development. These short-term changes in direction require a flexible approach to Requirements Engineering (RE) as well. In addition, agile methodologies (such as Scrum [1], Kanban [2] or Extreme Programming [3]) are often combined with Human-Centered Design (HCD) [4] activities in order to emphasize a value-driven approach to product development [5, 6]. To this end, the field of agile RE has emerged during the last decade.
Focusing on user needs and value delivery becomes an important aspect in product development due to the increasing competition in all areas. With regard to ASD, plan-driven organizations moved away to value-driven organizations. On the one hand, people in plan-driven organizations often negotiate about project plans, pricing models and the amount of features they can develop with the available resources. They are emphasizing the generated outputs such as number of created features during a time period. On the other hand, people in value-driven organizations discuss visions, experiences and human values as well as the way to address them through the product. They focus on the outcomes that the delivered outputs entail.
Anzeige
Compared to sequential approaches to RE, which comprise a requirement analysis phase before the development can even begin, agile RE is carried out along with the development itself. Therefore, continuous management of requirements is a crucial attribute. Requirements are regularly described from a user perspective in the form of epics and user stories [7] instead of creating a requirements document [8]. Recent research is showing that there are several ways of running RE in an agile environment while involving users and stakeholders [5, 9‐12].
Performing agile RE can lead to challenges organizations have to deal with. In literature, there can be found some studies investigating challenges in agile RE (see [11‐15]). However, the related work still lacks in giving a general overview of the challenges in current industry.
This study pursues the main objective of identifying the most important challenges in agile RE industry has to address today. We aim to build a shared understanding concerning these challenges among voices that matter by means of experts in the field of agile RE. Thus, the research questions we pose are listed below:
RQ1: What are the key challenges in Agile Requirements Engineering?
RQ2: How can we deal with the identified key challenges?
The paper is structured as follows: Sect. 2 briefly summarizes the related work and points out the research gap. Section 3 presents the applied research method and describes the iterative expert judgement process. Then, Sect. 4 identifies the findings and discusses both on their meaning and on the limitations of this study. Finally, Sect. 5 provides the conclusions as well as an outlook on future research.
Anzeige
2 Related Work
There are related studies in the literature that investigate challenges in agile RE by means of different research methods. Table 1 shows an overview of the reported challenges and used research methods.
Problems with cost and schedule estimation; inadequate or inappropriate architecture; neglect of non-functional requirements; customer access and participation; prioritization on a single dimension; inadequate requirements verification; minimal documentation
Minimal documentation; customer availability; inappropriate architecture; budget and time estimation; neglecting non-functional requirements (NFRs); customer inability and agreement; contractual limitations; requirements change and its evaluation
Problems with client or customer representatives; insufficiency of user story format; difficulties in prioritization of requirements; growing technical debt; reliance on tacit requirements knowledge; imprecise effort estimates
Requirement prioritization; non-functional requirements identification; lack of information; volatility of requirements; requirements definition; dependence among requirements; prediction of impacts of changes; user dependence; communication and collaboration with users; requirements validation
Analyzing the related work, we can state that the authors use two different kinds of research approaches in general. On the one hand, Ramesh et al. [13] and Bjarnason et al. [14] utilize case studies to investigate the challenges in the field. On the other hand, Inayat et al. [11], Heikkila et al. [15] and Soares et al. [12] report challenges in agile RE by analyzing primary studies with the aim to identify available evidence in existing research.
Ramesh et al. [13] results were published in 2010. However, as ASD is a rapidly changing research area and the body of knowledge has evolved over the last years, we need to clarify whether the reported challenges are still relevant today. For instance, NFRs may not be longer a challenge for industry since the concept of the Definition of Done and the usage of acceptance criteria are widely spread. Bjarnason et al. [14] carry out a case study in only one company, therefore the results may not be applicable to other companies and may not be representative in general. In comparison, Inayat et al. [11], Heikkila et al. [15] and Soares et al. [12] review primary studies by analyzing existing literature, which is a good approach to get an impression of relevant aspects from a theoretical viewpoint. Nevertheless, one could argue that this is not an appropriate approach to investigate the existing challenges in practice.
To this end, the aim of this study is to identify the most important challenges in agile RE industry has to face up today by getting insights from 26 experts in the field. To the best of our knowledge there is no existing study investigating these challenges by means of a qualitative study with practicing experts in ASD working for many different companies.
3 Research Method
We used an iterative expert judgement process rooted in a Delphi study [16‐18] in order to respond to our RQs. We applied a modified Delphi study where measuring consensus and stability at group level among several iterations was not the most crucial part. On the contrary, we shifted the focus to applying the valuable features of Delphi for conducting our iterative expert judgement process [19]:
Anonymity among experts to avoid influence of dominant individuals
Iterative approach
Controlled feedback with statistical group response
The main benefit of our modified approach was utilizing the learnings from a previous iteration for carrying out the following ones.
3.1 General Study Design
The study was performed in three complementary rounds. Figure 1 gives a general overview of the process. At the beginning of each round, we started designing the questionnaire, optimized by a pretest. Once finished, the invitation was sent to the experts via email. In the second and third round, we attached the results of the previous rounds to the invitation in order to share the outcomes among the panel. The experts had two weeks to fill in the questionnaire. During the following two weeks we evaluated the results, created the report, specified the criteria for dropping items for the following round and designed the questionnaire for the next round.
Fig. 1.
General process of study
×
We conducted the study in German since most of the experts are native speaker. Since we are aware that the term agile RE is not very accepted in the agile community and some experts understand this as a contradiction in itself, we decided not to ask for challenges in agile RE directly. On the contrary, we phrased our questions differently and described the context of our study within the introduction part of each questionnaire.
We used google forms for the first and second round, whereas limesurvey was used for the third round due to the complexity of the questionnaire. In general, we decided to use 7-point Likert items since this has been proven to be the best choice in terms of avoiding interpolations within related research fields [20]. Besides, we adapted the quality criteria proposed by Diamond et al. [17] so as to ensure the quality of our study.
3.2 Panel of Experts
We selected our panelists specifically for their knowledge or position regarding the issue under study. As shown in previous work, the research field of agile RE is very close to existing work practices in industry [5]. To this end, we defined the reproducible criteria for selecting participants as follows:
Many years of experience as professional in the field of ASD
Working experience in one or more of the following roles: Product Owner, Scrum Master, Agile Coach, Consultant for Agile Transition, Kanban Expert or Lean Startup Expert
The panel consisted of 26 experts who are working in 19 different companies located in Germany and Switzerland. In general, they had 2–10 years of experience working in ASD (average = 6.14 years). In comparison, experts have about 0–16 years of experience with RE (average = 6.65 years). Even though one expert stated that he had no experience with RE at all, we decided to include his answers into the study, since he has long experience in ASD and in general there do not exist a specific role of a requirements engineer.
Figure 2 shows the kind of process models experts have been working with. It is worth mentioning that most of the experts have experience both with sequential approaches and with agile approaches.
Fig. 2.
Process models used by experts
×
In addition, Table 2 displays the know-how level in terms of ASD rated by experts themselves.
Table 2.
Know-how of panel in terms of ASD
know-how very poor
1
2
3
4
5
know-how very high
0.0%
0.0%
15.4%
69.2%
15.4%
3.3 Round 1
The questionnaire of the first round comprised two open questions, repeated 15 times. On the one hand, the experts were asked what the most important challenges with requirements in terms of ASD were. On the other hand, they should give a statement for each challenge to clarify why they considered this challenge as important. The minimum number of required answers was 3, whereas the maximum was 15. In sum, we received 107 answers (items) from 26 experts. Table 3 shows an example of an item consisting of a challenge and a statement concerning importance. The full results can be found in [21].
Table 3.
Exemplary item in round 1
Question round 1
Answer given by expert
What challenge do you perceive with requirements in terms of Agile Software Development?
Stakeholders affected by requirements or changing the system are not involved
Why do you consider this challenge as important?
In one of my projects, representatives of end users did not really knew the pain of end users. Even the early UI prototypes were tested by incorrect stakeholders, which led to risks of conflicts and failure
With respect to data analysis, each challenge was categorized by the authors during a workshop. Those items, which could not be categorized easily, were discussed within the group of authors. We used the following categories: stakeholder and user involvement, collaboration within the team, vision and big picture, iteration planning and estimation, granularity of requirements, dependencies of requirements, understanding agile and agile values, continuous delivery of value, roles and responsibilities, need for security, requirement validation, RE methods, format of requirements, clarity of requirements, prioritization, refinement, discovery and transparency.
Additionally, the reported challenges were categorized according to their agile RE activity (see Table 4).
Table 4.
Agile RE activities
Agile RE activity
Description
Discovery
Eliciting new ideas/requirements
Refinement
Clarifying and analyzing new ideas/requirements
Prioritization
Measuring the value that the development will add to the product
Review
Checking if requirement is implemented in the manner to deliver value
Documentation
Capturing discussion and decisions around the requirement
3.4 Round 2
We checked each item of round one critically, whether or not it was appropriate for answering our RQs and being queried in the next round. Thus, items of round 1 were consolidated or excluded. In the end, we identified 34 items as relevant for assessing them in round 2. Based on those items, we created the questionnaire for the second round. The resulting questionnaire assessed 34 items related to the following topics: stakeholder and user involvement (6 items), understanding agile and agile values (6 items), RE methods (10 items), iteration planning and estimation (6 items) and format of requirements (6 items).
The experts rated each item using 7-point Likert items (see Fig. 3). Moreover, they could choose giving no statement. To sum up, we received responses from 23 experts. For each item we calculated mean, variance and standard deviation. Additionally, we created a diagram showing the distribution of experts’ opinion (see Fig. 3) and discussed on the meaning of findings. The results of round two can be found in [22].
Fig. 3.
Exemplary item of round 2
×
3.5 Round 3
We reduced the number of items when designing the questionnaire for the third round. Considering items from round 2, we assessed each item according to (a) its relevance in terms of our RQs, (b) the importance in terms of the attributes of agile RE, (c) the opinion of the experts and the comprehensibility of the items.
The final questionnaire comprised two parts. The first part queried in sum 20 potential key challenges of agile RE (see Appendix). The experts were asked to rate each item, whether or not it is a challenge in agile RE. Moreover, they had the option to choose giving no statement. Then, the second part evaluated those items that experts identified as challenge in terms of importance, following 7-point Likert items (totally important, important, rather important, neutral, rather unimportant, unimportant, totally unimportant, no statement). In addition, experts optionally had the chance to provide a solution for solving the challenge.
In sum, 22 experts filled in the questionnaire. We classified each of the 20 items as challenge in Agile RE since we derived all items from the results of the previous rounds. Besides, we calculated the number of experts who rated each item as a challenge. Then, we defined challenges as key in those cases where 2/3 of the experts’ answers were: “Yes, it is a challenge”. Finally, we calculated the importance for those items. The results of round 3 can be found in [23].
4 Results and Discussion
Summarizing the results of the three complementary rounds, we derived 20 challenges that companies have to cope with in terms of agile RE (see Appendix). We categorized such challenges into stakeholder and user (3 items), requirements management (7 items), methods and artifacts (5 items) and format of requirements (5 items).
4.1 (RQ1) What Are the Key Challenges in Agile Requirements Engineering?
We identified six key challenges industry has to face today in terms of agile RE (see Table 5). In general experts weighted the identified challenges as important [23] and none of them rated one of the six key challenges as unimportant.
Table 5.
Key challenges in agile RE
ID
Key challenge
N
Yes
No
C1
In agile software development functional or technical dependencies with other teams are a challenge because a considerable coordination effort is required
17
14 (82.4%)
3 (17.6%)
C2
In agile software development it is a challenge that stakeholders understand that the development team can make independent (detailed) decisions
20
15 (75.0%)
5 (25.0%)
C3
In agile software development it is a challenge not to lose sight of the big picture during the implementation of complex requirements
20
15 (75.0%)
5 (25.0%)
C4
In agile software development continuous management of requirements is a challenge since not all of them are fixed at the beginning and they may change over the course of the project
22
16 (72.7%)
6 (27.3%)
C5
In agile software development it is a challenge to work out user requirements and quality of use in cooperation with direct users (end users) of the product
18
13 (72.2%)
5 (27.8%)
C6
In agile software development it is a challenge to involve stakeholders throughout the whole development process in regular iterations, so that product development will succeed
20
14 (70.0%)
6 (30.0%)
All challenges related to the category stakeholder and user are classified as key challenges (C2, C5, C6). Therefore, we can conclude that organizations still struggle to the agile transition. Evolving an agile mindset within a whole organization even in parts that are not close to development is still a challenge companies have to address.
Typically, agile transformation starts in development-oriented parts of an organization. Transforming an organization to become more agile implies a change within the whole organization. The results show that there is a gap between knowledge and understanding agile values [24] within organizations. Development-oriented techniques evolve rapidly. In comparison, there are still challenges involving stakeholders and users into the agile processes (C2, C5, C6).
Two challenges (C1, C4), related to category requirements management, are key in agile RE. On the one hand, companies have an issue with the continuous management of requirements. On the other hand, they have a problem with technical or functional dependencies due to raising effort in coordination. Besides, one challenge of methods and artifacts (C3) is a key challenge.
ASD is commonly used in environments where people have to solve complex adaptive problems [25]. Concerning C1, C3, and C4 we can state that there are still challenges to be solved, due to the complexity of problems, which are not addressed by agile techniques properly. To this end, existing techniques and methods must be adapted or new techniques need to be found.
Figure 4 offers an overview of the categorized key challenges.
Fig. 4.
Categorized key challenges in agile RE
×
4.2 (RQ2) How Can We Deal with the Identified Key Challenges?
Experts recommend techniques, methods and tools in order to deal with the challenges in agile RE. Below, we will list the techniques and methods proposed by the panel for each key challenge.
C1: In agile software development functional or technical dependencies with other teams are a challenge because a considerable coordination effort is required.
More than three experts recommended using scaled frameworks such as LeSS, SAFe or Scrum of Scrum. Moreover, they proposed the use of the following techniques: creating a common understanding among all, enhancing continuous communication and collaboration, training the ability to solve dependencies, holding weekly coordination meetings, organizing teams in matrix management, building communities of practices for transcending topics, release planning (SAFe), team-transcending availability of product und sprint backlogs, involving temporary representatives in other teams, enforcing continuous integration, improving API-driven development and microservices.
C2: In agile software development it is a challenge that stakeholders understand that the development team can make independent (detailed) decisions.
The following techniques were suggested: continuous coordination and presenting possible solutions to stakeholder, providing transparency about rationales of the decisions, strengthening product owner with competency in decision making and helping stakeholders become aware of the consequences of interfering into detailed decisions.
More than three experts recommended providing alternative solutions for one requirement. In addition, it is useful to demonstrate that the recommended solution of a stakeholder is an alternative out of many. In previous rounds, more than one expert stated that product owner and stakeholder altogether decide what to be developed. In contrast, the development team decides how the requirement should be developed.
C3: In agile software development it is a challenge not to lose sight of the big picture during the implementation of complex requirements.
The following techniques were recommended: creating a shared understanding regarding the meaning of the big picture by means of a product vision, defining epics or subgoals in the beginning, managing the big picture as a responsibility of the product owner, providing transparency concerning changes among all, understanding connections among user stories by means of story mapping, visualizing customer journey in the beginning, involving users continuously in order to focus on the problem to be solved and identifying central contact person for related topics to enable rapid coordination. Moreover, the experts advised to use visualization by means of roadmaps, sketches of the system and processes, and value streams.
C4: In agile software development continuous management of requirements is a challenge since not all of them are fixed at the beginning and they may change over the course of the project.
The experts proposed the following techniques, methods and tools: collaborating closely with the requesting stakeholder, communicating regularly within the team, refining and prioritizing continuously the product backlog, grooming on demand (Kanban), describing in detail the requirements in the sprint backlog, reviewing the results regularly, discussing the maturity level of a requirement with the team, grouping user stories to epics, using Kano analysis, screening and scoring the theme, weighting relatively, utilizing spike stories to evaluate uncertainty in requirements and using ticketing tools (e.g. JIRA).
C5: In agile software development it is a challenge to work out user requirements and quality of use in cooperation with direct users (end users) of the product.
The experts recommended utilizing the following techniques: prototypes, interviews, observing users by the think aloud method, A/B testing, UX labs, analyzing usage behavior, friendly user tests, alpha/beta/silent launches, improving continuously a released version, utilizing a UX-board for play back user insights and testing hypotheses with real users. In addition, one expert suggested adapting user research to ASD by reducing the methods to the minimal, evaluate within the team without report creation, reduce financial restrictions for user involvement as well as problems of accessing real user by means of panels or a prior recruitment.
C6: In agile software development it is a challenge to involve stakeholders throughout the whole development process in regular iterations so that product development will succeed.
The following techniques were proposed: defining stakeholders and their involvement in regular iterations, proposing goals instead of prescribing solutions, involving all possible stakeholders in the beginning and reducing the amount of people over time.
More than eight experts suggested involving stakeholders by regular planning and review meetings to gather feedback and useful information. In light of this, they recommended clarifying the purpose of the meetings and the importance of the outcomes to be discussed beforehand.
4.3 Meaning of Findings
Comparing our findings to the identified challenges of the related work (see Table 1), we can conclude that 16 out of our challenges are not reported by the related studies.
Our key challenge C5 (user involvement) is reported by all related studies. In addition, three studies [11‐13] report issues with non-functional requirements, which is comparable to our challenge C13. There is also a relation between the key challenge C4 (continuous requirements management) and the challenge “requirements change and its evaluation” reported by [11]. Moreover, the key challenge C1 (technical or functional dependencies to other teams) is reported by [12] in a slightly different manner since they phrase it like “dependence between requirements”.
Moreover, the results show that the identified challenges are often not limited to ASD, but they rather refer to software development in general. Therefore, we can conclude that organizations still struggle with agile transition and understanding agile values, in particular, in terms of stakeholder and user involvement.
4.4 Limitations
We are aware that the design of a questionnaire is important for the process of data gathering. To this end, we made several pretests of each questionnaire we used with participants matching our criteria of expert selection. Nevertheless, we observed two experts struggling with the user experience of the questionnaire tool (Google Forms) used in round 1. Therefore, we decided to use another tool (LimeSurvey) for the questionnaire in round 3, which was more complex than the previous two.
To carry out the study, the group of authors created summaries of the results and made decisions concerning the kind of items they had to query in the following rounds. That may lead to bias in the opinion building process of the panel. We tried to prevent this point by being very accurate in terms of data analysis and by creating the reports. In addition, we selected items for the following rounds through the selection criteria defined earlier.
5 Conclusions and Future Work
This paper has addressed the identification of the most important challenges in agile RE industry has to face up today. Moreover, we examined how to deal with those challenges. For that purpose, we carried out an iterative expert judgement process comprising three complementary rounds. The learnings from previous iterations were used for carrying out the following ones. Our panel consisted of 26 experts in the field of ASD working for 19 different companies.
We have contributed to the body of knowledge of software development by identifying 20 challenges industry has to address at present in terms of agile RE. Six of these challenges have been defined as key challenges. In addition, we have analyzed options to deal with those key challenges by means of agile techniques recommended by the experts.
Future research may specifically identify challenges in agile RE by means of an international panel of experts, for instance with experts from Scandinavian countries. Our aim is to conduct a comparative analysis among the statements of German-speaking experts with the viewpoint of international experts. In addition, we are creating a tool that supports practitioners solving the identified challenges using agile techniques. Therefore, we are working on agile RE patterns. Some experts stated that the queried challenges are not limited to ASD. To this end, future studies may analyze whether the challenges appear in terms of RE in general.
Acknowledgements
First of all, we would like to thank all experts for their participation and sharing their valuable knowledge. Moreover, we would like to thank all participants in our pretests for their collaboration. This research has been supported by the MeGUS project (TIN2013-46928-C3-3-R), Pololas project (TIN2016-76956-C3-2-R) and by SoftPLM Network (TIN2015-71938-REDT) of the Spanish Ministry of Economy and Competitiveness.
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
In agile software development functional or technical dependencies with other teams are a challenge because a considerable coordination effort is required
17
14 (82.4%)
3 (17.6%)
C2
In agile software development it is a challenge that stakeholders understand that the development team can make independent (detailed) decisions
20
15 (75.0%)
5 (25.0%)
C3
In agile software development it is a challenge not to lose sight of the big picture during the implementation of complex requirements
20
15 (75.0%)
5 (25.0%)
C4
In agile software development continuous management of requirements is a challenge since not all of them are fixed at the beginning and they may change over the course of the project
22
16 (72.7%)
6 (27.3%)
C5
In agile software development it is a challenge to work out user requirements and quality of use in cooperation with direct users (end users) of the product
18
13 (72.2%)
5 (27.8%)
C6
In agile software development it is a challenge to involve stakeholders throughout the whole development process in regular iterations so that product development will succeed
20
14 (70.0%)
6 (30.0%)
C7
In agile software development it is a challenge that the requirements to be implemented are clearly defined from the development start since the priorities often change in the short term
21
13 (61.9%)
8 (38.1%)
C8
In agile software development it is a challenge to analyze requirements with regard to the past development in order to avoid side effects
15
9 (60.0%)
6 (40.0%)
C9
In agile software development it is a challenge to formulate requirements as objectives that describe the problem area so that the creativity in solution finding is not restricted
22
13 (59.0%)
9 (41.0%)
C10
In agile software development it is a challenge to slice requirements in such a way that they offer added value for the product
20
11 (55.0%)
9 (45.0%)
C11
In agile software development it is a challenge to justify the benefits of the requirements in order to make the added value of the implementation clear as well as decisions for a specific requirement comprehensible
21
11 (52.4%)
10 (47.6%)
C12
In agile software development it is a challenge to document changes to the requirements comprehensibly
18
9 (50.0%)
9 (50.0%)
C13
In agile software development it is a challenge to establish non-functional requirements
19
9 (47.4%)
10 (52.6%)
C14
In agile software development it is a challenge to focus only on the refinement of the requirements for the short-term iterations
22
10 (45.5%)
12 (54.5%)
C15
In agile software development it is a challenge to develop an outlook on the next iterations without making it a binding one
21
9 (42.9%)
12 (57.1%)
C16
In agile software development it is a challenge to design requirement documents in such a way that they can be adapted to changing surrounding factors at reasonable effort
21
9 (42.9%)
12 (57.1%)
C17
In agile software development it is a challenge to use methods for elicitation and evaluation of requirements in which the findings are shared with the development team
20
8 (40.0%)
12 (60.0%)
C18
In agile software development it is a challenge to capture requirements in such a way that detailed test cases can be derived from them for quality assurance
21
8 (38.1%)
13 (61.9%)
C19
In agile software development it is a challenge to formulate clear and comprehensible requirements in order to avoid uncertainties in the development
22
7 (31.8%)
15 (68.2%)
C20
In agile software development it is a challenge that elicitation and evaluation of requirements are not fast enough in the project context
