4.1 Technical Ability Challenges
For adequate planning, agile teams (regardless of their scale) need to bring together the ability to estimate required work, prioritize it with respect to business value, and to combine this knowledge into a good plan for the coming iteration(s). At a large-scale, where a hierarchy of product owners manages backlog items for a large number of teams, we identify (communication) challenges in all three parts.
“[Previously we] estimated on available days in the sprint, that is not a good way because you do not include the unexpected things” [OPO]
Another challenge with estimation in large-scale agile is the need to monitor discussions during story estimation. Our interviewees reported that without systematic monitoring of such discussions, they could go on in circles for hours without making significant progress.
In the context of cross-functional teams, an unstructured estimation session can also lead to a pathological situation, where team members should
estimate tasks that do not fit their role, as for example shown by the following quote from one of our interviewees:
“[Sometimes we have a] tester estimating design tasks and a designer estimating test tasks. It is important to know whose estimation should be looked at”. [Line manager]
Most of our interviewees stated that they are not experts in estimation and the challenges of large-scale agile estimation create a steep learning curve, especially for new team members. Estimation is based learning from past iterations and experience in the team.
“The challenge is if you have a lot of small backlog you are not in control at all because if you have one common backlog and you decide on a program level, that is how we work […] if not everything is visible on the common backlog program and only visible in the XFTs backlog then you maybe having a mismatch.” [OPO]
Such potential inconsistencies between different backlogs as well as lack of their transparency are a big challenge. According to our interviewees, prioritization decisions need to be done on the program level. In contrast, information that is only available in the local backlog of a cross-functional team cannot be accessed and taken into consideration. Thus, if significant information is only available on the team’s level and not visible on the program level, prioritization decisions cannot be optimal and mismatches will be the consequence. It is impossible to share all information (e.g. about decisions, new technology, dependencies) with other teams, product owners, and the whole organization. It is also very hard to understand the significance of information on the team level for prioritization. This potential mismatch makes it hard for the program board to establish a prioritization that teams will follow.
Unclear requirements, one of the major challenges in large-scale agile planning, affect both estimation and prioritization. Our interviewees reported to be challenged to gain knowledge about the underlying user needs of a feature. This includes a potential lack of experience and knowledge about new features, starting with the area product owner.
Another challenge is the unclear role of operational product owners and our interviewees mention slightly different challenges in the two different products. In Product A, our interviewees expressed confusion about to what level teams should be involved in planning. In contrast, the responsibilities of operational product owners in Product B where clearer. They interacted quite naturally with informal leaders in the teams during planning, thus allowing involvement of the team, without distracting the team members too much from their development tasks. Accordingly, it is beneficial to share the plan with the whole team, without giving the team the formal responsibility to report on it.
Interviewees in both products agreed that balancing the involvement of teams in large-scale agile planning is challenging and that it is crucial to find a good process for their involvement. It is difficult for teams to engage in long-term planning beyond the next 18 month and while this might be necessary for large-scale agile development, our interviewees indicated that the teams do not benefit from participating in such long-term planning and do not understand why their participation should be necessary. While such a long-term plan is interesting to program leaders, program managers, and to the product line, teams should focus on producing results and cut the lead-time.
Finally, our interviews reveal technical dependencies as well as dependencies between hardware and software as a planning challenge. If known, they represent constraints on planning. However, often such technical dependencies are hidden, leading to duplicate work or waiting time. While such waste cannot fully be avoided during large-scale agile planning, approaches to mitigate its impact are needed.
4.2 Contextual Challenges
In addition to the general ability to estimate, prioritize, and plan, our data also revealed a number of challenges with respect of the context of planning, including work environment, team build-up, and team spirit.
“…we have scrum meetings in open office space […]. You kind of get disturbed when other teams are having their scrum meeting in the open setting. It is better [if] every team has their different rooms.“[Dev.]
Another issue mentioned by our interviewees is that sometimes other team members disturb them by walking into their teams to ask for help. While this is important inter-team communication, by-passing the operative product owners can be problematic when it happens too often or on non-trivial issues.
A common practice is to move team members to other teams that require additional resources (such as special knowledge). Our interviewees indicate difficulties in finding candidates to be moved between teams. Also, they mention doubts about the effectivity of this practice, since the team member to be moved does not possess deep knowledge about the target team and their context. Our interviewees claimed that instead competence broadening of established teams should be emphasized to address missing team capabilities. Such a solution of course implies a longer lead-time and our interviewees pointed out that it is challenging for them to learn new roles when they already are experts in other required roles.
4.3 Ceremonial Agreement
In addition to technical abilities and context of planning, we discovered themes in our interview data that affect the room between those two domains: Ceremonial agreement allows a group of agile teams and product owners to align planning abilities with the teams’ context. Efficient and effective information flows are necessary for keeping every employee aware about important decisions. According to our interviewees, it is however challenging to share knowledge about decisions within the large development context due to a lack of suitable information channels. Our interviewees said that they do not get updated on what (other) teams are doing due to insufficient time. If a product owner is responsible for several teams and these teams have their stand-up meetings at the same time, he needs to decide. But also the opposite can happen, when a team or an operative product owner have to interact with two area product owners who are typically very busy. In this situation, it is impossible to have frequent face-to-face meetings with them, resulting in asynchronous communication via email and social media. Such communication is not as effective as face-to-face communication and can result in long response times.
Coordination meetings (such as scrum-of-scrum (SoS)) are a potential solution, but were also criticized as boring or too short by our interviewees. They pointed out that in most of the SoS meetings it is difficult to have a thorough discussion and arrive at a good conclusion. Most of the times they have to close the meetings when they get into interesting technical discussion. One OPO mentioned that such discussions in the meetings might not be interesting to all participants.
While it is important not to by-pass the operative product owner when communicating with the team, this also introduces some indirection, e.g. between release leaders and the team. This requires building trusted relationships between release engineer, operative product owner, and team. A lot of such communication is the consequence of
inadequate anatomy of features, i.e.
“the relation between different features and parts of features”, as one of our interviewees put it. With other words, the way features are split up and assigned to sprint backlogs leads to dependencies between teams and creates the challenge of inter-team communication and coordination within the larger product portfolio. We found senior developers and product owners to rely on their
personal network to coordinate across program boundaries:
“…I have a colleague that works as […] operative product owner in other program and we try to collaborate between the programs and to align features for the customers and user experience”. [OPO]
Thus, we conclude that typical agile ceremonies are well adopted locally within teams, but challenges remain largely on the levels of inter-team and inter-product communication across a portfolio of products.
“The biggest challenge I pick is the coordination of the feature portfolio, […] on top of getting out features in our program fast and efficient, we need to collaborate on a portfolio basis to align the features over two programs”. [OPO]
Again, our findings resonate with Sauer’s recommendation to facilitate team spirit with opportunities for informal exchange, such as coffee breaks [
14]. Ceremonial agreements should support large-scale agile planning in similar ways.