Agile Processes, in Software Engineering, and Extreme Programming

The integration of user-centred design and Agile development is becoming increasingly common in companies and appears promising. However, it may also present some critical points, or communication breakdowns, such as a variable interpretation of user involvement, a mismatch in the value of documentation, and a misalignment in iterations. We refine these themes, emerging from both literature and previous fieldwork, by analysing a case study performed in an IT company that adopts both software engineering approaches, and we further extend the framework with a new theme related to task ownership. We argue that communication breakdowns can become focal points to drive action and decision for establishing an organisational context acknowledging the value of user involvement: to this end, we suggest the adoption of design thinking and the active engagement of the customer in embracing its values.

Many tools have been created for measuring the agility of software teams, thus creating a saturation in the field. Three agile measurement tools were selected in order to validate whether they yield similar results. The surveys of the tools were given to teams in Company A ($$N=30$$N=30). The questions were grouped into agile practices which were checked for correlation in order to establish convergent validity. In addition, we checked whether the questions identified to be the same among the tools would be given the same replies by the respondents. We could not establish convergent validity since the correlations of the data gathered were very few and low. In addition, the questions which were identified to have the same meaning among the tools did not have the same answers from the respondents. We conclude that the area of measuring agility is still immature and more work needs to be done. Not all tools are applicable to every team but they should be selected on the basis of how a team has transitioned to agile.

One of the benefits of agile is close collaboration of customer and developer. This ensures good commitment and excellent knowledge flows of information about priorities and efforts. However, it is unclear if this benefit can be leveraged at scale. Clearly, it is infeasible to use practices such as planning game with several agile teams in the room. In this paper, we investigate how a large-scale agile organization manages, what challenges exist, and which opportunities can be leveraged. We found challenges in three areas: (i) the ability to estimate, prioritize, and plan; (ii) the context of planning with respect to working environment, team build-up, and team spirit; and (iii) the ceremonial agreement which promises to allow leveraging abilities in a given context.

With agile teams becoming increasingly multi-disciplinary and including all functions, the role of customer feedback is gaining momentum. Today, companies collect feedback directly from customers, as well as indirectly from their products. As a result, companies face a situation in which the amount of data from which they can learn about their customers is larger than ever before. In previous studies, the collection of data is often identified as challenging. However, and as illustrated in our research, the challenge is not the collection of data but rather how to share this data among people in order to make effective use of it. In this paper, and based on case study research in three large software-intensive companies, we (1) provide empirical evidence that ‘lack of sharing’ is the primary reason for insufficient use of customer and product data, and (2) develop a model in which we identify what data is collected, by whom data is collected and in what development phases it is used. In particular, the model depicts critical hand-overs where certain types of data get lost, as well as the implications associated with this. We conclude that companies benefit from a very limited part of the data they collect, and that lack of sharing of data drives inaccurate assumptions of what constitutes customer value.

A bad software development process leads to wasted effort and inferior products. In order to improve a software process, it must be first understood. Our unique approach in this paper uses code and test changes to understand conformance to the Test Driven Development (TDD) process.We designed and implemented TDDViz, a tool that supports developers in better understanding how they conform to TDD. TDDViz supports this understanding by providing novel visualizations of developers’ TDD process. To enable TDDViz ’s visualizations, we developed a novel automatic inferencer that identifies the phases that make up the TDD process solely based on code and test changes.We evaluate TDDViz using two complementary methods: a controlled experiment with 35 participants to evaluate the visualization, and a case study with 2601 TDD Sessions to evaluate the inference algorithm. The controlled experiment shows that, in comparison to existing visualizations, participants performed significantly better when using TDDViz to answer questions about code evolution. In addition, the case study shows that the inferencing algorithm in TDDViz infers TDD phases with an accuracy (F-measure) of 87%.

Startups operate with small resources in time pressure. Thus, building minimal product versions to test and validate ideas has emerged as a way to avoid wasteful creation of complicated products which may be proven unsuccessful in the markets. Often, design of these early product versions needs to be done fast and with little advance information from end-users. In this paper we introduce the Minimum Viable User eXperience (MVUX) that aims at providing users a good enough user experience already in the early, minimal versions of the product. MVUX enables communication of the envisioned product value, gathering of meaningful feedback, and it can promote positive word of mouth. To understand what MVUX consists of, we conducted an interview study with 17 entrepreneurs from 12 small startups. The main elements of MVUX recognized are Attractiveness, Approachability, Professionalism, and Selling the Idea. We present the structured framework and elements’ contributing qualities.

Multi-project agile software development is a relatively new area of research. While original Scrum caters to co-located teams working on a single project, multi-project Scrum teams are a day-to-day reality, especially in small organizations. Multitasking across projects is frequently associated with loss of effectiveness, but this assumption is not sufficiently supported by empirical evidence. In order to better understand the phenomenon, we review existing literature across scientific domains and execute an action research project. Our findings show that the Team Portfolio Scrum (TPS) practice designed to support multitasking across projects is perceived to be useful, but with an associated increase in overhead.

Various agile methods have several quality assurance mechanisms embedded in the process itself, without any explicit QA role. In principle, the team takes care of quality assurance during sprints and as part of daily stand-ups, sprint reviews and retrospectives. We have defined SafeScrum, a variant of Scrum with some additional XP techniques that can be used to develop safety-critical software and have the software certified according to the IEC 61508 standard. This imposes a load of additional requirements on the process. In a recent industrial case, we have experienced that the quality assurance mechanisms in Scrum becomes insufficient. We have therefore analyzed the standard, consulted an independent assessor and worked with the Scrum team to identify necessary additional tasks for a team-internal QA role to be added to the SafeScrum process.

Software developers are both users of development tools but also designers of new software systems. This dual role makes developers special users of work-related software. To increase the understanding of developers as users and to evaluate the ability of common measurement scales to address developer experience, we conducted a survey measuring developers’ flow state, intrinsic motivation and user experience. Scales used were the Short Dispositional Flow Scale, items from the Intrinsic Motivation Inventory, the Short AttrakDiff-2, and our own DEXI scale. 57 developers from 25 countries responded and results indicate that intrinsic motivation and autotelic experience are significant predictors of developers’ UX whereas hedonic, pragmatic, and general quality are not. In addition, developers’ needs are characterized by efficiency, informativeness, intuitiveness, and flexibility of the tool.

Minimum viable product (MVP) is the main focus of both business and product development activities in software startups. We empirically explored five early stage software startups to understand how MVP are used in early stages. Data was collected from interviews, observation and documents. We looked at the MVP usage from two angles, software prototyping and boundary spanning theory. We found that roles of MVPs in startups were not fully aware by entrepreneurs. Besides supporting validated learning, MVPs are used to facilitate product design, to bridge communication gaps and to facilitate cost-effective product development activities. Entrepreneurs should consider a systematic approach to fully explore the value of MVP, as a multiple facet product (MFP). The work also implies several research directions about prototyping practices and patterns in software startups.

Many software engineering organizations around the world are adopting DevOps. One of the goals of DevOps is to foster better collaboration between development and operations personnel, in order to improve organizational efficiency. Since DevOps is lacking a common definition, there are several approaches to adopt it, and organizations largely need to determine how to apply DevOps for themselves. In this paper, we present results from a case study in which a software organization adopts DevOps. The focus of this research is to study the impact of mixing the responsibilities between development and operations engineers. We interviewed 14 employees in the organization during the study, and results indicate several benefits of the chosen approach, such as improved collaboration and trust, and smoother work flow. This comes at the cost of a number of complications, such as new sources for friction among the employees, risk for holistically sub-optimal service configurations, and more.

In this paper, we present an analysis of more than 500 K comments from open-source repositories of software systems developed using agile methodologies. Our aim is to empirically determine how developers interact with each other under certain psychological conditions generated by politeness, sentiment and emotion expressed within developers’ comments. Developers involved in an open-source projects do not usually know each other; they mainly communicate through mailing lists, chat, and tools such as issue tracking systems. The way in which they communicate affects the development process and the productivity of the people involved in the project. We evaluated politeness, sentiment and emotions of comments posted by agile developers and studied the communication flow to understand how they interacted in the presence of impolite and negative comments (and vice versa). Our analysis shows that “firefighters” prevail. When in presence of impolite or negative comments, the probability of the next comment being impolite or negative is 13 % and 25 %, respectively; ANGER however, has a probability of 40 % of being followed by a further ANGER comment. The result could help managers take control the development phases of a system, since social aspects can seriously affect a developer’s productivity. In a distributed agile environment this may have a particular resonance.

In the last decade, Kanban has been promoted as a means for bringing visibility to work while improving the software development flow, team communication and collaboration. However, little empirical evidence exists regarding Kanban use in the software industry. This paper aims to investigate the factors that users perceive to be important for Kanban use. We conducted a survey in 2015 among Kanban practitioners in the LeanKanban LinkedIn community. The survey results consist of 146 responses from 27 different organisations, with all respondents being experienced in using Kanban. The results show that practitioners perceived Kanban as easy to learn and useful in individual and team work. They also consider organisational support and social influence to be important determinants for Kanban use. Respondents noted various perceived benefits for using Kanban, such as bringing visibility to work, helping to reduce work in progress, improving development flow, increasing team communication and facilitating coordination. Despite the benefits, participants also identified challenges to using Kanban, such as organisational support and culture, difficulties in Kanban implementation, lack of training and misunderstanding of key concepts. The paper summarises the results and includes a discussion of implications for effective deployment of Kanban before describing future research needs.

Software startups are challenging endeavours, with various road blocks on their path to success. The current understanding of the challenges that software startups may encounter is very limited. In this paper, we use the research framework of learning and product development stages to analyse the key challenges that software startups have to deal with at different life cycle stages, from problem definition to solution validation and from concept to mature product. Based on an analysis of the empirical data collected by a large survey of 4100 startups, we find out that what perceived as biggest challenges by software startups do vary across different life cycle stages. Building product is the biggest obstacle for software startups, even though its significance decreases when the learning focuses of the startups move from problem to solution and their products mature. Business related challenges such as customer acquisition and scaling are more noticeable at the later stages. Our study raises the awareness of these challenges and suggests to tackle right challenges at the right time.

The Mob Programming technique proves to be an effective learning instrument with a group of less experienced developers. It is also used to explore topics outside of just software development.This paper describes how, with a set of weekly Mob Programming sessions, the teams as a whole and all its individuals have grown much faster than they could have done otherwise. They improved their coding skills, mastery of tools, involvement in Scrum ceremonies, estimation skills, process modeling (!) and learned to be much more self-sufficient.This didn’t happen without plenty of experimentation, and some dead ends. I will describe the different approaches we tried, how we ended up with a surprisingly strict process for our mobbing sessions, and how acceptance was easier with a team that had fewer ingrained habits of work.

Agile development with continuous integration and constant releases is only sustainable followed by a rock solid quality process. At blip/betfair we work very hard to build and continuously improve our quality process to provide at the same time a unique reliability experience to our customers and new features fast. Three major components of this process are: Mind maps to help us learn more about our product and represent our knowledge about it in a structure way, Exploratory Testing that must be free and creative and happen as soon as possible in the process to allow fast feedback cycles and CI pipelines with high levels of automation testing to avoid regression. Agile development with continuous integration and constant releases is only possible with a rock solid quality process.

Organizations take up agile transformation as silver bullet for all their business problems, but the fact is transformation journey is an eye opener to discover the real problems which were previously unnoticed. The authors were part of such a journey. It’s easy to reap the obvious benefits of agile, but difficult to sustain and solve systemic obstacles like long build time, complex code base and legacy architecture that become a way of life over a long period of time. Here we describe the challenges we faced in sustaining our transformation beyond early victories and our efforts towards identifying and solving systemic obstacles across the organization by setting up an effective CI environment and addressing top people issues.

Kainos is a software company based in Belfast, Northern Ireland. As well as bespoke development, its work includes service contracts for the maintenance of software created elsewhere. This type of work is challenging because of the knowledge transition involved. The experience reported here is of tackling such projects in a way that integrates with the agile processes of the client. Background on agile practice in Kainos is discussed before focusing on a specific project for the UK Government Cabinet Office.

In 2012 the University of Vienna started a project, named Student Service Portal (SSP), to create a new portal for the universtiy´s students, university teachers, and administrative staff. The university signed a fixed price project with an external main contractor. Although a lot of effort was put into writing detailed requirements documents, it remained unclear what the exact scope was. Project management was lacking, technical problems arose, and finally the university and the supplier got caught up in each other’s blame instead of working together. After two years without tangible results the rectorship of the university stopped the project and ordered a restart – this time with an agile approach. The main contractor was replaced. The IT and the business department took over full responsibility for the product together.

This paper continues telling the story begun in “It has been a long journey, and it is not over yet” (published in Agile Process in Software Engineering and Extreme Programming XP2015, Helsinki – 2015). This experience report tells the tale of the quest to define the role of the architect and of architecture in an agile environment. The primary observation here is that people skills are a key factor in that role.

What do you do when you have endured an Agile experience where things didn’t go so well? You can abandon Agile altogether or you can take those lessons learned and apply them to future Agile projects. This paper discusses the journey travelled from that painful experience to becoming a more confident and experienced Agile practitioner. We will also look at some of the challenges that I still encounter today.

It is not surprising that agile methods are tailored or customized in various contexts and projects. However, there is little advice for practitioners for how to go about tailoring agile methods in large-scale projects. Henceforth, the aim of this experience report is to highlight some of the challenges with large-scale agile software development and especially how to deal with these challenges involves continuous tailoring of the agile method in use. In so doing, we report from a large-scale agile software development effort involving more than 120 participants in a Governmental organization and running for 3,5 years. The project consisted of three deliverables, partly developed in parallel after a delivery model based on Scrum. After a much troubled start related to scaling challenges and architecture complexity during the first deliverable, the project was turnaround and the second and third deliverables were portrayed fairly successful by both supplier and customer. From a practitioner’s perspective, we found that novel practices emerged through out the project that improved the way of working – especially across teams and stakeholders. Based on this, we describe some guidelines for tailoring agile in the large.

Hiring senior software engineers with experience in Agile and Lean has always been difficult. Training university graduates or engineers from other backgrounds takes time and can cause disruption to software teams. 7digital addressed both of these problems by starting a Technical Academy; a 6 month programme of classroom sessions, pairing, deliberate practice, personal project work and guided learning. Backed by key metrics and qualitative data, the paper explores the positive impact that the technical academy has had on the technology team and wider organisation at 7digital. It investigates the changes in technique, curriculum and structure that the team made over the three iterations of the academy. It goes on to detail the challenges that the team faced around justifying the time away from usual activities, measuring the impact, attempting to predict the long term benefit and make the result of extra diversity in the team more apparent.

This paper describes my experience as a female software developer with 17 years’ industry experience. Originally I worked with a more traditional waterfall approach to software design and development, but in recent years I have worked with XP. I have experienced many difficulties associated with being in a minority, but a lot of those problems have been alleviated since I started working with XP. My belief is that XP creates a more conducive environment for women and other minorities within the industry. I believe that XP can – and should – pave the way to making the tech industry a more welcoming and attractive place for women.

This experience report offers a beginner’s perspective on pair-programming with experienced developers. It discusses issues faced by juniors and seniors when working together and highlights the importance of emotional maturity in pairs with disparate skill sets. This paper considers personal characteristics of junior and senior developers in identifying their needs from the pairing session and shares tactics used to improve pair-programming experience on individual and team-wide levels.

Sketching is an important means of communication in software engineering practice. Yet, there is little research investigating the use of sketches. We want to contribute a better understanding of sketching, in particular its use during program comprehension. We propose a controlled experiment to investigate the effectiveness and efficiency of program comprehension with the support of sketches as well as what sketches are used in what way.

As part of our research concerning the integration of assurance case development with Scrum, we are planning to conduct semi-structured interviews with participants to gain feedback on a proposed approach. We will be interviewing individuals who have been involved with safety-critical systems development and Agile methods. Participants will be presented with an overview of the challenges associated with applying the 4+1 software safety assurance principles to Scrum. Initial recommendations concerning how the principles can be accommodated within a Scrum development will also be presented. Participants will be led through a series of questions to gain feedback on the feasibility of the approach, and for an assessment as to whether the 4+1 principles can be addressed without compromising agility. The motivation behind this research is to gain a deeper insight into the difficulties experienced when integrating assurance case in to Scrum process.

The software engineering community is continuously producing tools to tackle software construction problems. This paper presents a research study to identify which tools, artifacts, and commands developers use during task solving and how one can design software that can suggest and convince the developer to use specific software construction techniques. We want to understand under which conditions developers accept suggestions for a more efficient and effective usage of the available instruments, and if observed usage patterns correlate with observable improvements in the process or product. The expected results include detailed logs of how developers construct software during XP 2016, their preferences for software construction recommendations, and which effects accepted suggestions have on task execution and outcome.

In the last two decades, Agile and Lean approaches have gained wide acceptance in the software industry. In this realm, Kanban emerged in 2004 with a strong practitioner-driven support movement and today, Kanban is increasingly adopted to complement Scrum and other Agile methods. Kanban tends to focus on fast production, rapid and continual user feedback and interaction.

Software Quality Assurance is a complex and time-expensive task. In this study we want to observe how agile developers react to just-in-time metrics about the code smells they introduce, and how the metrics influence the quality of the output.

The integration of user-centred design and Agile development is becoming increasingly common in companies and appears promising. However, it may also present some critical points, or communication breakdowns, which manifest in working practices. A solution is likely to be found in a supportive organisational context: in this sense, communication breakdowns can become focal points to drive action and decision for establishing an organisational environment acknowledging the value of user involvement and actively endorsing it also with the customer.Expected graduation year: 2017.Supervisor: prof. Antonella De Angeli, Department of Information Engineering and Computer Science, University of Trento, Italy.Email: antonella.deangeli@disi.unitn.it

Empirical studies covering Agility at the organisational scale are few in number. Organisations seeking clarity about the efficacy of any approach to business Agility must turn to the commercial literature for information and guidance. As a whole, research into Agile Software Development suffers from a lack of rigour and theoretical grounding, a problem also evident in Information Systems research in general. These issues have led to recent calls for a clear research agenda for scaling Agility and for the quality of contributions to be addressed. Diffusions research has a long history in a wide range of domains and provides a clear theoretical framework for this qualitative PhD study.

Unlike traditional software development approaches, Agile embraces change. The resulting dynamism of requirements makes it challenging to estimate effort accurately. Current practice relies on expert-judgment that can be biased, labor intensive and inaccurate. Therefore, a systematic yet lightweight effort estimation methodology is needed to support expert judgment and improve its effectiveness. Such an approach will utilize the quantification of the impact of a requirement on software artifacts potentially affected by it. It will further introduce an explicit consideration of effort drivers that contribute to effort overhead. The aim is to synthesize research from three often orthogonal areas of research: (1) change impact analysis, (2) effort estimation (model and expert driven) and (3) software visualization. Hence, resulting in a hybrid methodology with tool support that incorporates expert knowledge, change impact analysis and enables an explicit consideration of cost drivers by experts to improve the effectiveness of effort estimation process.

Agile methods are increasingly being applied to large scale and distributed software development. While there is much evidence to support the efficiency of agile practices in small co-located team, less is known about the applicability of these practices to large scale projects. This paper gives an outline of planned research on the scaling of retrospectives. By using retrospectives as an empirical lens I will try to gain insight into the limitations and benefits of agile practices in large scale and distributed development.

This proposal is design to address the proposed research work on agile software development and architecture co-existence. The objective of this research is to answer how architecting activities emerge and evolve with agile software development in industry. The architecting activities are architectural analysis (AA), architectural synthesis (AS), architectural evaluation (AE), architectural implementation (AI), architectural maintenance and evolution (AME), architectural recovery (AR), architectural description (ADp), architectural understanding (AU), architectural impact analysis (AIA), architectural reuse (ARu) and architectural refactoring (ARf). This research objective could achieve by using multiple research methods. We are planning to use comprehensively report the pure ‘state- of- practice’ for architecting activities in ASD from industry and practitioners point of views. Therefore, we decided to use the case studies, survey and semi structure interview as research methods. The result of this research work can provide the baseline information for architecture evolution frameworks for agile software development, challenges and solutions in ASD for SA activities, expected evolvable dimensions of the software system, methods that may help for minimizing the architectural and agile co-existence issues and architectural technical debt in agile software development.