1 Introduction
1.1 Background
2 Mind Mapping
2.1 Early Days and Old Process
2.2 The Introduction of Mind Maps
2.3 Mind Maps for Test Scenarios
2.4 Mind Maps to Represent User Stories
2.5 Repurposing Mind Maps - The Oracles Breakthrough
3 Exploratory Testing
3.1 Testing After CI Pipeline
3.2 Setting the Local Environment
4 Continuous Integration Pipeline
4.1 EDS Pipeline Structure
4.2 Current Challenges
-
Stability - This is our number one priority. Our tests aren’t as stable as we wished they were, and we have some flakiness at times. Our approach to improve this issue has been to tackle the most unstable tests first, disabling and fixing them. We are making some progress but we still have some tests (the ones in worst shape) that show a failure rate of around 13 % which is not that good.
-
Speed - The entire pipeline, from the first to the last step takes about 19 min. We are always concerned about this, because we don’t want our pipeline to become a delivery bottleneck for our User stories, and as we grow our codebase, the number of regression tests keeps increasing. Our latest improvements were to use as much parallelization as possible: our functional test suites (steps 3 to 5) run simultaneously as well as the suites themselves also parallelize their tests, using a selenium grid.