5. Test

Today, nearly every company is in the software business. Although an organization may sell a tangible product, their use of software to streamline and enhance the customer experience means they must place higher importance on quality application delivery. Frequently, applications that are rushed through the development cycle without adequate testing often encounter costly defects that impact the customer relationship.

Just like building a house, the foundation is key to successful software construction. If the foundation has issues, there is a high likelihood for expensive delays further into the process. Using the right development tools at the onset will help ensure that the software foundation is properly defined, constructed, and tested while keeping quality and end user goals top of mind.

The second part of the testing trifecta and an area rife for improvement in the software development lifecycle is in test data management. Every tester needs quality test data and quickly. The challenge is getting the right data to match their tests when they need it. As companies have improved their development processes, moving from Waterfall to agile, testing has lagged behind. Again, manual processes cannot keep pace with a company’s test data need, with companies relying heavily on teams of people constantly creating and maintaining test data.

Another major challenge when managing test data is ensuring compliance with legal and regulatory requirements. Many organizations apply the necessary rigor when protecting personal and sensitive customer information in production, but neglect to consider the implications when working with data in non-production environments. In the event of non-compliance, this can mean significant consequences, not the least brand reputation, but also financial loss due to fines and penalties.

Many industry-specific regulations come with their own unique sets of test data challenges, and some introduce new complexities. Take the General Data Protection Regulation (GDPR) for example. The GDPR is designed to protect the rights of European Union (EU) citizens where the processing of their personal data is concerned.

Although many companies will have already adopted privacy processes and procedures consistent with the directive, the GDPR contains a number of new protections for EU data subjects and threatens significant fines and penalties for non-compliance (up to 4 percent of annual global turnover or 20m euros, whichever is greater) once it comes into force in May 2018.

GDPR introduces many new obligations in areas such as data anonymization, breach notification, and trans-border data transfers, to name just a few. Many have implications for test data management. One example is the “right to erase,” where individuals may notify businesses processing their data what they may or may not use that data for, including testing.

Complying with obligations like this carries a huge overhead. If customers state they don’t want their data used (even it is masked), then testers will need to acquire subsets of data and apply filtering rules. They will also need to ensure their methods can track every record not approved for testing and be fully auditable.

Many businesses might pursue programmatic solutions to these test data problems, but this only increases the development burden and potentially introduces additional fragility. One alternative, of course, is to use more modern synthetic test data generation to avoid these problems completely.

There is a fundamental shift in the way enterprises build applications today. In the early days of mainframe and client/server applications, you had a much more limited scope of applications—all of the components from the database to the UI could be under one development and testing team’s control.

After the Dot-Com days of the early 2000s, a new style of composite applications arose. The new approach to developing software, including agile, created two new challenges for organizations

These complications increased the complexity and cost of developing and maintaining composite applications.

Applications today are the result of many decades of “building systems on top of systems,” which creates huge chains of dependencies. These complex architectures mean software development is more difficult, more costly, and more complex than ever before.

Many large organizations now find that many of systems they depend on such as mainframes, databases, and external services are constrained and not accessible by developers and testers when they are most needed.

For instance, a needed mainframe may be off-limits, a system of record could have bad data, or a third-party service may still be under development. Attempts to reproduce these environments—by manually coding stubs and managing test data—are costly and inconsistent.

One customer with constraint issues put it this way, “I can’t do anything until I have everything… and I never have everything!”

In a recent Voke Market Snapshot Report on Service Virtualization (January 2015), over 500 companies validated that constraints are a major hurdle to innovation in the software development lifecycle.⁴ The report mentions that:

When developers and testers use service virtualization, the services behave and perform similar to the real thing, but without the underlying hardware and software complexity of a physical system. Development and testing continue just as they always have, but less constrained, and without contention between teams for environments, labs, test data, and so on.

Although service virtualization solves many different development problems, four common ones are seen repeatedly:

One common problem that service virtualization solves is in the area of integration. Customers buy companies, they provide service to third parties, or they are updating applications for functionality, compliance, or architecture. Each of these challenges presents an opportunity for service virtualization to improve the software development process. Integration teams and customers have the resources they need for software development and testing, without the added expense of acquiring additional hardware and software.

When time-to-market matters, service virtualization offers an excellent opportunity to shorten development lifecycles. Service virtualization reduces the constraints of software development, allowing more teams to effectively work in parallel, without underlying dependencies. Typically, service virtualization users experience a 25-50 percent reduction in release times.

A huge opportunity for service virtualization is in the area of performance engineering. Creating a lab capable of handling and testing to production capacity loads is difficult and resource intensive. Furthermore, ready access to systems such as mainframes and transaction servers may be impossible. All this makes performance testing expensive, unreliable, and inconsistent. Some teams might have a small window for testing, while others will have to wait for an entire application architecture to be assembled before any testing can commence. But by using service virtualization, teams can performance test each individual component, identifying many performance problems earlier in the lifecycle, and reduce, even eliminate, the amount of final performance testing needed in a production-like lab.

Using traditional Waterfall methodologies for developing software, much of the activity of development and testing of the application happens in a series of steps, one after another. But by eliminating constraints common in typical software development practices, service virtualization enables much of the SDLC to operate in parallel and the steps within it become less time consuming.

Using service virtualization, developers can have their own private environments for coding, directly from the laptop. They don’t share environments and don’t need to wait for other developers to finish their work.

With service virtualization, much of the testing at a component level can “shift left,” or be moved earlier in the SDLC. Because each component can be tested individually (instead of waiting for a complete assembly), unit and regression testing happens sooner and is more complete, and defects are identified long before integration or user acceptance testing. Finding defects earlier means developers fix issues at the point in time they incur the lowest cost. This avoids defects leaking into later stages or even into production and become harder to resolve because developers have been moved onto other projects.

As teams increase service virtualization maturity, regression and individual component testing become increasingly automated. Now validation as early as code check-in is possible, making defect detection a consistent and repeatable process. Again this is possible because service virtualization allows component level testing in isolation, without underlying dependencies.

Once automation is implemented, you can easily make it a continuous process. Using this approach, any change breaking interfaces, contracts, or use patterns are easily detected before the code disrupts other services or applications.

The deployment of service virtualization at one large bank solved two critical challenges. First, by eliminating system dependencies, testing began far earlier in the development cycle. Defects in code no longer lurked until UAT, but were found much earlier.

Additionally, the bank’s formally serial processes were set in parallel, dramatically reducing release times.

Using virtual services reduces the demand for physical hardware and test labs. This approach is distinctly different, and complementary to, hardware virtualization. With service virtualization, you virtualizes services and business functionality instead of hardware.

When demands for hardware decrease, so do costs. The challenges and costs of provisioning labs and equipment, software and configurations disappear. The physical hardware demand decreases dramatically, freeing budgets for application and business investments instead of capital assets. Demand for data center rack space, power, and storage also decrease.

In performance testing, service virtualization helps customers reduce cost and increase quality and flexibility in several ways. Customers can load test at the component level. Instead of waiting until the application is complete, components are tested for volume and capacity independently, locating bottlenecks and issues early.

Production-only systems such as master databases, mainframes, and third-party systems not normally available for load testing are virtualized, creating an always-ready, highly scalable virtual back-end immune to traditional load testing constraints. This benefits users both in convenience and by reducing the cost associated with replicating expensive back-end systems.

Customers may also face third-party access or software license fees. Service virtualization eliminates the need for highly scalable versions of these systems by virtualizing their behavior. For organizations selling services, virtualized versions of their entire platform are made available to customers in virtual form. Validating against a virtual back-end ensures production readiness without the complexities of full production-style dev/test implementations.

Using service virtualization addresses many thorny issues associated with test data management. For example, organizations struggled to set up just the right scenarios, only to “burn” them with a test cycle. Or, find it difficult to construct test scenarios for edge conditions and business logic. In such cases, it often becomes more expensive to set up the test harness than do the test!

Virtualizing behaviors such as edge conditions, negative test scenarios, and error handling are easily configured in the behavior of the virtual service and are never “burned” since the virtual service is simply playing back behavior responses.

With service virtualization, “test data” and scenarios are easily versioned and changed for each new requirement. In addition, when two test cycles or teams have differing needs for test data, they will not collide in the test lab.