Have you been involved in a software project that ran over budget and blew past every deadline? Of course, you have – we all have. In fact, if you haven’t, you are a unicorn and I would like to hear from you.
Early in my software development career, I learned the importance of working backwards from a deadline. If a project must be done by a certain date and testing will take a certain amount of time, then we can use that information to work backwards and choose a due date for our project. Perfect, right?
Well, not quite. While building in time for testing reduced some stress in the final days of projects, there were still too many surprises.
Building in time for QA testing is great in theory but quickly falls apart in practice once the first bug or defect is identified.
How long will this defect take to fix? How much will it impact the timeline? Will new bugs be introduced? How will we ensure each fix is verified with time to fix anything we broke while we were fixing the first thing?
Ultimately, I was never able to find the correct amount of time to allocate for QA. Inevitably, rushed fixes were merged at the last minute; I learned to keep my calendar clear for a couple of weeks after big launches so that I could triage all of the issues we missed (or introduced) in our mad dashes to the finish.
The problem, at the end of the day, was not the time available for testing but rather the timing of the testing. I needed testing sooner and more often. I needed shift left testing.
What Is Shift Left Testing?
If we imagine our software development process as a timeline flowing from left to right, then “shift left testing” becomes somewhat self-explanatory. Simply put, it is the practice of testing earlier and more often in the development life cycle.
What is Shift Left Testing?
The V-Model of Software Development
The V-model is a useful way to conceptualize software development cycles. If we take the traditional waterfall flow and “flip” the Y-axis at the implementation phase, we get the V-model.
A development cycle begins with high-level requirements. These requirements are narrowed down with each successive step down the “V” until we reach the code-level implementation itself. We then verify the implementation, starting with the most granular unit tests and working our way up the “V” to more abstract user acceptance testing.
In a waterfall process, the entire project is made up of a single “V.” As an industry, we have learned that when you leave all of your validation to the very end of a complex project, you are basically setting yourself up to fail.
In an iterative process, we can think of each sprint or iteration as a smaller “V.” We have theoretically achieved our goals of shift left: testing sooner and more often. Problem solved, right? Well… not quite.
Types of Shift Left Testing
You may have noticed that there are two labels on the feedback channel built into the V-model: verification and validation. These are both important.
We need to validate that our user requirements actually solve the problems we set out to solve. We also need to verify that our implementation matches the specifications we get from those user requirements.
Automated testing can be applied to both validations and verifications. BDD (Behavior Driven Design) has led to the creation of technologies such as Cucumber that can automate some parts of the validation process. For the purposes of this article, we will focus on automated testing for verification.
Unit Testing
Unit tests verify the functionality of a specific module within a larger application. The module is tested in isolation, and any communication with other external processes is simulated or mocked. Unit testing and TDD represent the first phase in shift left testing.
Integration Testing
Integration tests attempt to verify the overall functionality of a service or application, including side effects. This is an anti-pattern for reasons we will discuss later.
API Testing / Contract Testing
API tests verify the external endpoints of a single service. The scope of API tests is similar to the scope of integration tests; However, in an SOA or Microservices context, we can think of API tests as the new unit tests.
UI Testing
UI tests verify the complete functionality of an application from the user interface layer. Tools like Selenium make automated UI testing widely accessible.
More Than Just Automation
Shift left isn’t just about automation. Another way to test earlier and more often is to make sure that your QA specialists are involved in every step of your process, beginning with discovery and requirements gathering. Test engineers can do better when they have a greater understanding of the overall implementation, and their insights can help make the architecture more transparent and resilient.
The Benefits of Shift Left Testing
The shorter feedback loops built into shift left processes empower us in several ways. Defects can be found faster, fixes can be applied more efficiently, and lessons learned in one iteration can be applied in the next, to name a few.
Whatever project management methodology or release cadence your team has, you can benefit from the shorter verification feedback loops from shift left testing.
Cost Savings
A defect found by an automated unit test on a developer’s local machine is orders of magnitude cheaper to identify and fix than a defect that has made it all the way to a customer-facing environment.
Developer Wellbeing
When done properly, automated testing and CI can provide the confidence that software engineers need to deploy often — even on Fridays. Finding defects sooner means fewer panicked all-hands moments. Since releases are so painless, fixing the few errors that do make it through is faster and easier too.
Resilient Architecture
Just as more accessible software is usually easier to use for all of us, more testable software can be easier to reason about and maintain. Thinking about testing early can lead to better separation of concern and a more resilient overall architecture.
Greater Overall Quality
Improving the customer experience is our ultimate goal. Shift left can eliminate some incidents that end users might experience and reduce the impact of other incidents. We can use observability to complete this feedback loop and improve our overall software health.
The Dangers of Shift Left Testing
With powerful automation tools at our disposal, it can be tempting to implement every kind of testing on every line of code. This is a dangerous path.
Testing side effects — did that record actually get saved to the database? — is an attractive idea. But testing implementation details is an anti-pattern because these types of tests are extremely brittle. They might need to be changed every time your application is changed. The user interface is also an implementation detail, so UI tests land in this same boat.
Verification tests just care about the “what,” not the “how” or the “why.” Ideally, the user requirements have been designed to validate “why.” To answer “how” we can rely on a more powerful automation in the form of an observability platform.
How to Get Started with Shift Left Testing
When we think of testing “sooner and more often” a certain word comes to mind: continuous. Many (most) software development teams are practicing some form of continuous integration and continuous delivery. Continuous testing is a vital feedback loop in this DevOps cycle.
Continuous Testing
If we think of TDD as “Shift left for monoliths,” then continuous testing is “shift left for distributed architectures.”
TDD had us focus on unit testing. For continuous testing we should focus on API and contract tests. API tests have a number of benefits:
- API tests can prevent one of the most common ways to introduce errors in a microservices application: changing a dependency out of sync with its upstream or downstream services.
- API tests can be owned by the same team that owns the tested service.
- API tests avoid the brittleness of testing side effects and implementation details.
Ideally, these API tests will be run continuously against both production and pre-production environments. Contract testing tools can help automate this process, but that requires additional infrastructure.
What if we could use continuous API testing built into our observability tool? The upcoming synthetic API testing feature from Instana will let you continuously run API tests against all of your environments with minimal effort.
Shift Left vs Shift Right
Shift right testing is the practice of testing later in the development process, usually in production environments. While it may seem strange, shift left and shift right testing are complimentary.
Shift right testing allows us to identify production issues before our customers do. The shorter feedback loops from shift left testing give us the ability to respond to and remediate these production issues rapidly.
Synthetic API testing as a part of your observability platform is the perfect way to combine the benefits of shift left and shift right practices.
The post Shift Left Testing: What Is It and Why Does It Matter? appeared first on Instana.
Top comments (0)