What is shift-left?
Shift-left is a practice intended to shorten the feedback loop in the lifecycle in order to take decisive action.
The idea is to improve overall quality and security and more by moving tasks to the left as early in the lifecycle as possible.
What are the main benefits?
Automation
- Much fewer human errors
- Increased test confidence
- Fewer production issues
Increased delivery speed
- The time between releases can reduce significantly
- The quality of software improves
Increased satisfaction
- Faster delivery of software with less defects is a major benefit for all; from product owners, stakeholders and even customers!
Where are some of the places we can shift-left?
Testing
When building API’s for example, the team owns the API product, its design, its code, its quality, in order to ensure a high-quality product is produced for its customers (consumers), the engineering team is responsible for testing its own product. From unit-testing to performance testing and all those in-between, the list above is not an exhaustive one, and there are many other testing practices you can put in place.
I do hope many of the items listed above are being carried out by an engineering team who owns the product already, for those who are not, the next step is how? How do we, “shift-left”? Well, we need to automate, if you can automate these items, and let’s be honest, who hasn’t automated some of these before in a CI (Continuous Integration) pipeline? - but not everyone has. If you haven’t, these should be an essential items in your shift-left shopping basket!
Security
Security is paramount these days, and a lot of the time, InfoSec (Information Security) or security teams are on the right. Quite often, these teams are not large teams, and adopt a decentralised model, which puts the responsibility on engineering teams. This means the pressure is already on the engineering team to shift-left and bake security practices into the lifecycle, early and often.
Through automation, we can achieve fantastic results, as part of CI, it is easy to achieve static / dynamic code analysis, security scanning of internal and open source libraries, and even container scanning. This all means as an engineer, you’ll know very early on if something isn’t quite right, instead of waiting later, and possibly relying on another team.
As the world of tech continues its love affair with containerisation, security is always at the forefront of security engineers, and with shift-left, it now should also be at the forefront of every software engineer. With containers specifically, container scanning tools are abundant, and some container registries also scan images that are stored. Scanning before pushing a container and when it lands in the registry, is a typical practice to follow.
With great power comes great responsibility, security should never be an afterthought.
Infrastructure
Infrastructure is something we simply can’t avoid, even with serverless; serverless may have integration on vnets (virtual networks). Usually there will be a centralised Operations or Platform team who owns, designs, builds and operates all infrastructure. This can become a big blocker for engineering teams, teams who cannot deploy their own API’s for example, are reliant on another team, and this will in-turn break team autonomy.
So what do we do, we shift-left? Now, each group of teams will be different, each infrastructure will be different, it is likely you may not shift-left 100%, but you can easily shift-left quite a lot in most setups.
A centralised Operations / Platform team can design, build, and operate baseline, foundational, bedrock infrastructure. Thereby, product teams can build on top of this, and come together when needed, for instance, virtual networks, firewalls, a SEIM as examples.
As a product team, what can you now do? Well, now the engineering team can shift-left; they can build on top of the foundational infrastructure. This improves team autonomy, a sense of ownership, and responsibility, furthermore, you breakdown that barrier to release frequently - Deployment frequency is a key DevOps metric.
One way to achieve this is through infrastructure as code (IaC), and we treat it like any other code, it lives in git. We create PR’s (Pull Requests) and go through CI and CD (Continuous Deployment), it can all be automated, checks can be executed, linters, most tools such as Terraform and Pulumi can preview the infrastructure changes before they are applied.
If you’ve never done IaC before, don’t be afraid… It’s just like learning a new language, and as an engineer, it’s a healthy and a nice, challenging thing to do. Terraform is HCL - Hashicorp Language; although they are building cloud development kits (CDK’s) in other languages now. Then there is Pulumi, a fantastic platform where the barrier to entry is incredibly low, if you’re engineer building API’s and use C#, Go, Python or even JavaScript with NodeJS, you’ll feel at home with Pulumi. Finally, if you’re more front-end, Pulumi supports JavaScript as mentioned, and also TypeScript too! These are just two mainstream examples, cloud vendors also support IaC with their solutions, Azure with ARM Templates (Bicep is a new slimmer version that is in very early alpha), AWS with Cloud Formation etc. The point is to embrace IaC into product teams so an engineering team is more in control of their own destiny. It doesn’t mean never speak to these centralised Operations / Platform teams, if means an engineering team is less reliant and can go at their pace - which is hopefully at a good fast. One key element to remember is...
DevOps is not a role, it’s a culture!
Operations
Once an API, or a server-side rendered web application, or even a statically generated web application is in production, it needs to be monitored; I wonder who should do that….? A centralised team…. or the team that designed it, built it and deployed it….? It’s of course the engineering team that designed it, built it etc.
In many organisations just like infrastructure above, the Operations or Platform teams operate the web applications and API’s etc., this is rather right-heavy so-to-speak. As with infrastructure, the same problems occur with operating services, they define monitoring and alerting, are on-call and more. This can work great for smaller organisations, however, if you are in an organisation that is growing and growing as rapidly as NewDay for instance, this model starts to become a hinderance. Now, what do we do, that’s right, we shift left!
As part of our normal engineering lifecycle, we define performance metrics, degradation metrics, any kind of metric we need for our apps and API’s, we go further and define alerts, anything that is perhaps close to breaching our metrics where we have clear and concise SLA’s (service level agreements) or SLO’s (service level objectives). As part of IaC, these can be baked in as code, we can test early and often in our engineering lifecycle. Once we get to production, it’s the job of the engineering team to monitor there apps and API’s, as they are the ones who designed it, built it and deployed it, the team will be the one to operate it. The engineering team will most likely be on-call if your organisation requires 24/7 operations, because if something goes wrong, the best people to fix it is the team that built it!
Finally
Shifting-left is a journey, it’s not a short journey, but it will be an enjoyable one, one where people come together not just as individual teams, but as a community of engineers to help drive forward-thinking, innovative change.
The shift-left philosophy will bring unprecedented value to engineers and your end product.
Top comments (1)
Wow, this is very comprehensive and insightful, Pete! Thanks for sharing. Shift-left testing is being used by an increasing number of companies in the modern era of software delivery to ensure that the software they release to the market can outperform the competition. But it is imperative for organizations to strategize the Shift-Left Testing appropriately. To know more, refer to this blog - bit.ly/3BBDjcF I also noticed an upcoming webinar on Shift Left Testing, which I think will be very useful. If interested, here is the link to register - bit.ly/3zxqWfG