DEV Community

loading...
Cover image for Shift-Left Engineering
NewDay Technology

Shift-Left Engineering

peteking profile image Pete King Updated on ・6 min read

What is shift-left?

Shift-left is a practice intended to shorten the feedback loop in the lifecycle in order to take decisive action.

The idea is to improve overall quality and security and more by moving tasks to the left as early in the lifecycle as possible.

Sounds awesome

What are the main benefits?

Automation

  • Much fewer human errors
  • Increased test confidence
  • Fewer production issues

Increased delivery speed

  • The time between releases can reduce significantly
  • The quality of software improves

Increased satisfaction

  • Faster delivery of software with less defects is a major benefit for all; from product owners, stakeholders and even customers!

Where are some of the places we can shift-left?

Shift-left testing security infrastructure operations


Testing

When building API’s for example, the team owns the API product, its design, its code, its quality, in order to ensure a high-quality product is produced for its customers (consumers), the engineering team is responsible for testing its own product. From unit-testing to performance testing and all those in-between, the list above is not an exhaustive one, and there are many other testing practices you can put in place.

I do hope many of the items listed above are being carried out by an engineering team who owns the product already, for those who are not, the next step is how? How do we, “shift-left”? Well, we need to automate, if you can automate these items, and let’s be honest, who hasn’t automated some of these before in a CI (Continuous Integration) pipeline? - but not everyone has. If you haven’t, these should be an essential items in your shift-left shopping basket!


Security

Security is paramount these days, and a lot of the time, InfoSec (Information Security) or security teams are on the right. Quite often, these teams are not large teams, and adopt a decentralised model, which puts the responsibility on engineering teams. This means the pressure is already on the engineering team to shift-left and bake security practices into the lifecycle, early and often.

Through automation, we can achieve fantastic results, as part of CI, it is easy to achieve static / dynamic code analysis, security scanning of internal and open source libraries, and even container scanning. This all means as an engineer, you’ll know very early on if something isn’t quite right, instead of waiting later, and possibly relying on another team.

As the world of tech continues its love affair with containerisation, security is always at the forefront of security engineers, and with shift-left, it now should also be at the forefront of every software engineer. With containers specifically, container scanning tools are abundant, and some container registries also scan images that are stored. Scanning before pushing a container and when it lands in the registry, is a typical practice to follow.

With great power comes great responsibility, security should never be an afterthought.

Great power


Infrastructure

Infrastructure is something we simply can’t avoid, even with serverless; serverless may have integration on vnets (virtual networks). Usually there will be a centralised Operations or Platform team who owns, designs, builds and operates all infrastructure. This can become a big blocker for engineering teams, teams who cannot deploy their own API’s for example, are reliant on another team, and this will in-turn break team autonomy.

So what do we do, we shift-left? Now, each group of teams will be different, each infrastructure will be different, it is likely you may not shift-left 100%, but you can easily shift-left quite a lot in most setups.

A centralised Operations / Platform team can design, build, and operate baseline, foundational, bedrock infrastructure. Thereby, product teams can build on top of this, and come together when needed, for instance, virtual networks, firewalls, a SEIM as examples.

As a product team, what can you now do? Well, now the engineering team can shift-left; they can build on top of the foundational infrastructure. This improves team autonomy, a sense of ownership, and responsibility, furthermore, you breakdown that barrier to release frequently - Deployment frequency is a key DevOps metric.

One way to achieve this is through infrastructure as code (IaC), and we treat it like any other code, it lives in git. We create PR’s (Pull Requests) and go through CI and CD (Continuous Deployment), it can all be automated, checks can be executed, linters, most tools such as Terraform and Pulumi can preview the infrastructure changes before they are applied.

If you’ve never done IaC before, don’t be afraid… It’s just like learning a new language, and as an engineer, it’s a healthy and a nice, challenging thing to do. Terraform is HCL - Hashicorp Language; although they are building cloud development kits (CDK’s) in other languages now. Then there is Pulumi, a fantastic platform where the barrier to entry is incredibly low, if you’re engineer building API’s and use C#, Go, Python or even JavaScript with NodeJS, you’ll feel at home with Pulumi. Finally, if you’re more front-end, Pulumi supports JavaScript as mentioned, and also TypeScript too! These are just two mainstream examples, cloud vendors also support IaC with their solutions, Azure with ARM Templates (Bicep is a new slimmer version that is in very early alpha), AWS with Cloud Formation etc. The point is to embrace IaC into product teams so an engineering team is more in control of their own destiny. It doesn’t mean never speak to these centralised Operations / Platform teams, if means an engineering team is less reliant and can go at their pace - which is hopefully at a good fast. One key element to remember is...

DevOps is not a role, it’s a culture!

team work


Operations

Once an API, or a server-side rendered web application, or even a statically generated web application is in production, it needs to be monitored; I wonder who should do that….? A centralised team…. or the team that designed it, built it and deployed it….? It’s of course the engineering team that designed it, built it etc.

In many organisations just like infrastructure above, the Operations or Platform teams operate the web applications and API’s etc., this is rather right-heavy so-to-speak. As with infrastructure, the same problems occur with operating services, they define monitoring and alerting, are on-call and more. This can work great for smaller organisations, however, if you are in an organisation that is growing and growing as rapidly as NewDay for instance, this model starts to become a hinderance. Now, what do we do, that’s right, we shift left!

As part of our normal engineering lifecycle, we define performance metrics, degradation metrics, any kind of metric we need for our apps and API’s, we go further and define alerts, anything that is perhaps close to breaching our metrics where we have clear and concise SLA’s (service level agreements) or SLO’s (service level objectives). As part of IaC, these can be baked in as code, we can test early and often in our engineering lifecycle. Once we get to production, it’s the job of the engineering team to monitor there apps and API’s, as they are the ones who designed it, built it and deployed it, the team will be the one to operate it. The engineering team will most likely be on-call if your organisation requires 24/7 operations, because if something goes wrong, the best people to fix it is the team that built it!


What's next?

If you've reached this far, thank you, I'm sure you're now wondering what's next? Well, we will be showcasing practical examples as part of a series soon.

Stay tuned!


NewDay → Shift-Left

Here at NewDay, we are on our journey to shift-left as we scale, it’s not a short journey, but it will be an enjoyable one, one where we come together not just as individual teams, but also as a community of engineers to help drive forward-thinking, innovative change.

The shift-left philosophy will bring unprecedented value to engineers and our customers lives.

NewDay embraces a culture of team autonomy, ownership, responsibility and collaboration. We always aim to do the right thing and pull together, taking on the challenge to aspire to the extraordinary and adapting as we build for the future.

Discussion (0)

pic
Editor guide