Adoption of serverless technology is surely growing, with emerging product companies who built on top of it. CloudZero recently wrote a detailed
article about their experience in achieving continuous delivery for their all-serverless system.
Serverless is a new paradigm in which we need to neither administer nor scale the servers that run our code. The promise is that much less operations work is needed to deliver value to customers.
The catch is that the unit of code deployment is now a function, not an application of arbitrary complexity. Deployment also includes configuring managed services and infrastructure to which functions connect. This means that new tools and practices are needed to achieve optimal developer productivity, which, as always, revolves around being able to move fast. As Ben Peterson of CloudZero notes, "our goal is to be less than five minutes away from shipping to production at all times". Here's how this is achieved:
1/ Replicate conditions on your serverless cloud provider in the development and test environments. For example, LocalStack uses Docker to provide AWS cloud APIs (DynamoDB, S3, Kinesis and others) on
localhost.
2/ Use a framework to manage code deployment. For example, the Serverless framework provides a single workflow for configuring and deploying applications on any cloud vendor. AWS customers can use SAM to define applications in code. Serverless applications don't have separate "build" and "deploy" steps, since there's no artifact that developers need to manage — this is handled by the framework, and your CI/CD tool should support this workflow natively.
3/ Securely store cloud keys and other secrets. Since deployment should be automated, the CI/CD tool needs to support this as well. For example, an AWS key shouldn't necessarily be accessible to everyone in the company, and certainly not to the whole internet. If your project is open source, continuous integration is for general public, delivery is private for the company. For example, Semaphore provides first-class support for storing and managing organization-wide secrets, which can be reused across projects and mixed with team permissions.
4/ Automated testing remains a key part of the workflow. Common JavaScript testing tools for front-end, and Python / Go / Java tools for backend functions apply. Serverless functions are small, and automated tests should run very quickly.
5/ Embrace polyglot programming. For example, CloudZero developers' language of choice is Python, but they also build data pipelines around AWS Kinesis using Kotlin and Gradle.
6/ Keep validation of the codebase (CI) separate from delivery (CD). As Ben Peterson notes:
Like our technology stacks, the set of applications we manage is also diverse, and that includes the development workflow and requirements for deployment to our AWS accounts. Semaphore helps out by providing flexibility along these axes - support for private repos and the ability to manage public repositories and contributor forks, have different logic for deployment of different branches, keep deployment details private, and deploy manually from certain branches. Support for different deployment configurations for each branch is also very useful.
For a more in-depth, practical dive into the specifics of developing and deploying serverless applications, read the full article on the CloudZero blog.
What are your tips for effective CI/CD with serverless? Feel free to share in comments.
This post originally appeared on Semaphore blog.
Top comments (4)
The C# Lambda I have where the build uses the aws cli to package the zip as an artifact begs to differ.
If you're managing that artifact directly then maybe you're not using a framework to delegate that job?
I wrote the bit to deploy, using cake (serverless using the AWS definition), someone else is porting to serverless (serverless.io). There's still the build step, even if the artifacts become 100% handled by serverless.io.
Here are some situations: