Bringing declarative infrastructure to developers

Over the past decade, Terraform has revolutionized how operations (Ops) teams manage cloud infrastructure by providing a simple, declarative way to define and provision resources.
Terraform allows us to declare cloud resources using a simple configuration language (HCL), which can then be executed to create or modify resources.

Here’s a simple example of declaring a bucket:

resource "aws_s3_bucket" "example_bucket" {
  bucket = "my-example-bucket"
  acl    = "private"

  tags = {
    Name        = "MyExampleBucket"
    Environment = "Dev"
  }

  versioning {
    enabled = true
  }
}

When you run terraform apply, terraform relies on providers to parse and translate the configuration into API requests. The provider acts as a bridge between Terraform and the AWS API, it constructs the necessary API calls to fulfill the requirements specification and sends them to AWS, where the actual creation of the bucket, tagging, and versioning happens.

Terraform handles all the communication with cloud providers, ensuring that resources are created or updated according to the configuration file, without requiring direct interaction with cloud APIs. But what about developers who also need to work with cloud infrastructure?

Our current workflow generally leads us towards building an application first, then handling the infrastructure.

This means that developers have a few choices:

• Create variables for all unknown configuration and throw the ball over the fence to an ops team
• Work intimately with the ops team to detail resource names, configuration etc.
• Do it themselves with tools they aren’t in love with

None of these options are particularly desirable as they rely on a lot of manual steps which often lead to communication breakdowns and runtime/provision time errors. We need a way of shifting the declaration of resources forward in the workflow without introducing additional complexity to our teams.

Declarative infrastructure for developers

I’m not suggesting that developers don’t need to know how storage buckets, databases or APIs work, they just don’t need to know the specifics of how they will be provisioned while they are writing their application code.

With Infrastructure from Code (IfC), developers can declaratively define and request cloud resources such as APIs, storage buckets, and queues directly within their application logic. This declarative approach allows developers to specify what they want without worrying about how they will be provisioned.

Here’s an example of how this might work, and, actually, this should look extremely familiar, the only addition a declaration of the developers intent - to allow reading and writing on the bucket and this information is the missing link to assign the right permissions for the resources to interact with each other.

import * as nitric from '@nitric/sdk'

// Define a bucket for storing images
const imagesBucket = nitric.bucket('images').allow('read', 'write')

// Define an API for serving images
const mainApi = nitric.api('main')
mainApi.get('/images/:key', async (ctx) => {
  // Your image-serving logic here
})

From this example it's simple enough to recognize that we need the following cloud resources:

• bucket('images').allow('read', 'write'): Creates a cloud storage bucket named "images" and sets access permissions for reading and writing.
• api('main'): Creates an API named "main" with a route to serve images. The developer can then define application logic for handling image requests.
• OpenAPI specification: From the routes configured we can automatically create the configuration required for our API gateways.

We can automatically generate the requirements specification -

{
  "resources": [
    {
      "id": {
        "type": "Bucket",
        "name": "images"
      },
      "bucket": {}
    },
    {
      "id": {
        "name": "main"
      },
      "api": {
        "openapi": "{\"components\":{},\"info\":{\"title\":\"main\",\"version\":\"v1\"},\"openapi\":\"3.0.1\",\"paths\":{\"/images/{key}\":{\"get\":{\"operationId\":\"imageskeyget\",\"responses\":{\"default\":{\"description\":\"\"}},\"x-nitric-target\":{\"name\":\"api-testing_services-hello\",\"type\":\"function\"}},\"parameters\":[{\"in\":\"path\",\"name\":\"key\",\"required\":true,\"schema\":{\"type\":\"string\"}}]}}}"
      }
    }
  ]
}

Bridging Runtime and Provisioning Gaps

One of the major challenges developers face is ensuring that the resources their application depends on are correctly provisioned at runtime. The generated specification ensures that the correct resources (storage buckets, APIs, etc.) are provisioned with the appropriate settings (permissions, paths, etc.). This removes the potential for mismatches between what the application requires and what is provisioned, ensuring a smoother deployment process.

We can now use it to orchestrate the necessary IaC, like the Terraform module above, to provision the required resources. This approach minimizes runtime and provision-time errors, ensuring that resources are automatically provisioned in line with application requirements.

It also means that the same application can be deployed across different cloud providers, which means you can pivot at any time to take advantage of the latest technologies and disruptors like AI.