DEV Community

Cover image for Building an (Actually) Serverless Private Terraform Registry on AWS
Sebastiano Caccaro
Sebastiano Caccaro

Posted on

Building an (Actually) Serverless Private Terraform Registry on AWS

#terraform #aws #devops #serverless

TL;DR I wrote a TF module to deploy a fully serverless Terraform registry on AWS for under $0.50/month. The registry also implements token-based authentication with three permission tiers, proxy mode, and module pinning.
You can get the module here https://registry.terraform.io/modules/sebacaccaro/serverless-module-registry/aws/latest

The problem with distributing private modules

If you've been working with Terraform for a while, you've probably encountered the need to distribute some private modules for teams in your company to use.

The terraform docs suggest using HCP Terraform's private registry feature as a first option. That's a fine option if you don't mind the pricing model and are planning on using the whole suite. Otherwise you're left with an overkill solution and yet another service to manage.

The second option would be to use Git Servers (most commonly GitHub and Bitbucket) or S3 to host your modules. This approach is easy enough, allows for basic authorization, but has some drawbacks.
For instance, versioning is not fully supported: you need to manually select the right version.

# Git Source
module "vpc" {
  source = "github.com/hashicorp/example//modules/vpc?ref=v1.2.0"
  #...
}

# Native Registry Source
module "consul" {
  source = "app.terraform.io/hashicorp/consul/aws"
  version = "~> 1.1.0" # 👈🏼 Notice you can use the ~> version selector
}
Enter fullscreen mode Exit fullscreen mode

Also using Git as a module source is inherently slow and can significantly slow down terraform init.

Evaluating different terraform registry options

So naturally, the only reasonable solution was to implement Terraform's Module Registry Protocol Reference and make it deployable via a Terraform module.
If you read the title, you may also already know I want the registry to be fully serverless and running on AWS. This will ensure the project is fast to deploy, reliable, and free when idle.
Projects that do something similar exist, but:

  • They require some config, or are not so easy to deploy. My aim is to keep the module configuration as slim as possible
  • They are not truly serverless and declare resources that can have some recurring base costs
  • They do not implement authentication

In this post I'll walk through the architecture, the key design decisions, and a few extra features I wanted on top of the basics: authentication, proxy mode, and module pinning.

Architecture

The registry follows a fairly standard pattern for serverless APIs on AWS. API Gateway sits in front, two Lambda functions handle auth and business logic, and the data layer is split between S3 (module archives) and DynamoDB (tokens). The diagram below shows how the pieces connect.

If you've built APIs on AWS before, your first instinct for auth might be Cognito. While it might have worked just fine, I deemed it overkill for this scope. The aim is to serve small teams and require as little maintenance as possible. OAuth flows, user attributes, and other advanced features are simply not needed here.

That was the bird's eye view. Now let's get technical.

Implementing the registry protocol

Terraform's Module Registry Protocol is simpler than you'd expect. To make terraform init work with a custom registry, you only need three things:

  1. Service discovery
  2. Version listing
  3. Module download

Service discovery is how Terraform figures out where the module API lives. When you write source = "registry.example.com/myorg/vpc/aws", the first thing Terraform does is hit https://registry.example.com/.well-known/terraform.json. That file just points to the base path of the modules API:

{
  "modules.v1": "/v1/modules/"
}
Enter fullscreen mode Exit fullscreen mode

From there, Terraform knows to look for modules under /v1/modules/.

Version listing comes next. Terraform calls GET /v1/modules/{namespace}/{name}/{system}/versions and gets back a JSON object with a list of version numbers. Nothing more. This is what lets version constraints like ~> 1.0 work.

Download is where it gets interesting. When Terraform picks a version, it calls GET /v1/modules/{namespace}/{name}/{system}/{version}/download. The response isn't the module itself — it's a 204 No Content with an X-Terraform-Get header pointing to the actual archive URL. Terraform follows that header to get the module.

That indirection is what makes the whole thing cheap. Instead of streaming archives through Lambda, the handler generates a presigned S3 URL and sticks it in X-Terraform-Get. Terraform downloads directly from S3, which keeps Lambda execution times short and avoids paying for data transfer through API Gateway.

In my implementation, there's no database tracking module versions. Modules are stored in S3 following a {namespace}/{name}/{system}/{version} key convention, and the version list is derived directly from the S3 prefixes. This keeps things simple and means you can also upload modules manually to the bucket if you prefer.

sequenceDiagram
    participant TF as Terraform CLI
    participant R as Registry
    participant S3 as S3

    TF->>R: GET /.well-known/terraform.json
    R-->>TF: {"modules.v1": "/v1/modules/"}

    TF->>R: GET /v1/modules/{ns}/{name}/{sys}/versions
    R-->>TF: version list (JSON)

    TF->>R: GET /v1/modules/{ns}/{name}/{sys}/{ver}/download
    R-->>TF: 204 + X-Terraform-Get: presigned S3 URL

    TF->>S3: GET presigned URL
    S3-->>TF: module archive
Enter fullscreen mode Exit fullscreen mode

On the upload side, the protocol doesn't define a standard. I went with a simple PUT /v1/modules/{namespace}/{name}/{system}/{version} that accepts a .tar.gz or .zip archive. If the version already exists, you get a 409.

curl -X PUT \
  "https://registry.example.com/v1/modules/myorg/vpc/aws/1.0.0" \
  -H "Authorization: Bearer your-uploader-token" \
  -H "Content-Type: application/octet-stream" \
  --data-binary @module.tar.gz
Enter fullscreen mode Exit fullscreen mode

User Management and Authentication

With the protocol covered, the next question is who gets to call these endpoints.

The registry uses a token-based system with three tiers: master, uploader, and downloader. Each tier inherits the permissions of the ones below it.

The master token is created automatically at deploy time and stored in Secrets Manager. Uploader and downloader tokens are created and managed via the POST/GET/DELETE /v1/tokens endpoints. Token values are returned only at creation and can't be retrieved afterwards.

Token Download / List Upload Pin Modules Manage Tokens
master ✓ ✓ ✓ ✓
uploader ✓ ✓
downloader ✓

On the Terraform side, you just add a credentials block to your .terraformrc:

credentials "registry.example.com" {
  token = "your-api-token"
}
Enter fullscreen mode Exit fullscreen mode

Proxy mode and module pinning

The registry can also act as a proxy to the public Terraform Registry. When a module is not found locally, the request is forwarded to registry.terraform.io. To enable it, set proxy_enabled = true:

module "registry" {
  source  = "sebacaccaro/serverless-module-registry/aws"

  domain_name     = "registry.example.com"
  certificate_arn = "arn:aws:acm:..."

  proxy_enabled = true
}
Enter fullscreen mode Exit fullscreen mode

You can control what gets proxied with allow and deny lists. Both match as prefixes against the module's namespace/name path. Deny takes precedence. When neither list is set, all public modules are eligible for proxying.

proxy_allow_list = ["hashicorp/", "myorg/vpc"]
proxy_deny_list  = ["internal/"]
Enter fullscreen mode Exit fullscreen mode

The problem with proxying is that you're still depending on the public registry at apply time. If it's down, or if a maintainer yanks a version, your terraform init breaks.

Module pinning solves this. A single API call downloads the archive from the public registry and stores it in your S3 bucket. From that point on, the registry serves the local copy and stops proxying that version entirely.

curl -X POST \
  "https://registry.example.com/v1/pins/hashicorp/consul/aws/0.12.0" \
  -H "Authorization: Bearer your-master-token"
Enter fullscreen mode Exit fullscreen mode

Combined with allow/deny lists, pinning gives platform engineers and DevOps teams a way to control and vet which modules and versions developers can use.

Deploying it

At minimum, the module only needs a custom domain name and an ACM certificate. If you don't need a custom domain, you can skip both and use the API Gateway endpoint directly.

module "registry" {
  source  = "sebacaccaro/serverless-module-registry/aws"

  domain_name     = "registry.example.com"
  certificate_arn = "arn:aws:acm:us-east-1:123456789012:certificate/abcd1234-..."
}

output "api_endpoint"        { value = module.registry.api_endpoint }
output "master_token_secret" { value = module.registry.master_token_secret_arn }
output "s3_bucket"           { value = module.registry.s3_bucket_name }
Enter fullscreen mode Exit fullscreen mode

If you're using a custom domain, point a DNS CNAME or Route 53 alias record at the custom_domain_regional_domain_name output after applying. Then grab the master token from Secrets Manager and you're up.

The full API is documented in the openapi.json file included in the repository, which you can import into any API client.

The only recurring cost when idle is the Secrets Manager secret at ~$0.50/month. Everything else is pay-per-use. For a small team running a few hundred terraform init calls per day, expect to stay well under $5/month.

Wrap-up

The registry currently covers the module part of the Terraform registry API, which is enough for most use cases. There are a few things I'd like to add down the line, like integrating with IAM so users can exchange their AWS credentials for a temporary registry token, instead of relying on static ones.

If you happen to use the module or have any questions, feel free to reach out on GitHub. The module is also on the Terraform Registry.

Cheers!

Top comments (0)