DEV Community

Cover image for How we unified our Terraform module repositories
Tom Masson
Tom Masson

Posted on

How we unified our Terraform module repositories

Ever tried managing 15+ separate GitHub repositories for your Terraform modules? That's pretty much what we faced at Payfit. Our infrastructure codebase had fragmented into a nightmare of individual repos with time, each with its own CI/CD pipeline, versioning scheme, and tooling setup. Cross-module changes became coordination nightmares.

The problem: When repositories multiply like rabbits

Picture this: Your infrastructure team maintains multiple GitHub repos, with each Terraform module kept in its own repository. A simple VPC module change requires you to: update the module repo, bump versions in 5 downstream repos, trigger 6 separate CI pipelines, and spend the day merging PRs.

That was us six months ago.

Our initial approach, a scaling nightmare

Each Terraform module lived in its own repository with dedicated CI pipelines, separate versioning, and isolated tooling. While this provided clear ownership boundaries, it created massive overhead:

  • Version management hell & poor velocity: Bumping versions across dependent modules required synchronized releases and multiple PRs
  • Tooling drift: Different tflint versions and terraform standards per repository
  • Context switching: Teams bounced between repositories constantly

The breakthrough: Nx monorepo consolidation

The turning point came when we consolidated all Terraform modules into a single monorepo. Nx provided the orchestration layer that made this transformation not just possible, but elegant.

Project structure transformed

Each Terraform module becomes an Nx library project under terraform-modules/:

terraform-modules/
├── aws-lambda/        # Nx project: terraform-aws-lambda
├── aws-ecr/          # Nx project: terraform-aws-ecr
└── aws-s3/           # Nx project: terraform-aws-s3
Enter fullscreen mode Exit fullscreen mode

Unified module management

Every Terraform module becomes a first-class Nx project with inferred tasks. Our internal local Nx plugin automatically detects terraform modules and provides standardized targets using Nx's inference system:

  • tofu-format: Consistent formatting across all modules
  • lint: Parallel linting with TFlint and validation

It looks something like this:

targets['tofu-format'] = {
      executor: 'nx:run-commands',
      cache: false,
      inputs: ['production', '^production'],
      outputs: [],
      options: {
        cwd: '{projectRoot}',
        command: 'tofu fmt -recursive',
      },
      configurations: {
        check: {
          args: '-check',
        },
        write: {
          args: '-write',
        },
      },
      defaultConfiguration: 'check',
      metadata: {
        technologies: ['tofu'],
        description: 'Format OpenTofu code',
      },
    }
    targets['lint'] = {
      executor: 'nx:run-commands',
      cache: false,
      inputs: ['production', '^production'],
      outputs: [],
      options: {
        cwd: '{projectRoot}',
        commands: [
          'tofu init -backend=false',
          'tflint --init',
          'tflint',
          'tofu validate',
        ],
      },
      env: {
        TFLINT_CONFIG_FILE: options.tflintConfigFile,
      },
      metadata: {
        technologies: ['tofu'],
        description: 'Lint OpenTofu code',
      },
    }
Enter fullscreen mode Exit fullscreen mode

The beauty? Zero configuration needed. Drop a terraform module in the repo, and Nx handles everything automatically. We will even be able to generate the skeleton of a classic module using Nx's generator system.

Nx Release: unified module publishing

The real transformation happened with Nx release. Our internal @payfit/nx-core plugin orchestrates coordinated releases across all publishable artifacts over @ Payfit, including terraform modules.

The release command analyzes git history and only releases modules that have actually changed:

npx nx release
Enter fullscreen mode Exit fullscreen mode

The result? Cross-module changes deploy together, while individual module updates release independently, in a single PR.

The trade-offs: nothing's perfect

Now, this monorepo approach isn't without its (small) downsides:

Pro Con
Smart Releases: Intelligent orchestration across modules Larger Repository: Single repo grows over time
Parallel Execution: Quality checks run simultaneously Nx Learning Curve: Team needs to understand Nx concepts
Automated Standards: Consistent tooling across all modules Migration Effort: Consolidating 15 repos requires planning & time

For us, the benefits far outweigh these costs. We've dramatically improved our infrastructure velocity and consistency.

Outcomes: from chaos to streamlined Terraform modules management

The impact was immediate and measurable:

  • Unified Management: Single Nx workspace orchestrates all 15+ modules
  • Zero Configuration: Plugin automatically provides all terraform targets
  • Automated Standards: Consistent tooling and validation across all modules
  • Velocity: Single PR needed to update one or several modules simultaneously

The key insight? Treat infrastructure modules like any other software project, with proper versioning, testing, and release management. Nx makes this possible at scale.

Have you faced similar repository chaos? How did you tackle it?

Top comments (0)