DEV Community

Piotr Gwiazda
Piotr Gwiazda

Posted on

Main-module approach for handling multiple environments in Terraform

Handling multiple environments in Terraform might be challenging and you can find many approaches and best practices on the Internet. Ths "main-module" approach is one of them using just built-in Terraform concepts (as an alternative to Terraform workspaces with Terraform Cloud/Enterprise or using wrapper tools like Terragrunt).

But why not just use “tfvars” files?
Because you cannot use variables in backend and provider configuration blocks.


  • Have separate backends for Terraform remote state-files per environment
  • Be able to use separate system accounts for different environments
  • Be able to use different versions of providers and Terraform itself per environment (and upgrade one by one)
  • Ensure that all required properties are provided per environment (Terraform validate won't pass if an environmental property is missing)
  • Ensure that all resources/modules are always added to all environments. It is not possible to "forget" about a whole module.

Here is a basic structure of a Terraform project with the main-module approach that worked for me and few other teams:

- terraform_project
  - env
     - dev01        <-- Terraform home, run from here 
        - .terraform    <-- git ignored of course
        -  <-- backend, env config, includes always _only_ the main module
     - dev02
        - .terraform
     - stg01
        - .terraform
     - prd01
        - .terraform
  - main        <-- main umbrella module
  - modules         <-- submodules of the main module
     - module_a
     - module_b
     - module_c
Enter fullscreen mode Exit fullscreen mode

And a real-life example screenshot:
Alt Text

Each env subdir is a Terraform "home".
Run terraform init/plan/apply in these directories. Change the environment by changing the directory. This prevents an error of merging state files of multiple environments.

Sample file (part):

provider "azurerm" {
  version = "~>1.42.0"

terraform {
  backend "azurerm" {
    resource_group_name  = "tradelens-host-rg"
    storage_account_name = "stterraformstate001"
    container_name       = "terraformstate"
    key                  = "dev.terraform.terraformstate"

module "main" {
  source               = "../../main"
  subscription_id      = "000000000-0000-0000-0000-00000000000"
  project_name         = "tlens"
  environment_name     = "dev"
  resource_group_name  = "tradelens-main-dev"
  tenant_id            = "790fd69f-41a3-4b51-8a42-685767c7d8zz"
  location             = "West Europe"
  developers_object_id = "58968a05-dc52-4b69-a7df-ff99f01e12zz"
  terraform_sp_app_id  = "8afb2166-9168-4919-ba27-6f3f9dfad3ff"

  kubernetes_version      = "1.14.8"
  kuberenetes_vm_size     = "Standard_B2ms"
  kuberenetes_nodes_count = 4

  enable_ddos_protection = false
  enable_waf             = false

Enter fullscreen mode Exit fullscreen mode

This gives a possibility to configure backend and provider per environment as well as provide all required parameters to the main module (i.e. the environment)


This is the main module that is an umbrella module for the whole Terraform project.

Having this module ensures that each environment will be as similar as possible to other environments. Adding a new resource or module to the main module will add this to all environments.

In order to implement differences between environments (pricing tiers etc.) use flags and parameters.

The file lists the required parameters for an environment. Environment-specific settings.

Sample file (part)

variable "subscription_id" {
  description = "Azure Subscription ID"
  type        = string

variable "project_name" {
  description = "Project name"
  type        = string

variable "environment_name" {
  description = "Environment name"
  type        = string

variable "resource_group_name" {
  description = "Target resource group name"
  type        = string


Enter fullscreen mode Exit fullscreen mode

The main module can include submodules and resources. However, it is a good practice to divide it into a few submodules to introduce some encapsulation, increase readability and limit dependencies between resources.

Any shared remote resources will be included here.

A sample file (part):

module "eventhub" {
  source              = "../modules/eventhub"
  environment_name    = var.environment_name
  resource_group_name = var.resource_group_name
  project_name        = var.project_name

module "dremio" {
  source                   = "../modules/dremio"
  environment_name         = var.environment_name
  resource_group_name      = var.resource_group_name
  project_name             = var.project_name
  linux_admin              = "dremio"
  linux_admin_password     = "1b4d7c79-a143-45e2-8ff1-ee63c0823090"
  dremio_data_name         = module.dremio_storage.dremio_data_name
  dremio_data_id           = module.dremio_storage.dremio_data_id
  dremio_data_disk_size_gb = module.dremio_storage.dremio_data_disk_size_gb

module "dremio_storage" {
  source              = "../modules/dremio-storage"
  resource_group_name = var.resource_group_name
  project_name        = var.project_name

module "datalake" {
  source              = "../modules/datalake"
  resource_group_name = var.resource_group_name
  project_name        = var.project_name
  environment_name    = var.environment_name

Enter fullscreen mode Exit fullscreen mode

This should be enough to visualize the concept.

Top comments (5)

r0bnet profile image

Nice article. In our projects it looks really very similar and we're using the same/comparable approach. It works fine for almost every situation.
The only problem we have is if we have bigger differences between environments. I know that there shouldn't be much differences except for example scaling and number of machines in a cluster etc. But sometimes we have one or the other resource which should only be deployed to prod for example. We're more or less fighting with count = 0 but that's a nasty workaround. Hope that there will be conditional modules sooner or later. Maybe i should take a look at Pulumi.

thgsiddhimorajkar profile image

Hello, I am following this method to manage multiple environments. But when i give the source to the module the way it is done here, i get an error - "Error: Unreadable module directory

Unable to evaluate directory symlink: CreateFile modules: The system cannot
find the file specified.

Error: Failed to read module directory

Module directory does not exist or cannot be read."

What might be the issue?

artisticcheese profile image
GSA • Edited

Same here, some instructions are missing. You shall ru your code with terraform -chdir="./clients/client1" init, by passing directory while execution from root folder, not actually switching to individual client directory

eliises profile image

I'm using something similar, but the biggest issue I've got is managing the outputs. I've got about 30, which leads to lots of duplication. Do you have ideas?

piotrgwiazda profile image
Piotr Gwiazda

@eliises for some reason this did not notify me about comments. I just don't use outputs. Most of the time one terraform layer 'leaves' some information in Kay Vault etc, and another layer picks it using "data".