Simplifying Keycloak Configuration with Terraform and Terragrunt

Keycloak, an open-source identity and access management solution, provides robust authentication and authorization services for modern applications. However, configuring Keycloak instances manually can be tedious and error-prone. In this blog post, we'll explore how to simplify Keycloak configuration using Terraform and Terragrunt, enabling infrastructure as code (IaC) practices for managing Keycloak realms, clients, users, and more.

---

Why Terraform and Terragrunt?

Terraform is an open-source tool that lets you manage your infrastructure as code. With Terraform, you can define your infrastructure in simple configuration files, and Terraform will automatically create, manage, and update your infrastructure according to those configurations.

Terragrunt is a thin wrapper for Terraform that provides extra tools for keeping your Terraform configurations DRY, managing remote state, and working with multiple Terraform modules. It helps you maintain clean, DRY, and repeatable Terraform code by providing extra functionality and best practices.

By leveraging both, we can have easy:

• Infrastructure as Code (IaC): By defining Keycloak configuration as code, we can version control, review changes, and ensure consistency across environments.
• Modularization: Modularize our Keycloak configuration, making it easier to manage complex setups.
• State Management: Manage the state of our infrastructure, preventing configuration drift and ensuring that our infrastructure remains in the desired state.

---

Let us go!

Install Terraform and Terragrunt

Make sure you have Terraform and Terragrunt installed on your machine. You can find installation instructions on the official Terraform and Terragrunt documentation

• Terraform: https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli
• Terragrunt: https://terragrunt.gruntwork.io/docs/getting-started/install/

Project Structure

Creating a one-size-fits-all structure for a Terraform project can be challenging because it largely depends on the specific requirements of each project. Below is the structure I've found most suitable for organizing Keycloak components, concepts, and setup.

.
├── README.md                 <- The project overview
├── .tool-versions            <- Used tools versions (managed by asdf. see https://asdf-vm.com) 
├── README.md                 <- The project overview
├── modules                   <- Terraform modules
|   └── common
│       ├── provider.tf
│       └── variables.tf
│       └── output.tf
│       └── main.tf
│       └── README.md
│       └── docs/
|   └── clients
│       ├── provider.tf
│       └── variables.tf
│       └── output.tf
│       └── main.tf
│       └── README.md
│       └── docs/
|   └── my-realm
│       ├── provider.tf
│       └── variables.tf
│       └── output.tf
│       └── main.tf
│       └── README.md
│       └── docs/
|   └── other
│       ├── provider.tf
│       └── variables.tf
│       └── output.tf
│       └── main.tf
│       └── README.md
│       └── docs/
└── how-to                    <- Documentation
└── stage                     <- Terraform for environment stage
    ├── .terraform.lock.hcl   <- Terraform lock file
    └── terragrunt.hcl        <- Terragrunt file
    └── env.yaml              <- environment related variables
    └── main.tf               <- environment modules
└── prod                      <- Terraform for environment prod
    ├── .terraform.lock.hcl   <- Terraform lock file
    └── terragrunt.hcl        <- Terragrunt file
    └── env.yaml              <- environment related variables
    └── main.tf               <- environment modules

└── local                     <- Terraform for environment local
    ├── .terraform.lock.hcl   <- Terraform lock file
    └── terragrunt.hcl        <- Terragrunt file
    └── env.yaml              <- environment related variables
    └── main.tf               <- environment modules

In this project structure, I've included a modules directory containing a set of modules shared across all environments. Each module includes a main.tf file encapsulating the module's resources, along with input.tf, output.tf, and variable.tf files for easy configuration across different environments.

Common Module

Let's assume that in the common module, we configure realm events by using the jboss-logging event listener with some non-default configurations. Below is an example of how the main.tf file may look:

resource "keycloak_realm_events" "realm_events" {
  realm_id                     = var.realm_id
  events_enabled               = true
  events_expiration            = 1800
  admin_events_enabled         = true
  admin_events_details_enabled = true
  ]

  events_listeners = [
    "jboss-logging"
  ]
}

To include the realm_id variable, it must be defined in the variables.tf file as shown below:

variable "realm_id" {
  description = "Realm ID"
  type        = string
}

And we must configure the used providers in our module. In this case, the provider.tf will look like as:

terraform {
  required_providers {
    keycloak = {
      source = "mrparkers/keycloak"
    }
  }
}

Master Realm Module

In the realm-master module, the main.tf file should reference the common module. Here is an example of how the main.tf file may look:

data "keycloak_realm" "master" {
  realm = "master"
}

module "realm-master" {
  source = "../../modules/common"
  realm_id = data.keycloak_realm.master.id
}

Similarly, the provider.tf file should be configured as follows:

terraform {
  required_providers {
    keycloak = {
      source = "mrparkers/keycloak"
    }
  }
}

Local environment

In the main.tf file, we need to define the master realm in order to reference the realm-master module in our project.

# Define master realm
module "realm-master" {
  source = "../modules/realm-master"
}

Within each environment (e.g., prod, stage and local), there's an env.yaml file containing all the environment-specific variables.

For example, the env.yaml file for the local environment may look like this:

---
environment: local
url: http://localhost:8080/keycloak

And of course, don't forget to include the terragrunt-local.hcl file, which should be defined in the parent module.

include "root" {
  path = find_in_parent_folders("terragrunt-local.hcl")
}

Terragrunt configuration

As mentioned earlier, Terragrunt is highly beneficial for keeping your configuration Don't Repeat Yourself (DRY). For example, the terragrunt.hcl file may look like this:

# Generates the backend for all modules.
remote_state {
  backend = "s3"
  config  = {
    encrypt        = true
    key            = "keycloak/${path_relative_to_include()}/terraform.tfstate"
    region         = "<AWS REGION>"
    bucket         = "terraform-states"
    dynamodb_table = "terraform-lock"
  }
}

# Read the local "env.yaml" in every environment.
locals {
  vars        = yamldecode(file("${path_relative_to_include()}/env.yaml"))
  environment = local.vars.environment
  url         = local.vars.url
}

# Generate the "provider.tf" file for every module.
generate "provider" {
  path      = "provider.tf"
  if_exists = "overwrite"
  contents  = <<EOF
terraform {
  required_providers {
    keycloak = {
      source  = "mrparkers/keycloak"
      version = "4.4.0"
    }

    http = {
      source  = "hashicorp/http"
      version = "3.2.1"
    }
  }
}

data "http" "config" {
  url = "<INTERNAL CONFIGURATION URL>"
}

provider "keycloak" {
  client_id     = jsondecode(data.http.config.response_body).terraform-client-id
  client_secret = jsondecode(data.http.config.response_body).terraform-client-secret
  url           = "${local.url}"
}

EOF
}

# Generate the "backend.tf" file for every module.
generate "backend" {
  path      = "backend.tf"
  if_exists = "overwrite"
  contents  = <<EOF
terraform {
  backend "s3" {}
}
EOF
}

If you're familiar with Terragrunt, you'll find the terragrunt.hcl file above quite familiar, except for the http config part, which I'll describe later in section.

In remote_state, we use AWS S3 to store the state of our environment configurations. Additionally, we generate the backend and provider for every environment.

The locals block is mainly used to read the env.yaml file and assign the variables to local variables.

Lastly, the http config is responsible for making an HTTP call to wherever you securely store your environment configurations (at least the Terraform client ID and secret) to pass them to the keycloak provider.

The terragrunt-local.hcl file is similar to the terragrunt.hcl file, except for the remote state configuration, which isn't necessary in this case. The local file is primarily for testing your configuration on a local Keycloak cluster setup.

---

The Docker Compose

To run our Terraform configurations, we require a Keycloak cluster setup. In below, we use Docker Compose to start a Keycloak cluster, enabling us to run our local Terraform configurations seamlessly against it.

version: '3'

volumes:
  postgres_data:
      driver: local

services:
  postgres:
      image: postgres
      volumes:
        - postgres_data:/var/lib/postgresql/data
      environment:
        POSTGRES_DB: keycloak
        POSTGRES_USER: keycloak
        POSTGRES_PASSWORD: password
      ports:
        - "5432:5432"
  keycloak:
      image: quay.io/keycloak/keycloak:23.0.6
      environment:
        KC_DB_USERNAME: keycloak
        KC_DB_PASSWORD: password
        KC_DB_URL_HOST: postgres
        KC_DB: postgres
        KC_DB_SCHEMA: public
        KC_HTTP_RELATIVE_PATH: /keycloak
        KC_HOSTNAME_ADMIN: 127.0.0.1
        KC_HOSTNAME: localhost
        KEYCLOAK_ADMIN: admin
        KEYCLOAK_ADMIN_PASSWORD: admin
      command:
        - start-dev
      ports:
        - "8080:8080"
        - "8787:8787"
      depends_on:
        - postgres
  config_keycloak:
    image: ubuntu
    volumes:
      - ./keycloak-docker-config.sh:/opt/keycloak-docker-config.sh
    command: ./opt/keycloak-docker-config.sh
    depends_on:
      - keycloak

The keycloak-docker-config.sh script is primarily used to configure a Terraform client with admin privileges, which Terraform will use during its operations.

#!/bin/bash

apt update -y && apt -y install jq curl

until $(curl --output /dev/null --silent --head --fail http://keycloak:8080/keycloak); do
    printf '.'
    sleep 5
done

# Get access token
TOKEN=$( \
  curl -X POST \
    -H "Content-Type: application/x-www-form-urlencoded" \
    -d "client_id=admin-cli" \
    -d "username=admin" \
    -d "password=admin" \
    -d "grant_type=password" \
    "http://keycloak:8080/keycloak/realms/master/protocol/openid-connect/token" | jq -r '.access_token')

# Create Terraform client (terraform/terraform)
curl -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{"clientId": "terraform", "name": "terraform", "enabled": true, "publicClient": false, "secret": "terraform", "serviceAccountsEnabled": true}' \
  "http://keycloak:8080/keycloak/admin/realms/master/clients"

# Get the Terraform service account user ID
USER_ID=$( \
  curl -X GET \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer ${TOKEN}" \
    "http://keycloak:8080/keycloak/admin/realms/master/users?username=service-account-terraform" | jq -r '.[0].id')

# Get the admin role ID
ROLE_ID=$( \
  curl -X GET \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer ${TOKEN}" \
    "http://keycloak:8080/keycloak/admin/realms/master/roles" | jq -r '.[] | select(.name == "admin") | .id')

# Add the admin role to the Terraform service account user
curl -kv -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '[{"id":"'"${ROLE_ID}"'", "name":"admin"}]' \
  "http://keycloak:8080/keycloak/admin/realms/master/users/$USER_ID/role-mappings/realm"

To run it, open the terminal and run the below docker compose command

docker-compose up --build

After the docker-compose containers are up and running, navigate to http://localhost:8080/keycloak and log in using admin/admin as credentials. Ensure that the Terraform client is configured within the master realm.

Now it's time to run your Terraform local configurations. Open your terminal and execute the following commands:

# Navigate to the local environment
$ cd local 
# Ensure that Terraform-related files, including the auto-generated backend.tf and provider.tf, are removed
$ rm -r backend.tf provider.tf terraform.tfstate terraform.tfstate.backup .terraform.lock.hcl .terraform 
# Initialize Terraform to create all necessary files
$ terragrunt init --terragrunt-config terragrunt-local.hcl 
# Apply the Terraform configurations
$ terragrunt apply --terragrunt-config terragrunt-local.hcl 

Now, open your web browser and go to http://localhost:8080/keycloak. Log in using your admin credentials and make sure your configurations are properly set up there!

Conclusion

Organizing and managing Keycloak configurations with Terraform can greatly streamline your development process. By following the structure and steps outlined in this guide, you can efficiently set up and maintain your Keycloak environments, ensuring consistency and scalability across your projects. If you have any questions or suggestions, feel free to leave them in the comments below.

I hope you find it useful!