DEV Community

Cover image for Aiven Databases and Terraform for Fun and Profit
Jonatas Baldin
Jonatas Baldin

Posted on

Aiven Databases and Terraform for Fun and Profit

Originally posted at Aiven's blog.


In the dawn of prehistory, infrastructure teams managed their systems by physically logging into each machine. Nowadays, with Terraform, infra teams can make changes to entire datacenters at once with a few lines of configuration code, in a versionable, reproducible, and predictable way.

For example, you can use the Aiven's CLI to deploy a PostgreSQL database on GCP:

$ avn service create --service-type pg --plan startup-4 --cloud google-europe-west3 postgresql
Enter fullscreen mode Exit fullscreen mode

And you can achieve the same with Terraform code:

resource "aiven_pg" "postgresql" {
  project                = "my-project-name"
  service_name           = "postgresql"
  cloud_name             = "google-europe-west3"
  plan                   = "startup-4"
}
Enter fullscreen mode Exit fullscreen mode

Luckily, Aiven develops and maintains its own Aiven Terraform Provider. With it, users can manage their entire open source data infrastructure with Terraform.

Today, our challenge is to get a PostgreSQL database up and running. Let's get to it. Here's the high-level process:

Terraform can be very picky about the resources it manages. If you create an Aiven service with Terraform, avoid changing it manually through other tools, like the Aiven Console or CLI.

1. Configure your Aiven account

First, you will need an Aiven API access token to authenticate with Terraform.

Log in to the Aiven platform. If you don't have an account, sign up and take advantage of our free trial. It's worth it, I promise.

Then you can follow our documentation to generate your authentication token.

Also, note down your Project Name (located on the upper left side of the page).

Project Name Window

2. Set up Terraform

Go to the Terraform page and download the latest Terraform CLI for your platform – you're going to need version v0.13.0 or newer. You can check your installation by running the following command:

$ terraform version
Terraform v0.15.3
Enter fullscreen mode Exit fullscreen mode

3. Configure the Aiven Provider

Terraform Providers are plugins that connect the power of Terraform with external services and cloud providers. We will now bootstrap our Terraform codebase using the Aiven Terraform Provider. Check the official Aiven documentation for more details on each resource.

Create a new directory for your project and add a file named main.tf with the content below:

variable "aiven_api_token" {
  type = string
}

variable "aiven_project_name" {
  type = string
}

terraform {
  required_providers {
    aiven = {
      source  = "aiven/aiven"
      version = "2.1.12" # check out the latest release in the docs page
    }
  }
}

provider "aiven" {
  api_token = var.aiven_api_token
}

data "aiven_project" "my_project" {
  project = var.aiven_project_name
}
Enter fullscreen mode Exit fullscreen mode

In the two first blocks, we define variables to receive the previously created Aiven access token and Project Name. The next two blocks configure the Provider. Lastly, we use a data source to retrieve all the information regarding the project.

You can also create Aiven Projects with the aiven_project resource.

Create another file named terraform.tfvars to add the Aiven access token and Project Name:

aiven_api_token    = "your_access_token_here"
aiven_project_name = "your_project_name_here"
Enter fullscreen mode Exit fullscreen mode

Finally, initialize Terraform (command output shortened for brevity):

$ terraform init
Enter fullscreen mode Exit fullscreen mode

4. Give me databases!

Alright, enough setup, Let's deploy some databases, starting with Aiven for PostgreSQL.

Beware! Terraform is exceptionally efficient in creating and destroying resources. With databases, every resource removal may result in data loss. In a production environment, read the Terraform plan output carefully before applying it.

First, we will add the aiven_pg resource in the main.tf file:

resource "aiven_pg" "postgresql" {
  project                = data.aiven_project.my_project.project
  service_name           = "postgresql"
  cloud_name             = "google-europe-west3"
  plan                   = "startup-4"

  termination_protection = true

  pg_user_config {
    pg_version = 13
    admin_username = "admin"

    pgbouncer {
      autodb_max_db_connections = 200
    }
  }
}

output "postgresql_service_uri" {
  value     = aiven_pg.postgresql.service_uri
  sensitive = true
}
Enter fullscreen mode Exit fullscreen mode

Here, we have configured the database location, size and parameters. You can find more about clouds, plans and pricing here and all the PostgreSQL related configurations at the official documentation.

Also, the termination_process parameter guarantees that Terraform won't delete the resource until the user sets the flag to false.

At the end, Terraform will output the PostgreSQL connection string.

In your terminal run terraform plan and see what it will create for us:

$ terraform plan
[...]

Terraform will perform the following actions:

  # aiven_pg.postgresql will be created
  + resource "aiven_pg" "postgresql" {
      + cloud_name             = "google-europe-west3"
      + components             = (known after apply)
      + id                     = (known after apply)
      + plan                   = "startup-4"
      + project                = "your_project_name_here"
      + service_host           = (known after apply)
      + service_name           = "postgresql"
      + service_password       = (sensitive value)
      + service_port           = (known after apply)
      + service_type           = (known after apply)
      + service_uri            = (sensitive value)
      + service_username       = (known after apply)
      + state                  = (known after apply)
      + termination_protection = true

      + pg {
          + dbname      = (known after apply)
          + host        = (known after apply)
          + password    = (sensitive value)
          + port        = (known after apply)
          + replica_uri = (sensitive value)
          + sslmode     = (known after apply)
          + uri         = (sensitive value)
          + user        = (known after apply)
        }

      + pg_user_config {
          + admin_username = "admin"
          + pg_version     = "13"

          + pgbouncer {
              + autodb_max_db_connections = "200"
            }
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + postgresql_service_uri = (sensitive value)

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.
Enter fullscreen mode Exit fullscreen mode

The command output tells how the Terraform plan looks like and what changes will be made. Please, for all that is holy, do a sanity check before applying it with the next command:

$ terraform apply
[...]

aiven_pg.postgresql: Creating...
aiven_pg.postgresql: Creation complete after 4m38s [id=my_project/postgresql]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

postgresql_service_uri = <sensitive>
Enter fullscreen mode Exit fullscreen mode

Awesome, our PostgreSQL is up and running. Now let's try to access it with psql using the postgresql_service_uri output:

$ psql "$(terraform output -raw postgresql_service_uri)"
psql (13.2)
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.

defaultdb=>
Enter fullscreen mode Exit fullscreen mode

This is everything you need to have a production-ready PostgreSQL server at GCP, managed by Aiven and deployed with code. Cool, right?!

What's next?

I hope you enjoyed this ride with me. Terraform and infrastructure as code can be daunting at first, but once we see all of its benefits, it's hard to move away from it.

Please check this GitHub repository to see the whole code and any future updates for it.

If you want to learn more about the tools and services used here, make sure to check the links below:

If you have any issues with Aiven's Terraform Provider or want to help us improve it, please check the GitHub repository and the official documentation.

Discussion (0)