DEV Community

Daniel Quackenbush
Daniel Quackenbush

Posted on • Originally published at danquack.dev

6

Terraform Ephemeral Resources

Storing sensitive data like database passwords and API keys in Terraform state files poses significant security risks. Terraform 1.10 introduced the ephemeral resource to address this issue, allowing dynamic retrieval of sensitive values during runtime without persisting them in the state file.

Resources

Ephemeral resources are defined using a new block type in HCL:

ephemeral "<resource type>" "<resource name>" {
  # attributes and nested blocks
}
Enter fullscreen mode Exit fullscreen mode

These resources are referenced similarly to data sources but are not stored in the state file. Attributes and nested blocks will vary per provider implementation; however, they are referenced similarly to data sources.

Example: Using AWS Secrets Manager with Postgres

In this example, the database credentials are retrieved from AWS Secrets Manager and used to configure the PostgreSQL provider without being stored in the state file.

data "aws_secretsmanager_secret" "example" {
    name = "myprodsecret"
}

ephemeral "aws_secretsmanager_secret_version" "db" {
  secret_id = data.aws_secretsmanager_secret.example.arn
}

locals {
  credentials = jsondecode(ephemeral.aws_secretsmanager_secret_version.db.secret_string)
}

provider "postgresql" {
  host     = "1.2.3.4"
  port     = "5432"
  username = local.credentials["username"]
  password = local.credentials["password"]
}

data "postgresql_tables" "my_tables" {
  database = "postgres"
}

output "tables" {
  value = data.postgresql_tables.my_tables.tables.*.object_name
}
Enter fullscreen mode Exit fullscreen mode

If we examine the items in the state, there is no reference to aws_secretsmanager_secret_version.

# terraform state list
data.aws_secretsmanager_secret.example
data.postgresql_tables.my_tables
Enter fullscreen mode Exit fullscreen mode

Supported Providers

At the time of writing, the current supported ephemeral blocks are:

Variables and Outputs

Variables can be declared as ephemeral with ephemeral = true.

variable "api_key" {
  type        = string
  ephemeral   = true
  description = "an api key not stored in state"
}
Enter fullscreen mode Exit fullscreen mode

Outputs in non-root modules can also be declared as ephemeral:

Example: Using Ephemeral Outputs in Modules

output "database_secret" {
  ephemeral = true
  value     = aws_secretsmanager_secret_version.db.secret_string
}
Enter fullscreen mode Exit fullscreen mode
module "database" {
  source = "./database"
}

locals {
  credentials = jsondecode(module.database.database_secret)
}

provider "postgresql" {
  host     = data.aws_db_instance.example.address
  port     = data.aws_db_instance.example.port
  username = local.credentials["username"]
  password = local.credentials["password"]
}
Enter fullscreen mode Exit fullscreen mode

Looking at the state again, there is still no reference to aws_secretsmanager_secret_version:

# terraform state list
data.postgresql_tables.my_tables
module.database.data.aws_secretsmanager_secret.example
Enter fullscreen mode Exit fullscreen mode

Benefits of Ephemeral Resources

  • Enhanced Security: Sensitive data is not stored in state files, reducing the risk of unauthorized access.
  • Dynamic Retrieval: Values are fetched at runtime, ensuring up-to-date information.
  • Compliance: Helps meet regulatory requirements by keeping secrets out of long-term storage.
  • Simplified Secret Management: Integrates with secret management solutions like AWS Secrets Manager.

Terraform's ephemeral feature significantly improves security posture by keeping sensitive data out of state files. By leveraging this feature, teams can maintain the benefits of infrastructure as code while adhering to security best practices. For more detailed information, refer to the Hashicorp documentation on ephemeral resources.

Image of Datadog

The Essential Toolkit for Front-end Developers

Take a user-centric approach to front-end monitoring that evolves alongside increasingly complex frameworks and single-page applications.

Get The Kit

Top comments (1)

Collapse
 
jorgecontreras profile image
Jorge Contreras

Interesting approach! Thanks for sharing

Image of Timescale

Timescale – the developer's data platform for modern apps, built on PostgreSQL

Timescale Cloud is PostgreSQL optimized for speed, scale, and performance. Over 3 million IoT, AI, crypto, and dev tool apps are powered by Timescale. Try it free today! No credit card required.

Try free

👋 Kindness is contagious

Please leave a ❤️ or a friendly comment on this post if you found it helpful!

Okay