Migrating a Terraform Module to Ephemeral Resources Without Breaking Existing Users

Terraform's ephemeral resources are a genuinely useful addition — secrets that are used during an apply but never written to state.

If you're a module maintainer, adopting ephemerals isn't as simple as swapping one block for another. You have existing users. Their state files already have the old resource tracked. Changing to an ephemeral version of the resource will blow away their managed resource.

To illustrate the issue, we'll pretend to be updating a module that generates a TLS private key and stores it in a Vault KV secret. This post documents the problem and a solution with an extra tid-bit that I learned.

---

The Setup

The original module was straightforward:

resource "tls_private_key" "legacy" {
  algorithm = "RSA"
  rsa_bits  = 4096
}

resource "vault_kv_secret_v2" "legacy" {
  mount     = "kvv2"
  name      = "mytls"
  data_json = jsonencode({ private_key = tls_private_key.legacy.private_key_pem })
}

The private key ends up in state. Vault KV ends up with a copy. Works fine, but the key is visible to anyone with state access.

The goal: keep this working for existing users, while letting new users opt into ephemeral key generation — where the key is used to write to Vault but is never persisted to state.

For solution TL;DR see The Complete Solution below

---

The Naive Approach (and Why It Fails)

The obvious move is a variable toggle with a shared local to unify both paths:

variable "use_ephemeral_key" {
  type    = bool
  default = false
}

resource "tls_private_key" "legacy" {
  count     = var.use_ephemeral_key ? 0 : 1
  algorithm = "RSA"
  rsa_bits  = 4096
}

ephemeral "tls_private_key" "ephemeral" {
  count     = var.use_ephemeral_key ? 1 : 0
  algorithm = "RSA"
  rsa_bits  = 4096
}

locals {
  private_key_pem = (
    var.use_ephemeral_key
    ? ephemeral.tls_private_key.ephemeral[0].private_key_pem
    : tls_private_key.legacy[0].private_key_pem
  )
}

resource "vault_kv_secret_v2" "legacy" {
  count     = var.use_ephemeral_key ? 0 : 1
  mount     = "kvv2"
  name      = "mytls"
  data_json = jsonencode({ private_key = local.private_key_pem })  
}

Running this produces:

│ Error: Invalid use of ephemeral value
│
│   with vault_kv_secret_v2.legacy,
│   on updated.tf line 34, in resource "vault_kv_secret_v2" "legacy":
│   34:     data_json = jsonencode({ private_key = local.private_key_pem })
│
│ Ephemeral values are not valid for "data_json", because it is not a
│ write-only attribute and must be persisted to state.

Why This Happens

The fundamental rule is: ephemeral values can only flow into write-only attributes. data_json must be persisted to state, so it can never accept an ephemeral value — full stop. That's true whether the ephemeral value arrives directly or through a local.

The shared local makes this worse in a subtle way. Terraform's ephemeral taint propagation is static, not runtime. Because one branch of the local.private_key_pem conditional references an ephemeral value, the entire local is marked ephemeral — regardless of which branch actually executes at runtime. So even if you reasoned "when use_ephemeral_key = false, the local resolves to a non-ephemeral value," Terraform doesn't do branch-aware type narrowing. The local is tainted, and the error fires before it even reaches data_json.

The fix is to remove the shared local entirely and have each vault resource reference its own source directly, keeping ephemeral and non-ephemeral values in completely separate expressions:

# Legacy vault write — only ever touches the non-ephemeral resource
resource "vault_kv_secret_v2" "legacy" {
  count     = var.use_ephemeral_key ? 0 : 1
  mount     = "kvv2"
  name      = "mytls"
  data_json = jsonencode({ private_key = tls_private_key.legacy[0].private_key_pem })
}

# Ephemeral vault write — data_json_wo is write-only, so ephemeral values are allowed
resource "vault_kv_secret_v2" "ephemeral" {
  count        = var.use_ephemeral_key ? 1 : 0
  mount        = "kvv2"
  name         = "mytls"
  data_json_wo = jsonencode({ private_key = ephemeral.tls_private_key.ephemeral[0].private_key_pem })
}

Rule to internalize: ephemeral values can only ever terminate at write-only attributes. Don't route them through shared locals or any expression that also serves a non-write-only path.

---

The Second Problem: moved Blocks

Most modules — including this one — were not originally written with count on these resources. Adding count changes the resource address in state:

Before	After
tls_private_key.legacy	tls_private_key.legacy[0]
vault_kv_secret_v2.legacy	vault_kv_secret_v2.legacy[0]

Terraform treats these as different resources. Without intervention, it will destroy the old address and create a new one — generating a brand new private key in the process. For existing users, that's an unannounced key rotation.

The fix is a moved block for each affected resource:

moved {
  from = tls_private_key.legacy
  to   = tls_private_key.legacy[0]
}

moved {
  from = vault_kv_secret_v2.legacy
  to   = vault_kv_secret_v2.legacy[0]
}

These blocks tell Terraform to update the state address in place — no destroy, no create, no key rotation. On the next plan, existing users will see:

# tls_private_key.legacy has moved to tls_private_key.legacy[0]
# vault_kv_secret_v2.legacy has moved to vault_kv_secret_v2.legacy[0]

Leave the moved blocks in for at least one release cycle so users who haven't applied yet still get a clean migration. They can be removed in a future major version once the old address no longer exists in any user's state.

---

The Complete Solution

moved {
  from = tls_private_key.legacy
  to   = tls_private_key.legacy[0]
}

moved {
  from = vault_kv_secret_v2.legacy
  to   = vault_kv_secret_v2.legacy[0]
}

variable "use_ephemeral_key" {
  description = "Use ephemeral key generation (key never written to state). New deployments should set true."
  type        = bool
  default     = false
}

# Legacy path — key stored in state
resource "tls_private_key" "legacy" {
  count     = var.use_ephemeral_key ? 0 : 1
  algorithm = "RSA"
  rsa_bits  = 4096
}

# Ephemeral path — key never written to state
ephemeral "tls_private_key" "ephemeral" {
  count     = var.use_ephemeral_key ? 1 : 0
  algorithm = "RSA"
  rsa_bits  = 4096
}

# Legacy vault write — data_json persists to state (acceptable, key is already in state)
resource "vault_kv_secret_v2" "legacy" {
  count     = var.use_ephemeral_key ? 0 : 1
  mount     = "kvv2"
  name      = "mytls"
  data_json = jsonencode({ private_key = tls_private_key.legacy[0].private_key_pem })
}

# Ephemeral vault write — data_json_wo is write-only, key never touches state
resource "vault_kv_secret_v2" "ephemeral" {
  count        = var.use_ephemeral_key ? 1 : 0
  mount        = "kvv2"
  name         = "mytls"
  data_json_wo = jsonencode({ private_key = ephemeral.tls_private_key.ephemeral[0].private_key_pem })
}

---

Versioning Strategy

With this in place, the recommended release approach is:

• Current minor version: ship the toggle with default = false. Existing users are unaffected. New users can opt in with use_ephemeral_key = true.
• Next major version: flip the default to true. Document the change. Users upgrading from the old version need to either set use_ephemeral_key = false explicitly to preserve legacy behavior, or accept that a new key will be generated on their next apply.

The moved blocks handle the state address migration for everyone regardless of which path they're on.