A Dry-Run Contract for Safer Automation

Automation becomes useful when it removes boring clicks. It becomes dangerous when a test run can quietly send an email, create a customer, or delete a resource. The gap is usually not a missing framework. It is a missing contract between “preview” and “execute”.

Here is a small pattern I like for developer tools: every operation declares its intent, and the dry-run path produces the same decision data as the real path. Only the final side effect is skipped. This makes automation easier to inspect, replay, and trust.

The hidden cost of a dry run

A weak dry run prints would do something and then follows a seperate code path. Over time, the preview drifts from production behavior. A fake email generator may produce a different address in CI than it does locally, or a deployment preview may ignore a permission check that the real call performs.

The result is false confidence. The green job proves that the preview code ran, not that the real operation is safe.

Define the contract

Start with a result that describes the decision before it describes the side effect:

type OperationPlan struct {
    Action      string
    Resource    string
    FixtureID   string
    Mutations   []string
}

func (p OperationPlan) Receipt() map[string]any {
    return map[string]any{
        "action": p.Action,
        "resource": p.Resource,
        "fixture_id": p.FixtureID,
        "mutations": p.Mutations,
    }
}

The planner can validate inputs, resolve permissions, and choose a stable fixture. An executor then receives that plan. In dry-run mode it records the receipt and stops before the mutation. In live mode it applies the exact same plan.

This separation is a simple mental model: plan once, apply optionally.

A small implementation

func Run(ctx context.Context, plan OperationPlan, dryRun bool) error {
    if err := validate(plan); err != nil {
        return fmt.Errorf("validate operation: %w", err)
    }

    if err := writeReceipt(plan.Receipt()); err != nil {
        return fmt.Errorf("write receipt: %w", err)
    }
    if dryRun {
        return nil
    }
    return applyMutation(ctx, plan)
}

The receipt is written before the mutation, so a failed live operation still leaves evidence of what it intended to do. Keep the fixture ID stable for retries. That makes a rerun less likely to create duplicate records.

For email workflows, use a dedicated test inbox and a generated address per test case. The address should be disposable test data, never a credential or a place to store sensitive messages. Teams often search for a fake e mail com during setup; document the approved fixture source in the repository so that search phrase does not become an accidental production dependency.

Make CI prove the behavior

Have CI run the planner twice: once with dryRun=true, and once against a disposable environment. Compare the receipts, excluding fields that are intentionally nondeterministic, such as timestamps. If the plans differ, fail early.

You can also assert that a dry run makes zero mutation calls:

plan -> validate -> receipt -> [dry run: stop]
                         \-> [live: mutate]

This is particularly handy beside build metadata in rollout emails and one source of truth for email checks. Both ideas reduce the amount of hidden state a pipeline must reconstruct after failure.

What to record

A useful receipt normally contains:

• operation name and version
• actor or service identity
• target resource
• fixture or idempotency key
• validation outcome
• planned mutations
• correlation ID
• timestamp and duration

Do not record raw email bodies, tokens, or passwords. A receipt should answer “what did we try?” without becoming a new data leak. This boundary is easy to miss when adding debug logging quickly.

Final checklist

Before calling an automation flow reliable, check that:

1. Dry run and live mode share the planner.
2. Validation happens before any side effect.
3. Receipts are durable and safe to inspect.
4. Retries use a stable idempotency key.
5. CI compares plans and counts mutation calls.
6. Test email data stays isolated from real users.

The pattern is small, but it changes the conversation around automation. Instead of asking whether a script “seems safe,” you can inspect the contract, review the receipt, and prove what happened. That is a much nicer place for a builder to work from.