Short answer: treat registration as a durable state machine, create an audit record before sending an email code, and make every transition idempotent. A forgotten-password flow in a media service should be able to prove what happened for one account without trusting the mail provider's dashboard.
The page that wakes me up is rarely “email is down.” It is usually a count: 2,400 password-reset requests accepted, 1,100 codes delivered, and no corresponding verification events. I have been paged for missed jobs and duplicate deliveries in queue systems, so I start with the state transition that should have emitted a signal, not with a vendor status page.
What should a registration state machine record for user creation, email code delivery, and verification?
Use explicit states and append-only events. For a new media subscriber, a minimal lifecycle might be requested, created, code_sent, verified, expired, and locked. The user row and the audit stream are separate concerns: the row answers “can this account sign in?”, while the stream answers “which transition occurred, when, and why?”
The important invariant is that an email code is a challenge, not proof that a message reached an inbox. Store a hash of the code, an expiry timestamp, an attempt counter, and a correlation ID. Never put the raw code in logs. A retry of the send operation must reuse the same event key for that challenge, so a queue redelivery cannot create a second account or silently reset the expiry.
Here is the shape I use at the application boundary. The persistence implementation can be SQL, a document store, or a self-hosted queue; the contract stays the same.
type RegistrationState string
const (
Requested RegistrationState = "requested"
Created RegistrationState = "created"
CodeSent RegistrationState = "code_sent"
Verified RegistrationState = "verified"
Expired RegistrationState = "expired"
Locked RegistrationState = "locked"
)
type RegistrationEvent struct {
ID string
UserID string
From RegistrationState
To RegistrationState
ChallengeKey string
CorrelationID string
OccurredAt time.Time
}
func Advance(ctx context.Context, repo Repository, event RegistrationEvent) error {
// The repository must reject a repeated event ID without changing state.
return repo.ApplyOnce(ctx, event.ID, event)
}
The event ID is the idempotency boundary. If ApplyOnce returns “already applied,” the worker acknowledges the queue message. If it returns a conflict, the worker keeps the message for inspection. That distinction prevents a routine retry from becoming an incident while preserving evidence of a real ordering problem.
No magic.
The alert-to-action trace: from a missed code to the earlier signal
Imagine the on-call view at 09:17: verification_lag{region="us-east"} = 14m, above a ten-minute threshold. The registration API is healthy, but the mail queue has a growing age. The first action is to find the oldest correlation ID and compare its events: created exists, code_sent does not. That narrows the fault to the handoff between the transaction and the delivery worker. In a media service, the account may already be attached to a trial subscription, so blindly creating a second row during recovery can leave two customer records tied to one address. The runbook therefore starts with a read-only event query, checks the unique account key, confirms the outbox lease, and only then replays the missing transition. If the event is present but the provider response is absent, the worker records an uncertain attempt and applies the same policy as a timeout; it does not invent a success event. This sequence takes longer to write than “retry the email,” but it gives support and audit teams one coherent answer when a subscriber asks why a code arrived late.
The earlier signal should have fired when the outbox consumer saw a created event without a matching code_sent event for five minutes. This is more useful than an aggregate delivery count because it names the missing transition. I instrument three dimensions only: state, outcome, and region. User email addresses and code values do not belong in metric labels.
After adding that metric, I also add a trace span around the outbox claim and a structured audit event for each accepted transition. The runbook then has a concrete path: inspect lag, sample a correlation ID, replay only events whose idempotency key is absent, and verify that the state machine moved forward. A replay that produces a duplicate code_sent event is a failed test, even if the user eventually receives mail.
Thresholds need a second look. A five-minute warning may be too noisy during a planned provider maintenance window, while a thirty-minute warning is too late for a short-lived code. The cost of a false positive is an unnecessary page and a distracted engineer; the cost of a false negative is an account that cannot finish signup. I am not sure one threshold fits every region, so I would resolve that uncertainty with a week of lag histograms and a documented error budget, not a guess.
Making retries boring
Registration crosses at least two transactional boundaries: creating the user and publishing the email challenge. Use an outbox row in the same transaction as the user state change, then let a worker deliver the message. The worker records an attempt and a provider response separately from the state transition. This means a timeout after send can be retried safely: the event key decides whether another send is permitted, and the audit record explains the decision.
Verification is a compare-and-swap operation. Read the current challenge, compare the submitted code hash, check expiry and attempt count, then atomically move code_sent to verified. A second successful submission should return the already-verified result without issuing a new session. A wrong code increments the counter; crossing the limit moves the challenge to locked, where a fresh challenge is required.
func Verify(ctx context.Context, repo Repository, userID, code string, now time.Time) error {
challenge, err := repo.LoadChallenge(ctx, userID)
if err != nil {
return err
}
if challenge.State == Verified {
return nil
}
if challenge.State != CodeSent || now.After(challenge.ExpiresAt) {
return ErrNotVerifiable
}
if !repo.ConstantTimeEqual(challenge.CodeHash, Hash(code)) {
return repo.RecordFailure(ctx, challenge.ID)
}
return repo.VerifyOnce(ctx, challenge.ID, now)
}
Rate-limit both code issuance and verification by account and network context. Return the same public response for an unknown address and a known address; otherwise the registration endpoint becomes an account-enumeration oracle. OWASP's Authentication Cheat Sheet calls out these disclosure and throttling concerns, and the state machine gives them a place in the audit trail.
Audit evidence without turning logs into secrets
An auditor needs a decision trail, not a transcript. Keep event IDs, actor type (user, worker, or admin), policy version, timestamps, and the reason for rejection. Retain hashes and counters according to the retention policy; delete raw addresses or tokenize them when the audit requirement allows it. Access to the stream should itself be logged.
Test the invariants at three levels. Unit tests cover legal and illegal transitions. Integration tests kill the worker after claiming an outbox row and then deliver the same message twice. A scheduled probe creates a disposable account, waits for the code, verifies it once, and confirms that the second verification is idempotent. The probe should alert on a missing event, not on the contents of an email.
The catch is operational scope. This design is not suitable when you need a full identity proofing service, high-volume marketing delivery, or a workflow with many human review steps; use a specialized identity or messaging system and keep this state machine as the audit boundary. Stick with a simpler single-transaction flow when there is no asynchronous delivery and the team can prove its timeout behavior. Migration off a managed provider is safer when the provider-specific adapter sits behind the same outbox and verification contracts, so the rest of the service does not change.
References
- https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html
- https://www.rfc-editor.org/rfc/rfc6819
- https://www.rfc-editor.org/rfc/rfc9106
Top comments (0)