Short answer: validate the JWT against a locally cached JWKS, refresh only when the key set says it should, and fail closed for unverifiable tokens while keeping the gateway itself available. In a gaming login flow, that means an expired or unknown signing key can reject one request without turning a phone one-time-code endpoint into a retry storm.
The failure signal is usually a timing problem
A microservice gateway sits between a game client and the service that exchanges a phone one-time code for a session token. The token may be perfectly valid when issued, yet a gateway can reject it during a signing-key rollover because its cache is stale. The inverse is worse: accepting a token after the issuer has retired its key leaves an abuse window.
I have seen this show up as a burst of 401 responses exactly when a scheduled key change began. The first dashboard blamed the OTP provider. A request trace later showed kid=mobile-2026-07 arriving before the gateway had fetched the new set. That detail matters: a 401 with an unknown key is a cache state transition, not proof that the player entered the wrong code.
Keep the signal narrow. Count unknown kid, expired-token, bad-signature, and issuer/audience failures separately. Alert on their rate and on refresh latency. A single counter called auth_failed is not a runbook; it is a dead end.
How should a gateway handle token validation, JWKS retrieval, cache rotation, and failure handling?
Treat the key set as versioned data with an explicit lifecycle. On a request, parse the JWT header without trusting claims, select the matching kid, and verify the signature and registered claims using the cached JWKS. If the kid is absent, perform one bounded refresh for that issuer, then retry verification once. Never fetch on every request.
The refresh path needs its own guardrails:
- Use an expiry time from cache metadata and a short refresh margin so normal traffic does not wait for network I/O.
- Coalesce concurrent misses per issuer. One request refreshes; the others use the result or receive the same failure.
- Accept a new key set only after it parses, contains an allowed algorithm, and passes issuer and key-usage checks. Swap the pointer atomically.
- Keep the previous set briefly for tokens already in flight, but do not extend token lifetime because a stale key remains in memory.
- Apply exponential backoff with jitter after a failed refresh. A failed fetch must not become a fan-out of identical retries.
The cache is a security control and a reliability control. Its freshness policy should be written down beside the issuer configuration, including the maximum age, refresh timeout, and behavior when the issuer is unreachable.
A compact Go shape for the decision point looks like this (the JWKS parser and signature verifier are deliberately injected so the policy can be tested without a network call):
package auth
import (
"context"
"errors"
"fmt"
"sync"
"time"
)
type KeySet interface {
Verify(token string, now time.Time) error
}
type Fetcher interface {
Fetch(ctx context.Context) (KeySet, time.Time, error)
}
type Validator struct {
mu sync.RWMutex
current KeySet
expires time.Time
fetcher Fetcher
refresh chan struct{}
}
func (v *Validator) Validate(ctx context.Context, token string, now time.Time) error {
v.mu.RLock()
keys, expires := v.current, v.expires
v.mu.RUnlock()
if keys != nil && now.Before(expires) {
if err := keys.Verify(token, now); err == nil {
return nil
} else if !errors.Is(err, ErrUnknownKey) {
return err
}
}
if err := v.refreshOnce(ctx); err != nil {
return fmt.Errorf("key refresh: %w", err)
}
v.mu.RLock()
keys = v.current
v.mu.RUnlock()
if keys == nil {
return errors.New("no verification keys")
}
return keys.Verify(token, now)
}
func (v *Validator) refreshOnce(ctx context.Context) error {
select {
case v.refresh <- struct{}{}:
defer func() { <-v.refresh }()
case <-ctx.Done():
return ctx.Err()
}
keys, expires, err := v.fetcher.Fetch(ctx)
if err != nil {
return err
}
if keys == nil || !expires.After(time.Now()) {
return errors.New("invalid key-set response")
}
v.mu.Lock()
v.current, v.expires = keys, expires
v.mu.Unlock()
return nil
}
ErrUnknownKey is the only verification error that should trigger a refresh. Bad signatures, wrong issuer, wrong audience, and expired claims should remain hard rejects. That distinction prevents an attacker from turning arbitrary garbage into outbound key requests.
What does a safe rotation runbook verify?
Before a scheduled rotation, publish the new public key while the old key is still valid. The gateway should observe both kid values, fetch the set, and verify tokens signed by each. After the issuer stops signing with the old key, wait longer than the maximum token lifetime plus clock skew before removing it from the published set. Rotation is a overlap exercise, not a single deploy event.
Use a staging issuer or a captured, non-sensitive JWKS fixture to test four cases: a fresh key, an unknown kid, a malformed key document, and an unreachable issuer. Assert both the HTTP result and the metric labels. For the gaming flow, replay the same session token against two gateway instances and confirm that validation is deterministic; a local cache must not create instance-specific authorization.
One short check belongs in the runbook.
If refresh latency rises, shed refresh work before shedding player traffic.
The gateway should serve requests signed by a still-valid cached key until its documented maximum age. Once that age is exceeded, fail closed for tokens that cannot be verified. Return a generic authentication response to clients, while logging a correlation ID and the issuer, kid, cache age, and failure class internally. Never log the raw token or the phone number.
Rollback is a configuration action: restore the previous issuer metadata or key endpoint, clear only the affected cache entry, and watch unknown-kid and refresh-error rates fall. Do not roll back by accepting any algorithm or by extending the cache forever. Your mileage may vary on cache age because issuer rotation cadence and token lifetime differ; record the assumption in configuration and revisit it with the identity team.
Trade-offs that belong in the design review
| Choice | Benefit | Cost or boundary |
|---|---|---|
| Cache-first verification | Low latency and no per-request dependency on the issuer | A stale set can reject newly signed tokens until refresh |
Refresh on unknown kid
|
Handles rollover quickly | Adds a network dependency on a rare path and needs coalescing |
| Short cache lifetime | Limits staleness | More issuer traffic and more chances for transient fetch failure |
| Keep old keys during overlap | Lets in-flight tokens finish | Extends the period in which a retired key remains accepted |
| Fail closed after maximum age | Preserves signature guarantees | Can interrupt login during an issuer outage |
The catch is that no policy satisfies both zero issuer dependency and instant revocation. Choose the maximum stale window with the abuse team, based on token lifetime, replay risk, and the cost of blocking a legitimate login. A gateway serving a high-value competitive game may choose a shorter window than an internal admin console.
This design is not suitable when the issuer cannot publish stable key metadata or when tokens are opaque and require introspection on every request; use the issuer's supported introspection contract in that case. Stick with a cache-and-verify model when the token is a signed JWT and the issuer documents JWKS rotation semantics.
Top comments (0)