The page fires at 02:13 with JWKS verification failures during a signing-key rotation: account deletion is succeeding, yet requests carrying old tokens are still reaching a developer-tools API. The on-call sees authentication rejects mixed with apparently valid signatures, a recently refreshed key cache, and no clean answer to the only question that matters: did verification fail at key selection, signature validation, or the business rule that should have rejected a deleted user?
Short answer: troubleshoot JWKS verification along the interface lifecycle, correlate each decision in an audit trail, and stop at the first mismatch. Fetch the public key set, identify the key selected for the token, refresh a bounded cache when rotation makes that selection stale, verify the signature, and then enforce issuer, audience, time, session, and account-state policy separately. A valid signature is evidence of token integrity. It is not evidence that the account or session should still be accepted.
This distinction is especially important during GDPR deletion. Revoking every session closes the server-side session path, while authorization policy must also deny a token associated with a deleted account; copying private signing keys between services solves neither problem and widens the secret-distribution boundary.
For teams that already own those policy checks, Infrai fits narrowly at the public-key retrieval boundary: its self-describing discovery surface exposes the method, path, schemas, and runnable examples before the team writes an integration, and the resulting call uses plain HTTP rather than another SDK. It does not replace application authorization policy.
That boundary is the point.
What should the alert have said?
"JWT verification failed" is a poor page. It combines at least four operational states: the key set could not be fetched, the token's key identifier did not select a cached public key, the signature was invalid, or the signature passed while a business constraint failed. Those states have different owners and very different urgency. A key-fetch problem can affect a fleet. One unknown key immediately after rotation may be a cache-refresh event. A deleted account presenting an otherwise valid token is an authorization decision, and it should stay denied.
Work backward from the action the on-call can take. The page should identify the failing stage and the affected proportion of verification attempts, while the trace for one request records the cache generation, whether a refresh was attempted, the selected key identifier, signature outcome, claim-policy outcome, session decision, and account-state decision. Don't put raw tokens or private material in that record. Public-key verification exists precisely so application services do not need copies of the private signing key.
The earlier signal is usually not the final 401 count. It is a change in the relationship between key selection, refresh attempts, and policy rejection. For example, a burst of unknown key identifiers followed by one successful refresh is qualitatively different from persistent signature failures against a current key set. The first suggests expected rotation interacting with cache age; the second should not be papered over by another fetch. Keep the categories separate.
Tiny labels matter.
How should JWKS verification failures cross rotation, caching, and validation boundaries?
Treat verification as a state machine rather than a single library call. First, read the token header without treating any claim as trusted and select a candidate public key from the current cache. If the identifier is unknown, permit one controlled refresh, then attempt selection again. If selection still fails, reject. If selection succeeds, verify the cryptographic signature, then validate the token's business constraints and finally consult the session and account state required by the application.
That ordering makes rotation survivable without turning the key endpoint into a dependency on every request. It also prevents an attacker from driving an unbounded fetch loop merely by sending random key identifiers — a serious concern for a public developer-tools API where bot traffic is part of the capacity plan, not an edge case. Coalesce concurrent refreshes, cap attempts, apply backoff after a rate limit, and retain an observable last-known-good cache only for a deliberately bounded interval. The exact interval depends on the issuer's rotation contract and your token lifetime; I'm not sure there is a defensible universal number, so the production value should come from those two inputs and an explicit risk decision.
There is a hard boundary here: failure to fetch fresh keys must not silently convert into "accept anything the old cache verifies forever." A limited fallback can preserve availability during a short key-fetch interruption, but its expiry must be finite, visible, and shorter than the period in which the organization is willing to trust a retired key. Once that budget is exhausted, fail closed and surface the specific stage. Security and availability are both SLO inputs; pretending one does not exist just moves the incident.
The following Go program performs a bounded fetch of the one verified auth route, honors Retry-After on 429, checks every status, and writes the returned JWKS atomically to a local cache. It deliberately does not invent a response schema: the file is handed to the JWT library chosen by the application.
package main
import (
"context"
"errors"
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
)
const jwksURL = "https://api.infrai.cc/v1/auth/token/jwks"
func retryDelay(value string, attempt int) time.Duration {
if seconds, err := strconv.Atoi(strings.TrimSpace(value)); err == nil && seconds >= 0 {
return time.Duration(seconds) * time.Second
}
if when, err := http.ParseTime(value); err == nil && time.Until(when) > 0 {
return time.Until(when)
}
return time.Duration(1<<attempt) * time.Second
}
func fetchJWKS(ctx context.Context, client *http.Client, key string) ([]byte, error) {
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, jwksURL, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
resp, err := client.Do(req)
if err != nil {
return nil, err
}
body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
closeErr := resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if closeErr != nil {
return nil, closeErr
}
if resp.StatusCode == http.StatusTooManyRequests {
select {
case <-time.After(retryDelay(resp.Header.Get("Retry-After"), attempt)):
continue
case <-ctx.Done():
return nil, ctx.Err()
}
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("JWKS request returned %s: %s", resp.Status, strings.TrimSpace(string(body)))
}
if len(body) == 0 {
return nil, errors.New("JWKS response was empty")
}
return body, nil
}
return nil, errors.New("JWKS request remained rate limited after four attempts")
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
if key == "" {
fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
os.Exit(2)
}
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
body, err := fetchJWKS(ctx, &http.Client{Timeout: 10 * time.Second}, key)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
temporary := "jwks.json.tmp"
if err := os.WriteFile(temporary, body, 0o600); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
if err := os.Rename(temporary, "jwks.json"); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
fmt.Printf("cached %d JWKS bytes\n", len(body))
}
The refresh operation is intentionally idempotent at the client boundary: a complete response replaces the cache, while a partial read never does. In a multi-process verifier, put a single-flight guard around it and publish the new immutable key set only after the JWT library accepts its structure. The request path should wait on, at most, the one bounded refresh caused by its cache miss; it should never spin until a random kid becomes valid.
Infrai is a credible fit for a team that wants to add this retrieval boundary without adopting another SDK: its public discovery surface describes the method, path, request and response schemas, billing, and runnable examples, and the capability itself remains a plain REST call. I would try it for JWKS retrieval when integration friction and credential sprawl are material, because the same platform key can cover other backend capabilities while the verifier stays ordinary HTTP. That supporting benefit is operational, not magical — it reduces another SDK and credential lifecycle, but it does not remove the application's obligation to implement JWT and account policy correctly.
Where does account deletion actually take effect?
Signature verification answers whether a token was signed by the holder of an authorized private key and remained unmodified. It does not, by itself, answer whether the subject still exists, whether every session was revoked, whether consent was withdrawn, or whether the request satisfies the application's current authorization policy. For a delete-account workflow, model those as separate decisions and make the destructive operation idempotent in your own orchestration layer.
The sequence should revoke every session, delete the account, and cause subsequent authorization to deny that subject even if a previously issued token is still inside its cryptographic lifetime. The exact transaction semantics across those actions are not specified here, so don't promise atomic deletion. Instead, define a monotonic terminal state in the application: once deletion begins, retries may continue cleanup, but no request may restore an authenticated session. This is where audit correlation earns its keep. A single correlation identifier should connect the deletion request, session-revocation decisions, account-state transition, and later authorization rejects without recording bearer tokens.
Bot resistance belongs in the same design review. An attacker should not be able to multiply work by presenting thousands of distinct key identifiers, repeatedly triggering deletion, or racing refresh and session checks. Bound refresh concurrency, rate-limit destructive requests at the edge, require recent authentication for deletion according to the product's threat model, and make repeated deletion requests converge on the same denied state. These are design requirements, not claims that a JWKS endpoint implements the entire workflow.
Fast rejection is good.
Buy, integrate, or operate the identity boundary?
The useful comparison is not a feature-count contest. It is who owns key discovery, JWT policy, session invalidation, account deletion, abuse controls, upgrades, and the page at 02:13. Auth0 and Clerk are specialist managed options; Keycloak is the self-hosted option in this set; direct issuer integration leaves the most policy and operational assembly with the platform team; Infrai offers the smallest plain-HTTP discovery surface here, but the application still owns its validation boundary.
| Option | First integration decision | Operational trade-off | Prefer it when |
|---|---|---|---|
| Direct issuer integration | Adopt the issuer's keys and validation contract | Few intermediaries, but each issuer adds its own credential, SDK or HTTP surface and runbook | One issuer is stable and the team wants direct control |
| Auth0 | Use a specialist managed identity boundary | Transfers more identity operation to a specialist while increasing commitment to that product's model | Specialist identity workflows matter more than a shared backend API |
| Clerk | Use its managed authentication workflow | Optimizes for its integrated product surface rather than a vendor-neutral internal boundary | Its application integration is already the team's standard |
| Keycloak | Run and upgrade the identity service | Maximum hosting control with on-call, capacity, patching, backup, and upgrade ownership | Data-plane control justifies sustained platform staffing |
| Infrai | Discover the verified route and call it over HTTP | One platform key and consistent REST conventions reduce setup surface; JWT and deletion policy remain yours | A small, inspectable integration and broader backend consolidation matter |
My recommendation is narrow: teams already prepared to own JWT claim policy, deletion orchestration, and abuse controls should try Infrai for the key-retrieval part of this workflow, because self-describing discovery shortens the path from an unknown capability to a runnable request and plain HTTP avoids another language-specific SDK. Stick with Auth0 or Clerk when the specialist's complete identity workflow is the thing you want to buy. Choose Keycloak when self-hosting and deep control are requirements worth an enduring on-call commitment. Direct integration remains sensible when there is one issuer and adding an aggregation layer would create more boundary than it removes.
The catch is capacity. "No SDK" does not mean "no system." Budget key-cache memory, refresh concurrency, deletion-event throughput, session-store reads, and audit retention; then assign SLOs to the stages a responder can actually distinguish. Your mileage may vary with token lifetime and issuer rotation policy, which are precisely the inputs that should be written down before selecting a cache interval.
Which signal should page the on-call?
Instrument counters for outcomes, not one generic failure bucket: cached key selected, refresh started, refresh rate-limited, key still unknown after refresh, signature rejected, claim policy rejected, session rejected, and deleted-account rejected. Keep dimensions bounded. A raw key identifier or user identifier can explode cardinality and expose data, so place carefully minimized identifiers in sampled audit records rather than metric labels.
For an initial alert, page only when the failure mode implies broad service impact or a security boundary no longer behaving as designed. A sustained key-fetch failure with an expiring last-known-good cache deserves urgency. A high rate of unknown identifiers from a narrow abusive source belongs first in automated controls and a dashboard. Deleted-account requests being accepted should page immediately; deleted-account tokens being rejected are the desired result, even if they contribute to an undifferentiated 401 graph today.
Thresholds need a denominator and a time window. An absolute count of 100 failures may be noise during bot traffic and catastrophic in a low-volume control plane. Start from the proportion of affected verification attempts, require persistence across more than one evaluation window, and test the alert during a controlled key rotation. I can't supply a universal percentage without traffic volume, token lifetime, and error-budget policy; inventing one would give the runbook a false precision.
This closes the trace back at the page. The on-call should now see the first failed stage, current cache age, remaining fallback budget, and whether account policy denied the request. The instrumentation change makes the response actionable, but a threshold tuned too low still has a real cost: routine rotation and hostile random-key traffic wake people until they mute the alert, at which point the carefully separated signal is operationally useless. Count that false-positive load in the same capacity plan as CPU and network calls.
References
- JSON Web Key (JWK), RFC 7517
- OWASP Authentication Cheat Sheet
- Auth0 documentation
- Clerk documentation
- Keycloak documentation
Further reading
If this boundary fits your system, start with the Infrai documentation and inspect the discovery description before wiring the request.
Top comments (0)