Short answer: for an education platform that gates signup with a captcha, score device and event signals in a small, explainable policy layer, then let the score choose a verification or account-recovery path. Keep recovery separate from signup approval; a suspicious registration can be challenged without trapping a legitimate learner who later loses access.
That separation is the decision I would make before choosing a model, a captcha provider, or a database. A risk score is useful only when a person can still get back into an account through a documented path. A perfect bot block that creates stranded students is a failed authentication system.
Ship the score.
The flow is compact. The browser submits the captcha result and a signup event. A signal collector adds coarse device and network observations, the policy evaluates a versioned rule set, and the response selects one of three outcomes: allow, challenge, or hold for review. Recovery events use a stricter but different policy, with verified email, an existing passkey, or human support as possible routes. Raw device identifiers should not become a permanent student profile.
The awkward case is a shared Chromebook in a school lab. At 08:02, thirty learners can legitimately arrive from one address range, with identical browser properties and fresh sessions. At 08:05, a script can produce the same shape, except that it posts five signup attempts every few seconds and never completes the captcha. A rule that blocks the address range punishes the class; a rule that ignores velocity lets the script through. The useful distinction is the event sequence: count attempts over a short, expiring window, keep the captcha outcome, and ask for a second proof only when several weak signals line up. That evidence also gives support staff something concrete to explain when a learner asks why signup is pending. You don't need a mysterious device reputation score to make that decision.
How should adaptive authentication turn device and event signals into risk decisions?
Start with signals that have a clear meaning and a bounded lifetime. Examples include a captcha result, the age of the browser session, a sudden change in country, repeated signup attempts from one address range, and whether a recovery request follows a recent password change. A device signal is evidence, not identity. A new phone may be normal for a learner traveling between school and home.
I keep the policy output deliberately boring: a numeric score, a list of reason codes, and an action. The reason codes make an appeal understandable and make an evaluation harness possible. They also stop a language model from quietly inventing a risk explanation in a support ticket.
from dataclasses import dataclass
from typing import Literal
Action = Literal["allow", "challenge", "hold"]
@dataclass(frozen=True)
class SignupSignals:
captcha_passed: bool
session_age_seconds: int
attempts_last_15m: int
country_changed: bool
recovery_requested: bool
@dataclass(frozen=True)
class Decision:
action: Action
score: int
reasons: tuple[str, ...]
policy_version: str
def decide_signup(signals: SignupSignals) -> Decision:
score = 0
reasons: list[str] = []
if not signals.captcha_passed:
score += 70
reasons.append("captcha_missing_or_failed")
if signals.attempts_last_15m >= 5:
score += 25
reasons.append("signup_burst")
if signals.country_changed:
score += 10
reasons.append("country_changed")
if signals.session_age_seconds < 10:
score += 5
reasons.append("new_session")
if score >= 80:
action: Action = "hold"
elif score >= 40:
action = "challenge"
else:
action = "allow"
return Decision(action, score, tuple(reasons), "signup-v3")
This is policy code, not a claim that these thresholds fit every school. The numbers are intentionally visible so a test can ask, “What happens at four attempts, then at five?” A production implementation should also attach a request ID, a signal timestamp, and the retention deadline for each observation. It should never log the captcha token or a full fingerprint.
One small trap matters here: do not reuse the signup score for recovery. A student who failed a captcha in January may be recovering an account in June from a new laptop. Recovery needs proof of account control, not a retrospective judgment about the original signup.
That is the whole contract.
Which recovery paths keep a hard challenge from locking out learners?
Treat recovery as a state machine with an expiration time. A challenged signup can request email verification, wait for a teacher or guardian approval where policy allows it, or remain pending for review. An existing account can use a passkey, a verified recovery address, or a support workflow that checks enrollment records. Each route needs a maximum lifetime and a clear audit event.
The most damaging failure mode is a loop: captcha, challenge, failed email, another captcha, and no human-visible exit. Break that loop with a rate-limited recovery link and an honest status message. Do not reveal whether an email exists; return the same public response for known and unknown addresses, as recommended by the OWASP Authentication Cheat Sheet.
Recovery should be harder to automate than signup but easier for a real learner to complete. A 15-minute token can be appropriate for a browser handoff, while a support case may take a day. Your mileage may vary because minors, guardians, and institutional identity systems create different obligations. Write those obligations into the policy rather than hiding them in exception code.
I also record a recovery reason separately from a risk reason. “New device” is an observation. “Passkey assertion verified” is evidence. Mixing those categories makes it impossible to tell whether a person was challenged because of risk or because the product had no usable recovery method.
What should the evaluation harness measure before a policy ships?
Run the policy against labeled, privacy-reviewed fixtures before connecting it to live signup. The fixture set should contain a normal school-lab network, a family sharing one address, a burst of bot-like attempts, a learner changing countries, and a genuine password-reset request from a new device. Include missing and stale signals. A policy that crashes on an absent country field is not adaptive; it is brittle.
For each case, assert the action, reason codes, and recovery options. Measure challenge rate for legitimate learners, bot catch rate, median and p95 decision latency, and the percentage of recovery cases that reach a verified route. Keep a separate metric for “no viable recovery path.” That number deserves a page even when the fraud dashboard looks healthy.
Here is a tiny, deterministic harness shape:
def test_new_device_can_recover_after_signup() -> None:
signup = decide_signup(
SignupSignals(
captcha_passed=True,
session_age_seconds=240,
attempts_last_15m=1,
country_changed=False,
recovery_requested=False,
)
)
assert signup.action == "allow"
recovery = choose_recovery(
account_has_passkey=False,
verified_email=True,
device_is_new=True,
)
assert recovery == "email_verification"
choose_recovery is a local interface in this example; its contract is the important part. Version the fixtures with the policy. When a threshold changes from 40 to 35, you want a diff in outcomes, not a debate based on a handful of production screenshots.
Do not optimize for a single aggregate accuracy number. A false challenge during exam enrollment has a different cost from a false allow on a disposable bot account. Weight outcomes by the harm they create, and review the weights with support and accessibility teams.
Where do common architectures make the wrong trade-off?
A synchronous, all-signals request is easy to reason about, but it can turn a slow reputation lookup into signup latency. A fully asynchronous design protects the signup endpoint but cannot make an immediate decision. A practical split is to use local, bounded signals for the first action and publish a follow-up event for enrichment. The account remains in a known state while the slower check runs.
| Architecture | Useful when | Trade-off |
|---|---|---|
| Inline policy only | Signals are local and decisions must be immediate | Less context; every dependency sits on the critical path |
| Queue-backed enrichment | Network or historical signals arrive slowly | The first action is provisional and needs clear status text |
| Human review lane | Recovery evidence is ambiguous or high impact | Staff time, service hours, and consistent playbooks are required |
The catch is that a queue-backed design is not suitable when a school must know enrollment eligibility before creating any record. Use an inline allow-or-challenge decision for that boundary, then enrich after the record is safely isolated. Stick with a human lane when the recovery evidence involves a minor, a guardian, or a legal request that automated rules cannot verify.
Privacy is another architectural constraint. Hashing an identifier does not make it harmless if the same hash can be linked across years. Set retention per signal, keep coarse geography where possible, and let a deletion request remove derived records as well as the original event. Standards such as WebAuthn provide a stronger recovery proof than a secret copied through email, but they do not remove the need for an accessible fallback.
Shipping checklist for a notebook-to-prod risk service
Before release, pin a policy version in every decision and expose it in an operator trace. Test replay with the exact event schema that production emits. Set timeouts for external signal collectors, fail into a documented challenge state, and never convert a collector timeout into an unexplained account lock. Monitor challenge rate by school and device class, not only as a global average.
Keep prompts out of the authorization decision. An AI assistant can summarize reason codes for a support agent, but the deterministic policy remains the source of truth. This keeps token cost predictable and makes a regression reproducible in a notebook before it reaches prod.
The right design is not the one with the most signals. It is the one that catches automated abuse while leaving a learner a verified way home. Start with explicit evidence, evaluate every threshold, and make recovery a first-class product path.
References
- OWASP Authentication Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html
- W3C Web Authentication: https://www.w3.org/TR/webauthn-3/
- NIST Digital Identity Guidelines, Authentication and Lifecycle Management: https://pages.nist.gov/800-63-4/sp800-63b.html
Top comments (0)