Many apps need a gate: a point where something gets approved, held for review, or rejected. User posts, uploaded profiles, refund requests, signups that look suspicious. Doing this by hand doesn't scale; doing it with a pile of keyword rules misses context. This post walks through a gating pattern built on typed AI decisions, where the model returns a probability instead of an essay.
The three-lane gate
The core idea is simple. Instead of a binary allow/block, use three lanes:
| Probability that the item violates policy | Action |
|---|---|
| Low (e.g., < 0.2) | Auto-approve |
| Middle (0.2 – 0.8) | Human review queue |
| High (> 0.8) | Auto-hold or reject |
The thresholds are yours to set. The important part is that the middle band exists, so the model only acts alone when it's confident.
Step 1: Write questions as narrow yes/no checks
One broad question ("Is this bad?") is hard to tune. Several narrow ones are easier:
- "Does this post contain a link to an external shop?" → yes/no
- "Is the text harassing a specific person?" → yes/no
- "Is this request asking for a refund outside the stated window?" → yes/no
Narrow questions give you clearer probabilities, and you can set different thresholds for each based on how costly a mistake is.
Step 2: Send state, not just text
Context changes the answer. Include what the decision depends on: the content, the user's account age, the category it was posted in, your policy summary. A typed decision model like Jev — TypeSafe AI's decision model, documented on jevmodel.org — takes app state plus typed questions together and returns a probability per yes/no question.
Step 3: Map probabilities to lanes in code
Pseudo-code (check the docs for the actual request and response format):
p = result["contains_harassment"].probability
if p > 0.8:
hold(item, reason="harassment")
elif p > 0.2:
queue_for_review(item, score=p)
else:
approve(item)
Store the probability with the item. Reviewers can sort the queue by it and handle the most likely problems first.
Step 4: Calibrate with real data
Thresholds pulled from thin air will be wrong. Calibrate:
- Gather a sample of past items with known outcomes (approved/removed).
- Run them through the model.
- For each candidate threshold, count false approvals and false holds.
- Choose thresholds that match your tolerance — a kids' platform and a developer forum will choose very differently.
Re-run this when your policy or content mix changes.
Step 5: Keep humans in the loop
Automated gates fail in ways that matter to real people. Good practice:
- Give users a way to appeal a hold.
- Log the decision, probability and question so you can explain outcomes.
- Sample auto-approved items regularly to catch drift.
- Never let a model be the only check for high-stakes decisions (legal, safety, financial) without review.
Step 6: Add a score for prioritization
Besides yes/no, a score question can rank items: "How likely is this to need urgent attention, 1–5?" Combine it with your gate so the review queue isn't first-in-first-out but most-important-first.
Why not just use a chat model?
You can, but you'll spend effort forcing it into a fixed format, parsing its replies, and inferring confidence from wording. A model that returns typed outputs gives you a number to threshold directly. If you need to send a written explanation to the user, generate it afterward with a general LLM, once the decision is made.
Practical notes and costs
Jev is closed and hosted, so plan for network latency and service dependency in your design, and decide a fallback lane (usually "send to review") if the call fails.
On pricing, jevmodel.org is usage-based: $0.042 per 1M input tokens, output free. The Playground includes 10,000 free input tokens (100,000 after signing in), and a $9.90 one-time starter pack gives 15M input tokens that never expire. There's no unlimited free tier, so estimate your daily volume before you switch a gate on.
Summary
A good content gate has three lanes, narrow questions, calibrated thresholds and a human for the uncertain middle. Working with probabilities instead of free-text answers makes each of those pieces explicit and easier to tune.
Disclosure: jevmodel.org is an independent guide and access site for the Jev model. It is not affiliated with or endorsed by TypeSafe AI.
Top comments (0)