I run an automated crypto trading system with a two-brain architecture: a scanning engine that finds setups, and an AI advisor that re-litigates every setup before real money touches it. I've written about the advisor killing the same trade six times in an hour, and about it refusing to short a liquidation cascade. Both stories are about the word "no."
This one is about a word that's harder to handle: "yes" — specifically, the kind of yes that arrives with its fingers crossed.
In one 24-hour window last week, the advisor issued 198 final rulings on the experimental sub-account. Roughly half were vetoes. Forty-eight were "switch to a better ticker." And 37 were approvals — of which 27 tripped an internal tripwire that automatically tightened the stop-loss and cut the position size before the order was placed. Not because the advisor said no. Because of how it said yes.
The flinch, formalized
The advisor's output is structured: a ruling, a score delta, a self-reported confidence, and a free-text reason. The reason field is where the truth leaks out. A clean approval reads like "structure confirmed, direction aligned, RR acceptable." But a large share of approvals read more like this one, from a real ruling on a PUMP long during a counter-signal experiment:
PROCEED — "counter-trend risk is high, but stablecoin expansion and smart-money inflow offer an oversold-bounce window; suggest shrinking size to 0.6x and tightening the stop by 20%."
That is not conviction. That is someone saying "fine, do it" while backing toward the door. The advisor approved the trade and hedged its own approval in the same sentence.
Early on, I trusted the self-reported confidence field to capture this. It doesn't. The model will return conf=0.55 on one trade and conf=0.72 on another while expressing identical doubt in the prose — and occasionally high confidence alongside deeply worried text. The number is a posture. The vocabulary is the tell.
So I built a mechanism around the tell. Internally it's rule F-072, and the log line is brutally literal:
F-072: PROCEED with risk words: ['risk', 'overbought'] -> auto-tightening
If the advisor rules PROCEED but its reason contains any word from a risk lexicon — risk, questionable, overbought, overheated, cautious (the advisor thinks in two languages, so the list is bilingual; it once tripped on the English word "caution" alone) — a deterministic wrapper overrides the order parameters. No re-prompt, no second opinion, no appeal. The sentence itself is the evidence.
What the machine does with a flinch
The wrapper acts on two knobs, both visible in the decision records:
-
Stop tightening (
stop_tighten_pct): pulling invalidation closer to entry — applied values ranged from 5% to 20% across that window. A hesitant yes doesn't get the stop it asked for; it gets the stop its tone earned. -
Position scaling (
position_scale): multipliers of 0.5x to 0.9x instead of full allocation. The most nervous approvals — multiple risk words in one sentence, or a flinch stacked on a sub-account losing streak — traded at roughly half size.
The philosophy: a yes with a tremor is not a yes. It's a small yes. You don't ignore it — the setup still passed the engine's scoring and the advisor's review — but you refuse to fund it at the level a clean approval would get. Doubt is priced in mechanically, at the moment it's expressed, not remembered later by a human who might talk themselves out of it.
There's an asymmetry I consider a feature. F-072 only ever makes a trade more conservative. It can never widen a stop or add size. The word list can't be gamed into aggression, because there is no "confidence vocabulary" path that loosens anything.
The counterweight: the system also distrusts its own caution
Here's the tension that makes this interesting rather than just paranoid. While F-072 discounted hesitant approvals, a second mechanism ran in the opposite direction.
The advisor had been on a veto streak — dozens of consecutive rejections, mostly against the sub-account's counter-signal experiments. A hard approval threshold under a long veto streak is a death spiral: nothing is ever good enough, the system stops trading, and "safety" quietly becomes "never participates." So rules F-229/F-230 make the score threshold elastic. The log shows it breathing:
F-229/F-230: Elastic threshold: 80 → 75 (consecutive_veto=5, original=80, floor=60)
F-229/F-230: Elastic threshold: 80 → 70 (consecutive_veto=10, original=80, floor=60)
After five consecutive vetoes the bar relaxes from 80 to 75; after ten, to 70 — never below a hard floor of 60. The system treats its own refusal streak as a state to manage, not a virtue to maximize.
Put the two mechanisms side by side and you get the actual design principle: distrust streaks in both directions. A streak of enthusiastic setups is how you get run over (that's what the vetoes are for). A streak of reflexive rejections is how a trading system becomes an expensive dashboard. And a hesitant yes — neither streak nor conviction — gets executed, but at a size that respects the tremor. Every state has a mechanical response, and none of them depend on anyone's mood at 3 a.m.
Why not just fix the confidence score?
The obvious question: instead of parsing vocabulary, why not just calibrate the advisor's conf field better — better prompts, rubrics, logprobs?
We tried. Three problems. First, LLM self-reported confidence is notoriously decorrelated from accuracy, and prompt-engineering it just moves the decoration. Second, one scalar compresses "direction is right but timing is dangerous" and "direction is a coin flip" into the same number. Third — the subtle one — asking for calibrated confidence changes the prose: the model starts writing to justify its number. The unguarded sentence, produced as a side effect of explaining the ruling, is more honest than the field produced on request.
So the architecture uses both, asymmetrically: the number can veto a trade, but only the vocabulary can shrink one. The number is a gate; the prose is a dial.
What I'd tell anyone wiring an LLM into an execution path
- Don't trust declared confidence; instrument behavior. Word choice, hedging, self-contradiction — these leak what the schema can't hold.
- Make doubt expensive, automatically. If a human must notice the hesitation and manually cut size, it happens twice and then never again.
- One-way ratchets only. Hesitation may tighten risk; enthusiasm must never loosen it.
- Guard the guard. Whatever mechanism says "no" needs its own governor (our elastic threshold), or caution compounds into paralysis.
- Log the flinch. The F-072 lines are now among the most valuable entries in our post-mortems — they mark exactly which trades the system never believed in.
Build in public
I document the system's decision stream — vetoes, switches, hesitant approvals, elastic thresholds — as a public log. No PnL screenshots, no return figures, no promises. Just the mechanical reasoning of a system built to survive the market's enthusiasm and its own.
Full decision log and live system notes → https://kestrelquant.com
⚠️ Crypto derivatives are a high-risk market; leveraged positions can be liquidated quickly, and no rule-based safeguard eliminates that risk. Kestrel is a decision-support tool, not a signal service, copy-trading product, or asset manager. Nothing here is financial advice. The mechanisms described are internal engineering choices, shared for educational purposes.
Top comments (0)