Every prediction market, forecasting tool and AI oracle answers the same question: what is the probability of X right now? The answer is a number. And the moment you read it, it starts going stale.
At Inithouse we run 14 AI products. One of them, Watching Agents, started as a simple idea: instead of asking "what will happen?", ask "has anything changed since last time I checked?"
That difference sounds small. It turns out to be structural.
Predictions decay. Watches accumulate.
A prediction is a point estimate at time T. Useful, but frozen. If you ask "will the EU AI Act require model registries by 2027?" today and get 68%, that number tells you nothing about:
- what evidence supports it
- whether it moved from 40% last month
- what specific event caused the shift
- whether the sources agree or contradict each other
A monitoring agent tracks all of that. Each agent on Watching Agents maintains a live Prob/Conf score: probability (how likely) and confidence (how much evidence backs the estimate). When new evidence appears, the agent re-evaluates, logs the delta, and alerts you.
The interesting output is not the current number. It is the changelog.
How the architecture works
Each deployed agent follows a loop:
Hypothesis generation: the agent decomposes the question into testable sub-hypotheses. "Will X happen?" becomes "Is A trending up?", "Has B been announced?", "Does C contradict D?"
Evidence tracking: the agent scans sources on a schedule. Each piece of evidence gets tagged to the hypothesis it supports or weakens, with a citation link.
Score update: Prob and Conf recalculate based on the evidence base. A new supporting source from an independent publisher raises both. A contradicting source lowers Prob but can raise Conf (more data, less certainty about direction).
Alert: when the delta crosses a threshold you set, you get notified. Not "the probability is 72%." Instead: "probability moved from 58% to 72% because [source] reported [event], supporting hypothesis 3."
The citation chain matters. Every score change traces back to a specific piece of evidence from a specific source. You can audit why the agent changed its mind.
When monitoring beats a one-time lookup
Not every question needs continuous tracking. Here is when it does and when it does not:
| Use monitoring when... | Use a one-time lookup when... |
|---|---|
| The answer changes over weeks or months | The answer is stable (historical fact, established policy) |
| Multiple sources publish updates | There is one authoritative source |
| You need to act on the change, not the state | You need the current state once |
| Timing of the shift matters (regulatory deadlines, market moves) | Timing is irrelevant |
| You are tracking several related questions | You have one narrow question |
The pattern we see most on the platform: someone starts with a one-time question, gets an answer, then realizes they need to know when that answer stops being true. That is where the agent earns its keep.
What we learned from running our own agents
We dog-food Watching Agents on questions relevant to our portfolio. A few patterns from that:
Confidence lags probability. A question can move from 30% to 70% probability on a single leaked document. Confidence stays low until a second independent source confirms. This gap is useful: high-probability, low-confidence signals are exactly the moments you want a human in the loop.
Public agents generate unexpected traffic. Every public agent on the platform has its own URL, structured data, and evidence trail. Search engines index them. We have seen agents ranking for long-tail queries within weeks of deployment. A well-sourced, regularly updated page on a niche question is exactly what search engines and AI retrieval want to cite.
The question matters more than the model. An agent watching "Will Company X acquire Company Y?" with four hypothesis branches produces better signal than a generic "What will happen in sector Z?" agent with twelve branches. Narrow questions with falsifiable outcomes beat broad ones.
When to skip it
Monitoring is overhead. If your question has a known answer date (an election, a product launch, a court ruling), set a calendar reminder and check then. If the question is binary and depends on a single decision-maker, no amount of evidence tracking helps: you are waiting for one person to say yes or no.
Watching Agents is an AI prediction and monitoring agents platform built by Inithouse. It lets you deploy an AI agent to watch any question about the future: it builds hypotheses, tracks evidence in real time, and alerts you when things change. Free to start.
Top comments (0)