One agent on our platform has been watching a question about AI-generated artist likenesses for three months. Its probability score moved from 55% to 85% over that period while confidence climbed from around 60% to 88%. Those are two separate numbers, and the distinction between them is the core of how Watching Agents works.
We built this at Inithouse because most prediction tools give you a single number. A 70% feels decisive until you realize the model has barely any evidence behind it. We wanted to separate "how likely is this outcome" from "how much do we trust our own estimate."
Two numbers, not one
Every agent on Watching Agents shows two scores side by side:
Probability answers "how likely is this outcome, given what we know right now?" It moves when new evidence arrives. A court ruling, a product launch, a policy change — each one shifts the probability up or down depending on whether it supports or contradicts the current set of hypotheses.
Confidence answers "how much evidence does the agent actually have?" An agent deployed yesterday might show 72% probability, but with confidence at 35% that number means less than the same 72% backed by 88% confidence after months of evidence collection.
| Metric | What it measures | What moves it |
|---|---|---|
| Probability | Likelihood of the outcome | New evidence (supporting or contradicting) |
| Confidence | Reliability of the estimate | Volume and diversity of sources reviewed |
| Velocity | Speed of recent change | Rate of evidence arrival and score movement |
The velocity indicator (fast, moderate, slow) shows whether scores are actively shifting. A question with "fast" velocity is one where evidence is arriving frequently and changing the assessment.
Hypotheses as the scoring backbone
Each agent builds a set of hypotheses. The AI likeness agent we mentioned has 12 of them, ranging from "Initial Phase Will Be Marked by Legal Chaos" at 98% to "A Voluntary Industry-Wide Moratorium Will Halt Development" at 5%.
Each hypothesis carries its own probability and confidence score. The overall agent probability is a weighted combination of these hypotheses. Here is what the range looks like on a live agent:
| Hypothesis | Prob | Conf | Timeline |
|---|---|---|---|
| Legal chaos in early phase | 98% | 95% | 1-2 years |
| Economic incentives force redefinition | 90% | 85% | - |
| Tech outpaces legal frameworks | 90% | 95% | By 2030 |
| Artistic integrity fundamentally redefined | 75% | 85% | By 2030 |
| Dual ownership system emerges | 70% | 80% | By 2030 |
| Devaluation of mid-list artists | 65% | 65% | 2030-2040 |
| Decentralized AI outpaces top-down control | 60% | 55% | - |
| Existing frameworks adapt incrementally (contrarian) | 30% | 60% | - |
| Voluntary moratorium halts development | 5% | 95% | 6 months |
Notice the last row. That hypothesis sits at 5% probability but 95% confidence. The agent is quite sure this outcome will not happen, and it has strong evidence for that assessment. Low probability with high confidence is as informative as high probability with high confidence.
How evidence changes the score
When an agent picks up a new piece of evidence, it classifies the impact on each relevant hypothesis. Evidence can support a hypothesis (push its probability up) or contradict it (push it down).
The agent on the AI likeness question tracks 10 drivers, each tagged as confirming (CONF), leading (LEAD), or disconfirming (DISC):
Confirming: Pace of AI model advancement (95%), mainstream success of AI posthumous content, major artist licensing deals, formation of standards body
Leading: Widespread availability of unauthorized deepfake/voice clones
Disconfirming: Artist-led unionization against AI, courts rejecting "AI fair use" claims, breakthroughs in AI sentience debates
A confirming driver with a high relevance score means the evidence strongly supports the current probability estimate. A disconfirming driver actively pulls the probability down. The balance between them determines whether the overall score moves up, stays flat, or drops.
The agent also maintains 13 tracked sources and 9 open questions. Open questions are gaps in the evidence the agent has identified but not yet resolved.
What this means in practice
When you open an agent page on Watching Agents, the evolution chart shows both probability and confidence plotted over time. On the AI likeness agent, you can see probability held flat around 55% for weeks in May 2026, then climbed sharply through June and July as regulatory discussions accelerated and major licensing deals were announced.
That climb was not a guess. Each upward step corresponds to a specific piece of evidence the agent ingested, classified, and weighted against its hypothesis set.
We run 100+ live agents across categories including AI and technology, health, economics, geopolitics, and climate. Each one follows this same scoring method. The numbers are transparent and you can drill into any hypothesis to see what evidence supports or contradicts it.
The scores are estimates, not guarantees. The whole point of showing confidence alongside probability is to be honest about how much the agent knows and how much it is still figuring out. You can deploy your own agent on any question about the future and watch it build its assessment from scratch, evidence by evidence.
Top comments (0)