At Inithouse, a studio running parallel product experiments, we built Watching Agents to answer a question we kept asking ourselves: what happens to a fast-moving topic while nobody is watching? Across over a hundred deployed agents, we observed that most questions worth tracking shift meaningfully within the first two weeks. Here is how one agent tracks an AI-policy question from start to score change.
The setup: one question, five minutes
You type a question about the future. Not a search query, but a genuine open question. Something like "Will the EU AI Act enforcement lead to measurable product withdrawals from EU markets by Q1 2027?"
The agent reads the question and generates a set of competing hypotheses. It does not predict. It maps the space of plausible outcomes and assigns each an initial probability score (Prob) and a confidence score (Conf). Both start as rough priors. The agent knows it is guessing.
Here is what that looks like for the EU AI Act question after the first pass:
| Hypothesis | Prob | Conf |
|---|---|---|
| Major vendors pull specific product lines from EU | 0.35 | 0.20 |
| Vendors comply via feature-gating, no full withdrawal | 0.45 | 0.25 |
| Enforcement delayed or softened, status quo holds | 0.15 | 0.15 |
| Smaller SaaS vendors exit EU quietly, big players stay | 0.30 | 0.18 |
Prob reflects how likely the agent considers each outcome. Conf reflects how much evidence backs that estimate. Both move independently as new data arrives.
The evidence loop
Once hypotheses exist, the agent starts scanning. It pulls from news, regulatory filings, industry commentary, and public statements, anything relevant to the question. Each piece of evidence is scored against every hypothesis: does it support, contradict, or say nothing?
This runs continuously. The agent does not wait for you to check in.
The process works in three stages:
- Scan: the agent searches for new information relevant to the question, filtering noise from signal
- Score: each piece of evidence is mapped to the hypotheses it affects, adjusting Prob and Conf accordingly
- Alert: when a score crosses a threshold or shifts significantly since the last check, the agent flags it
A week into the EU AI Act question, the table looked different:
| Hypothesis | Prob | Conf | Direction |
|---|---|---|---|
| Major vendors pull specific product lines from EU | 0.20 | 0.45 | ↓ Prob, ↑ Conf |
| Vendors comply via feature-gating, no full withdrawal | 0.55 | 0.50 | ↑ Both |
| Enforcement delayed or softened, status quo holds | 0.10 | 0.40 | ↓ Prob, ↑ Conf |
| Smaller SaaS vendors exit EU quietly, big players stay | 0.40 | 0.42 | ↑ Both |
The feature-gating hypothesis gained ground as two major cloud providers published compliance roadmaps instead of exit plans. The "full withdrawal" hypothesis dropped after no vendor announced exits. Confidence rose across the board. More evidence means less guessing.
What the scores actually tell you
Prob and Conf together give you two dimensions that a single prediction cannot.
High Prob, low Conf means the agent leans toward an outcome but the evidence is thin. You are working with a best guess. High Prob, high Conf means the evidence is stacking up in one direction. Not certainty, but a grounded picture.
The combination matters because a fast-moving policy topic generates plenty of noise. A score that jumps on a single headline and drops the next day teaches you nothing. When Conf climbs steadily alongside Prob, the signal is real.
Where this fits alongside other approaches
Prediction markets like Metaculus or Polymarket aggregate crowd bets. They are useful when the crowd is large and informed. An agent works differently. It watches a question you care about personally, builds hypotheses specific to your framing, and surfaces evidence you might have missed. It does not require a liquid market or a crowd. It runs on your question, for you.
We noticed a parallel with another product in the Inithouse portfolio: Be Recommended, an AI visibility monitoring tool. Both products track something changing over time and surface the shift. The difference is the domain. Be Recommended watches how AI models perceive a brand, Watching Agents watches how a real-world question evolves. The pattern of "continuous monitoring + structured scoring + alert on change" turned out to be reusable across very different use cases.
The public agent layer
Every agent on Watching Agents can be set to public. Public agents serve as permanent, crawlable pages, each one a living document on a specific question. We measured over a hundred public agent pages indexing within the first month of launch, which validated the thesis that structured, question-focused content earns organic visibility faster than generic articles on the same topics.
This connects to something we measured across the broader Inithouse portfolio: products that generate their own structured content (like Verdict Buddy, our AI conflict mediator built on Gottman, NVC, and CBT frameworks) tend to build organic reach faster than products relying on external content marketing alone.
Try it
Pick a question you keep checking the news for. Deploy an agent at watchingagents.com and let it build hypotheses. Check back in a week and see how the scores moved. The gap between what you expected and what the evidence shows is usually where the interesting insight lives.
Inithouse is a studio shipping a growing portfolio of products in parallel. Watching Agents is one bet in that portfolio, built on the idea that structured, continuous monitoring beats periodic manual checking for questions that matter.
Top comments (0)