Your Monitoring Reports Are Accurate. Are They Still Worth Sending?

Juan Camilo Auriti built a monitoring product that compared website properties across runs.

The reports were correct. 78% of them showed a delta of exactly zero.

His first response was technical: the change-detection thresholds must be too coarse. He spent about a month tuning them.

That was the wrong layer.

The product let someone audit a site, fix issues such as robots.txt or schema markup, and then watch those properties over time. Once the fixes were made, the site could remain stable in the dimensions being measured.

That is a successful first task. It is not automatically a reason to send a new report every week.

An empty recurring report can mean two different things:

1. The customer completed the job, and there was nothing new to act on.
2. The report did not carry a decision for the recipient in the first place.

Those interpretations point to different engineering and product work.

If the product promises assurance that a site remains stable, “no change” can be the result the customer wants. The question is whether the promise and cadence match that outcome.

If the product promises to tell the customer what to do next, increasing sensitivity does not solve the problem by itself. A report can detect smaller changes and still change no decision.

Juan Camilo Auriti described the turning point in his own monitoring product this way:

“The thresholds weren't the problem. The artifact was the problem.”

One commenter gave the problem a useful name: decisionless. Before sending a recurring artifact, ask: what decision does this change? Auriti agreed that this is a property you can check before sending, rather than an interpretation you make afterward.

This is why threshold work is so attractive. It produces code changes, diffs, and a number. The question “should this artifact exist at this frequency?” produces no diff and may force a change to the product promise.

For a report, alert, digest, or recurring audit, separate these questions before adding another feature:

• Did the first task succeed so completely that the original problem stopped occurring?
• Or does the recurring artifact fail to connect to a decision the recipient still has to make?

The first points to the promise and cadence. The second points to the artifact and its handoff into a decision. They are different failure layers, so they should not share the same next fix.

The figure is self-reported and the underlying query is not independently audited. This is not a rule that monitoring products should stop sending weekly reports. Stability may be the product. The point is narrower: an empty report is not enough evidence to choose between threshold work, delivery work, and a promise change.

When your recurring report comes back empty, what decision was the next issue supposed to change?