Building an LLM Reporting Layer That Never Invents Numbers

Why a solar production report generator treats grounding as a hard constraint, not a nice-to-have — and what actually enforcing that looks like in a system prompt.

The problem

Most PV monitoring dashboards are good at showing data and bad at explaining it. A chart tells you your system produced 26 kWh yesterday; it doesn't tell you whether that's normal, whether something's wrong, or whether you should care. For anyone who isn't already fluent in reading production curves, a dashboard is a wall of numbers with no verdict attached.

solar-report is a small open-source CLI that turns periodic production data — plus, optionally, logged inverter events — into a short written report: an Overview, a Trend description, a list of Observations, and, when warranted, Recommendations. The interesting part isn't the report format. It's what had to be true about the pipeline before I trusted an LLM (Claude, via the Anthropic API) to write the paragraphs.

The LLM never calculates anything

The first design decision, and the one everything else follows from: the model that writes the report never sees raw time-series data, and it never does arithmetic.

Aggregation, baseline computation, and anomaly detection all happen in plain Python before any API call is made. What reaches the model is a pre-computed summary — total kWh, a rolling baseline, a short list of flagged anomalies, an optional list of events — already reduced to the numbers that matter. The model's job is narrower than "analyze this data and report on it." Its job is "narrate these specific facts, in this structure, without adding your own."

That distinction matters because it's the difference between a tool that occasionally hallucinates a comparison and a tool that structurally can't, because the comparison was never computed by the part of the system capable of hallucinating.

A threshold that came from data, not from a guess

The anomaly detection threshold is a good example of what "grounded" costs in practice, because getting it right took a wrong first attempt.

The first version flagged any day where production fell more than 15% below a rolling 4-week baseline. A reasonable-sounding number, chosen without much evidence. Running it against a few weeks of real production data flagged ordinary weather variability as an anomaly almost every week — a slightly cloudier Tuesday would trigger the same "notable event" language as an actual fault. The report cried wolf constantly, which is worse than not flagging anomalies at all: it teaches the reader to stop reading the Observations section.

Two changes, both driven by what the data actually looked like rather than what sounded plausible: raising the threshold to 25%, and restricting it to negative deviations only. Days that overperform aren't anomalies in any sense a system owner cares about — they're just a good day. Once both changes landed, the anomaly list started matching what a person looking at the data would actually flag as unusual.

The reason this is worth mentioning in an article about LLMs, and not just a footnote about statistics: the model never had a say in this. It doesn't decide what counts as anomalous — a Python function does, tested against real data, before the model ever runs. The LLM's only involvement with anomalies is describing ones it's handed.

Making the system prompt do less, on purpose

Once the input to the model is fully pre-computed, the system prompt's job shifts from "understand this data" to "don't add anything to it." That turns out to need more explicit engineering than the reverse.

A few of the rules that ended up in the prompt, close to verbatim:

STRICT OBSERVATIONS RULE: The Observations section must reflect ONLY
the entries in "ANOMALIES DETECTED" from the input. Do not compare
daily values yourself to identify additional patterns. Do not mention
days that are not in the anomalies list, even if they appear lower
than others in the daily breakdown.

STRICT EVENTS RULE: [...] Never infer or state a correlation between
an event and a production anomaly yourself; only report a correlation
that is already marked as such in the input.

That second rule exists because of a specific failure mode: once event logs (inverter alarms, derating warnings) were added as optional input, the model started drawing its own conclusions about which events explained which anomalies — sometimes correctly, sometimes not, and with no way for a reader to tell which. The fix wasn't a smarter prompt asking the model to "be careful." It was moving the correlation logic itself into Python — matching event timestamps against anomaly days before the model ever sees either — and then explicitly forbidding the model from performing that inference on its own. The input to the model already says [matches anomaly day] next to any event that correlates; anything not marked that way, the model isn't allowed to connect.

The same pattern shows up elsewhere in the prompt: numbers must always come with context (no bare figures), causal language about anomalies must stay hedged ("possible", "worth checking") rather than definitive, and — maybe the least glamorous rule, but one that matters for a recurring report — no fact is allowed to appear in more than one section, so the same anomaly doesn't get described three different ways in one report.

None of these rules makes the model smarter. They make it narrower, deliberately, so the parts of the report that look like judgment are actually judgment that happened in Python, upstream, where it can be tested.

Where it stands today

This is v0.1, and it's honest about what that means:

• Data source is CSV only for now. A structured production-data source is easy to swap — Home Assistant's REST API and long-term statistics are next — but right now, getting your data in means a CSV with a timestamp and production_wh column.
• Events are opt-in and vendor-agnostic. A second CSV of logged alarms/events enriches the report when present; the tool works exactly the same without it.
• No retry logic beyond the Anthropic SDK's defaults, no streaming. Fine for a report generated on a schedule; worth knowing if a transient API failure matters to your setup.

Try it

Here's what a generated report actually looks like:

The repo, including the sample data and the exact system prompt above, is at github.com/contimarco77/solar-report. It runs a --dry-run mode that exercises the full pipeline — CSV parsing, aggregation, anomaly detection — without calling the API, if you want to see the mechanics before spending anything on a real report.

If you've hit similar grounding problems with LLM output in a different domain, or think the anomaly threshold is wrong for a use case I haven't considered, I'd genuinely like to hear about it.