Most of my analytics job should not be done by an LLM.
The source metrics, dates and repeatable transformations should stay exact. I used to think that meant the whole scheduled job should be left to deterministic code, and I still kind of agree.
What changed my mind was not asking a model to calculate the report. It was putting a scheduled Codex job around reliable tools: run them, inspect their result, apply a small amount of bounded judgement, and leave behind an HTML artefact that explains what happened.
I now use that shape for two daily reports. One is mostly analytics. The other has a more editorial middle. The useful boundary is similar in both.
Keep the hard boundary boring
My billiem-analytics project collects website visitors, edge traffic, Google Search Console, DEV and Hashnode into dated local snapshots.
Those sources do not expose interchangeable numbers. Some are daily flows. DEV and Hashnode provide cumulative publishing counters. A real zero is different from an unavailable provider, missing configuration or failed collection.
The deterministic code owns those distinctions. It rebuilds the private dashboard and keeps the reporting periods consistent. Codex does not get to smooth over a missing source or turn an unreliable metric into a reliable one.
That is important because the final report is meant to be inspected, not merely produced. If one provider failed, I want that state to survive all the way to the page.
Give the model the awkward work around the edges
The scheduled job handles the surrounding work: run the existing commands, inspect their output, follow a bounded recovery rule and decide how to present the result.
That is a small role in the analytics workflow. Founder Brief gives the model more to do.
There, a Python collector gathers and ranks candidate material into an editorial packet. The scheduled run reads the packet, checks the strongest sources and useful discussion branches, rejects weak or promotional material, and writes a short briefing. Candidate IDs and validation keep the result connected to the collected evidence.
The judgement is real but limited. A fixed ranking can decide what to inspect first; it cannot comfortably decide whether an anecdote is useful, whether two observations form a pattern, or whether a source gap makes a conclusion too strong.
Codex can help with those decisions against written editorial rules. It still cannot guarantee access to a site. Authentication, access controls, rate limits and format changes remain actual boundaries. Flexibility is not the same as a scraper that can never break.
The report is what made the automation useful
Both jobs end in the same deliberately plain way: just chuck it on an HTML file.
That gives me a stable artefact instead of a chat transcript or terminal session I need to reconstruct. Dates, source states, comparisons and caveats are together in a form I can reopen.
Before the analytics report, checking the same picture meant opening several browser windows and lining up different periods myself. It was painful enough that I often did not bother. Now I get the nice graphs and the consistent dates in one private view.
That consolidation made a quiet period without publishing visible across the sources. It was not a surprising new metric. The useful part was finally being able to see the existing information together.
I also added the report to Raycast. Typing analytics opens it in Chrome and regenerates it as appropriate. That tiny access path matters more than it sounds: I am actually opening the report rather than deciding that a manual Search Console session can wait.
Recovery has to remain visible
The analytics job includes one narrow recovery rule. A sandboxed DNS or Keychain failure should be retried with the correct local permissions before the provider is declared broken. After that retry, every provider still receives its own explicit state.
This is useful orchestration, not a claim that the system heals itself. A local scheduled job can miss its time when the Mac is asleep or offline. Credentials expire. Providers change. Prompts can be wrong.
The model can respond to a known operational wrinkle without hiding a genuine failure behind a confident summary. That is the line I care about.
Tokens feel cheap enough to me that this extra layer is worth trying, and it does not need a frontier model. If a cheap model can wrap reliable code, handle a known failure path and occasionally surface something the fixed report would miss, why wouldn't I use it?
I have not arrived at the conclusion that every scheduled task needs an LLM. Plenty still need a plain scheduler and a script. The pattern works for me when the exact work stays exact, the judgement stays bounded, and the final artefact makes the whole run inspectable.
Want to talk about something I’ve written or built? Get in touch.
This article was adapted with AI assistance from an original article on billiem.uk. The original article was reviewed before publication.
Top comments (0)