DEV Community: Billie M

How I use scheduled Codex jobs to build daily reports

Billie M — Thu, 06 Aug 2026 09:14:22 +0000

Most of my analytics job should not be done by an LLM.

The source metrics, dates and repeatable transformations should stay exact. I used to think that meant the whole scheduled job should be left to deterministic code, and I still kind of agree.

What changed my mind was not asking a model to calculate the report. It was putting a scheduled Codex job around reliable tools: run them, inspect their result, apply a small amount of bounded judgement, and leave behind an HTML artefact that explains what happened.

I now use that shape for two daily reports. One is mostly analytics. The other has a more editorial middle. The useful boundary is similar in both.

Keep the hard boundary boring

My billiem-analytics project collects website visitors, edge traffic, Google Search Console, DEV and Hashnode into dated local snapshots.

Those sources do not expose interchangeable numbers. Some are daily flows. DEV and Hashnode provide cumulative publishing counters. A real zero is different from an unavailable provider, missing configuration or failed collection.

The deterministic code owns those distinctions. It rebuilds the private dashboard and keeps the reporting periods consistent. Codex does not get to smooth over a missing source or turn an unreliable metric into a reliable one.

That is important because the final report is meant to be inspected, not merely produced. If one provider failed, I want that state to survive all the way to the page.

Give the model the awkward work around the edges

The scheduled job handles the surrounding work: run the existing commands, inspect their output, follow a bounded recovery rule and decide how to present the result.

That is a small role in the analytics workflow. Founder Brief gives the model more to do.

There, a Python collector gathers and ranks candidate material into an editorial packet. The scheduled run reads the packet, checks the strongest sources and useful discussion branches, rejects weak or promotional material, and writes a short briefing. Candidate IDs and validation keep the result connected to the collected evidence.

The judgement is real but limited. A fixed ranking can decide what to inspect first; it cannot comfortably decide whether an anecdote is useful, whether two observations form a pattern, or whether a source gap makes a conclusion too strong.

Codex can help with those decisions against written editorial rules. It still cannot guarantee access to a site. Authentication, access controls, rate limits and format changes remain actual boundaries. Flexibility is not the same as a scraper that can never break.

The report is what made the automation useful

Both jobs end in the same deliberately plain way: just chuck it on an HTML file.

That gives me a stable artefact instead of a chat transcript or terminal session I need to reconstruct. Dates, source states, comparisons and caveats are together in a form I can reopen.

Before the analytics report, checking the same picture meant opening several browser windows and lining up different periods myself. It was painful enough that I often did not bother. Now I get the nice graphs and the consistent dates in one private view.

That consolidation made a quiet period without publishing visible across the sources. It was not a surprising new metric. The useful part was finally being able to see the existing information together.

I also added the report to Raycast. Typing analytics opens it in Chrome and regenerates it as appropriate. That tiny access path matters more than it sounds: I am actually opening the report rather than deciding that a manual Search Console session can wait.

Recovery has to remain visible

The analytics job includes one narrow recovery rule. A sandboxed DNS or Keychain failure should be retried with the correct local permissions before the provider is declared broken. After that retry, every provider still receives its own explicit state.

This is useful orchestration, not a claim that the system heals itself. A local scheduled job can miss its time when the Mac is asleep or offline. Credentials expire. Providers change. Prompts can be wrong.

The model can respond to a known operational wrinkle without hiding a genuine failure behind a confident summary. That is the line I care about.

Tokens feel cheap enough to me that this extra layer is worth trying, and it does not need a frontier model. If a cheap model can wrap reliable code, handle a known failure path and occasionally surface something the fixed report would miss, why wouldn't I use it?

I have not arrived at the conclusion that every scheduled task needs an LLM. Plenty still need a plain scheduler and a script. The pattern works for me when the exact work stays exact, the judgement stays bounded, and the final artefact makes the whole run inspectable.

Want to talk about something I’ve written or built? Get in touch.

This article was adapted with AI assistance from an original article on billiem.uk. The original article was reviewed before publication.

The LLM was better at building a solver than playing the game

Billie M — Tue, 04 Aug 2026 09:47:21 +0000

I started this project because an LLM annoyed me.

I gave a very strong model 322, a small Dota 2 drafting game. The choices looked like the kind of work a computer should enjoy: repeated packs of players and heroes, visible ratings, familiarity scores, chemistry, rerolls and a simulated tournament at the end.

I was disappointed by how well the LLM did. I am not a Dota expert, and I had only started watching it occasionally again during the previous six months or year. I still seemed to be doing better.

The interesting engineering question was not how to write a longer prompt. It was how to replace the card-by-card language-model judgement with a deterministic policy, then test that policy without confusing improvement with luck.

A stochastic benchmark needs shared randomness

The browser history gave us a useful irritation and almost no reliable comparison.

My earlier manual record contained 50 runs with a 14% title rate. The LLM won once in nine attempts. Putting 14% beside 11% looks temptingly quantitative, but the random offers, rejected packs and opponent fields were not preserved. The samples were small, unpaired and produced under different choices.

That is not a model benchmark. It is a reason to build one.

The offline solver generated every random choice from indexed tapes. Policy A and policy B received the same player offers, hero samples, field candidates and tournament randomness for a given episode. We could then compare the paired result: did the new policy win this exact episode where the old policy lost it?

This is the common-random-numbers idea in a practical form. Sharing the luck removes a large amount of noise that has nothing to do with the policy change.

Keep the simulator separate from the policy

Before evaluating a strategy, we reproduced the game.

The public client and seven data files were frozen with SHA-256 hashes. Draft legality, automatic hero allocation, chemistry, scoring and the tournament were ported into a deterministic Python engine. Automatic allocation evaluates all 120 player-to-hero permutations, so reproducing that detail mattered.

The engine also kept an important semantic boundary. A finished roster has an exact score under the copied rules. An unfinished draft does not. A policy can estimate the future value of a player, hero or reroll, but that partial-state estimate is not made exact by giving it several decimal places.

Keeping the simulator authoritative meant we could change policies without changing what success meant.

The redraw budget is part of the experiment

Opponent fields introduced a slightly odd control problem. If a solver can redraw for free forever, “find an easier field” eventually dominates every other decision.

We declared three finite budgets instead:

K=1 accepts the first opponent field.
K=4 selects from four fields.
K=16 selects from sixteen fields.

The budgets were nested. The first field in K=1 was also the first field in K=4 and K=16; the first four were shared too. Policies could be compared at each budget without quietly receiving different field draws.

This was a small design choice with a large effect on the claim. A title rate without the field budget would not describe a reproducible policy.

Make the final test expensive to touch

The solver could be improved and improved and improved. That was satisfying, but it created the usual risk: every time a result influences the next change, that result has become part of training.

We separated train, validation and test tapes. Candidate policies first went through successive halving on shared training episodes. Weak candidates stopped early; survivors received more budget. The selected candidate then needed to pass a fresh validation gate before the test partition could be opened.

The gate required positive title-rate results, no meaningful top-four or top-eight regression, zero failures, deterministic replay and acceptable runtime. Once the final test started, policy code, features, model bytes, parameters, seeds and episode counts were frozen.

The controls themselves still needed scrutiny. An independent audit found that an early replay check compared duplicate in-memory runs rather than the public replay path, and that the first runtime gate reused warm policy caches. Both were repaired before the production claim. The audit separately recomputed the final statistics from the raw logs.

The inconvenience was deliberate. It stopped “one more tweak” from leaking the answer back into the candidate.

A small deterministic model was enough

The final test compared four generations of policy on 10,000 paired episodes at each field budget. Across all policy and budget combinations, that was 120,000 outcomes with zero failures.

At K=16 the progression was:

Policy	Title rate
Pick the highest player rating	2.30%
Hand-written heuristic	14.77%
Assignment and chemistry components	22.33%
Frozen 37-feature value model	27.45%

The final value model was ridge-linear, deterministic and trained offline on 8,511 action rows. It used features for role-feasible future strength, partial hero allocation, current and potential chemistry, familiarity, draft stage and remaining rerolls. It did not call an LLM while playing.

Against the previous component policy, its K=16 title improvement was +5.12 percentage points with a paired 95% interval from +4.24 to +6.00. Top-four and top-eight results improved at every budget too.

The complete draft loop took a median 47.1 milliseconds with a cold model build. That was roughly 1.8 times the component policy's latency, and the runtime policy made no large-model request for any offer.

Put the LLM around the decision loop

We try to solve everything with LLMs now, and we do not necessarily need to.

For 322, the runtime decision-maker became ordinary deterministic code. The LLM was much more useful around it: inspecting the browser game, reconstructing mechanics, implementing the engine, proposing policies, running controlled experiments, auditing weak controls and producing visualisations that made the improvement legible.

The result was quite a cool little statistical project. The useful boundary was not “LLMs are bad at games.” It was more specific: repeated numeric decisions with a reproducible state and measurable outcome did not need an LLM in the hot path.

If I continued the project, I would use the LLM to choose and build the next experiment. I would leave the next card choice to the solver.

Want to talk about something I’ve written or built? Get in touch.

This article was adapted with AI assistance from an original article on billiem.uk. The original article was reviewed before publication.

Free compute worth claiming in August 2026

Billie M — Mon, 03 Aug 2026 11:26:44 +0000

A monthly free-compute list is only useful if it is willing to delete things.

Between my July snapshot and 3 August, GitHub Models disappeared, SageMaker Studio Lab closed to new users, and the current Hugging Face documentation stopped supporting the broad free CPU Spaces claim I had made. Paperspace, meanwhile, turned up with documented free M4000 notebook access. Neon put Functions and Object Storage into a free beta. Managed agent runtime and browser automation became harder to treat as side notes.

I also had to correct my own July guide. GitHub had already announced that Models would retire on 30 July, but I still described it as a recurring public preview on 10 July. That was wrong.

This is the useful August shortlist: what to remove, what changed, and which kind of free allowance fits a particular side project. It is documentation-backed rather than a claim that I opened and tested every account. Capacity, regions, billing checks, data-use terms, and unpublished guardrails still matter.

Remove these from the old shortlist

GitHub Models retired completely, including its playground, catalogue, inference API, and BYOK endpoints. It is no longer a free model API.

SageMaker Studio Lab closed to new customers on 30 July. Existing users retain free sessions with 15 GB storage, 16 GB RAM, up to eight CPU hours or four GPU hours per day, subject to capacity. A new reader cannot claim it.

Hugging Face Spaces needs a narrower description. Normal Gradio and Docker compute Spaces are gated behind paid plans in the current documentation. A free personal account can still host up to two Gradio ZeroGPU Spaces, but that is queued public-demo GPU capacity rather than a free Linux box.

One more correction is smaller but worth keeping: Axiom Personal currently includes two datasets, not the three I listed in July.

The August additions worth knowing about

A genuinely free GPU notebook appeared

Paperspace Notebooks now documents C4 CPU and M4000 GPU machines on its Free Gradient plan. It includes 5 GB storage and permits one running notebook at a time. A free-machine session can last up to six hours.

That does not mean guaranteed GPU hours. Capacity may queue, and there is no published monthly-hour entitlement. It is useful shared notebook access, not a persistent GPU server.

Neon temporarily became more than Postgres

Neon Functions and Object Storage are available on the Free plan during beta. The current offer is restricted to us-east-2, retains logs for three days, and uses unpublished rate and usage guardrails. AI Gateway is not included on Free.

That makes Neon an interesting temporary full-stack backend, but a beta is not a promise. The underlying free Postgres plan remains easier to reason about: up to 100 projects with 100 CU-hours and 0.5 GB storage per project, plus scale-to-zero after five minutes.

Agent compute now has several shapes

Google Agent Engine is listed with 180,000 vCPU-seconds and 360,000 GiB-seconds per month. Billing must be enabled. The table does not establish that Code Execution sandboxes share this allowance, so I would use the claim narrowly: managed agent runtime.

Upstash Box provides ten concurrent boxes and five active CPU-hours each month. A box has 2 vCPU, 4 GB RAM, and 5 GB disk, freezes after an idle hour, and currently runs only in us-east-1.

Cloudflare temporary accounts solve a different problem. An unauthenticated agent can deploy a Worker and supported resources with wrangler deploy --temporary, verify the result, and return a claim URL. The account disappears after 60 minutes if it is not claimed. This is throwaway deployment compute, not a permanent anonymous account.

Browser automation is now its own free-compute category

Cloudflare Browser Run gives ten browser minutes per day and three concurrent browsers. Cloudflare's general-purpose Sandbox SDK is separate and has no Workers Free allocation.

Browserbase includes one browser hour, three concurrent browsers, three agent runs, and 1,000 each of Search and Fetch uses per month. Sessions stop after 15 minutes.

Browserless uses a 1,000-unit monthly pool with two concurrent browsers and one-minute sessions. One unit covers up to 30 seconds, while proxy and CAPTCHA work can consume more. It does not translate cleanly into a neat number of hours.

Pick by the way the allowance can fail

The word “free” hides several different products. The failure mode matters more than the headline number.

For a persistent Linux VM

OCI Always Free is still the first claim to try. The current supported allocation is 1,500 Ampere A1 OCPU-hours and 9,000 GB-hours per month, equivalent to 2 OCPUs and 12 GB RAM. The tenancy also gets 200 GB combined boot and block storage plus 10 TB monthly outbound transfer.

The catches are practical: home-region capacity can be unavailable, and idle instances can be reclaimed. AWS EC2 T4g is useful during 2026, but its 750 aggregate t4g.small hours per month are a promotion ending on 31 December rather than durable infrastructure.

For functions and containers

Cloudflare Workers has a recurring hard cap of 100,000 requests per day, 10 ms CPU per HTTP request, and 128 MB memory. Static assets do not consume the dynamic request allowance.

Deno Deploy publishes a broader monthly cap: 1 million requests, 20 GB egress, 15 CPU-hours, 350 GB-hours memory, 20 apps, 1 GiB volume storage, and 1 GiB KV storage. The old Deploy Classic migration deadline has passed, and projects did not transfer automatically.

Google Cloud Run gives request-based services 2 million requests, 180,000 vCPU-seconds, and 360,000 GiB-seconds monthly. It is attached to billing, so registry, build, network, storage, or overage usage can still charge.

For data and product plumbing

Turso is the cleaner hard-capped SQLite-shaped option: 100 databases, 5 GB storage, 500 million rows read, and 10 million rows written per month. Requests stop at quota rather than becoming an invoice.

Cloudflare D1 fits naturally when the application already lives on Workers. The free allocation is ten databases, 500 MB each, 5 GB total, 5 million rows read and 100,000 rows written per day.

Upstash covers Redis, QStash, Workflow, Vector, and preview Search under separate free quotas. Cloudflare R2 provides 10 GB-month Standard storage, 1 million Class A operations, 10 million Class B operations, and free egress, but enabling it requires account checkout.

For GPU and hosted inference

Modal Starter gives $30 of recurring compute credit for serverless CPU, GPU, notebooks, schedules, sandboxes, and web functions. Lightning AI Free gives 15 credits per month plus one active CPU Studio that restarts every four hours. The GPU time those credits buy varies with the selected hardware and current rates.

Google Colab remains shared and dynamic, with no stable hardware or quota promise. Sessions can run for up to 12 hours depending on usage and availability. Paperspace is now the more concrete free M4000 notebook claim, while ZeroGPU is for public Gradio demos.

Hosted APIs are another category again. Cloudflare Workers AI stops a Free account after 10,000 Neurons per day. Gemini has model-dependent free pricing and may use free-tier content to improve Google products. OpenRouter Free gives 50 free-model requests per day unless the account has purchased credit.

FreeInference is useful only with its privacy boundary visible: its terms allow prompts and responses to be logged and anonymised derived data to be released. Never send it secrets, private code, or personal data.

Three stacks I would still assemble from the list

For a small real server: OCI for the VM, Cloudflare for DNS, CDN, and TLS, R2 or Tigris for objects, Neon or Aiven for Postgres, Upstash for queues and Redis, Resend for mail, and Grafana Cloud plus Sentry for visibility.

For a no-server product: Cloudflare Workers and static assets, D1 or Neon, R2, Queues or Workflows, Turnstile, Clerk or AuthKit, and Workers AI.

For an AI demo: Paperspace or Colab for notebook work, Modal for repeatable bursts, ZeroGPU for a public Gradio surface, and Workers AI, Gemini, Groq, or OpenRouter for hosted inference.

The full August guide on billiem.uk keeps the broader directory, including email, auth, observability, CI, storage, startup programmes, and academic credits.

In 24 days, one model API disappeared, one notebook closed to new users, and a supposed free hosting offer became much narrower. The next version may remove as much as it adds. That is fine. A dated guide is more useful when it is willing to get shorter.

Want to talk about something I’ve written or built? Get in touch.

This article was adapted with AI assistance from an original article on billiem.uk. The original article was reviewed before publication.

I let Codex build and test my first native Mac app

Billie M — Mon, 03 Aug 2026 11:19:01 +0000

The first public Billie Flow build launched, stayed alive, passed its signature check and could find its source and models. A replacement install still left a new user with no Settings window, which meant there was no visible way to install the local worker.

The process was healthy. The app was unusable.

That gap is the part of my local dictation experiment that I find most useful. I had let Codex work well beyond a repository: it built my first native Mac app, installed it, operated the macOS permission surfaces, exercised physical audio and inspected the clipboard. The interesting failures only appeared because it kept going after the code checks were green.

Testing the installed app changed the work

I started Billie Flow because I use Wispr Flow and wondered whether I could make the useful loop run locally. What led me to it was just curiosity. Could I really?

It was never meant to become a replacement. Wispr Flow gives me enough value that I still use it. I wanted to see whether a personal version could be made, and then I became much more interested in how involved I let Codex become.

In 5.6 it could literally write the thing, build the thing, install the dependencies for the thing, test the thing. For this project, "test" eventually meant opening the installed Swift app, working through microphone permissions with Computer Use, holding the global shortcut and checking whether the resulting text reached the clipboard.

At one point I was lying there waiting for it to work away when speech started coming out of my speakers. Billie Flow recorded it and transcribed it. I do not know what produced that speech, so I am not going to turn the observation into a tidier technical claim. I remember thinking that 5.5 would not have taken the same initiative, but that is an impression rather than a controlled comparison.

The global shortcut records only while held; releasing it submits the temporary audio for local processing.

The physical run eventually finished with non-empty text on the clipboard, a healthy app and worker, no new crash and no temporary recording left behind.

This is the visible end state from the installed app after local recognition and light cleanup completed.

The expected model was not the useful one

The app work started with a model decision. I had an audio-capable Gemma 12B route in my head, but I ran the same 35.3-second voice memo through several local recognition paths before building around that assumption.

I also kept speech recognition separate from cleanup. Otherwise a cleanup model could make a transcript sound polished while preserving the important names the recogniser had already got wrong.

Gemma completed the memo in 258.79 seconds and drifted around an overlapping chunk. MLX Whisper large-v3-turbo produced the most useful recognition in about 3.68 seconds, then Qwen2.5 1.5B ran the selected light-cleanup pass in about 0.62 seconds.

The model I first had in my head was not the right answer. Two smaller models produced the quicker and more useful result for this app. That is deliberately narrow: it was one memo on one machine, and every recognition path still missed at least one important project term. The full Billie Flow model analysis contains the other branches, timings and vocabulary failures.

Three different kinds of green failure

The absent Settings window was not the only thing the replacement-install pass found.

Cleanup was silently falling back to raw recognition because the pinned MLX library was receiving an obsolete argument. The worker request returned success, but the UI was claiming cleanup that had not happened. A final setup check could also exit before the app registered completion and leave setup stuck at verification.

Earlier microphone tests had exposed an actor-isolation crash on the first audio buffer, then a format mismatch while writing the converted WAV. These were not five versions of the same bug. They crossed Swift concurrency, Core Audio, a pinned Python dependency, process lifecycle and visible macOS presentation.

Codex kept following each failure into the layer that produced it. The release gate now checks for an actual first-launch window and warning-free cleanup instead of treating a living process or successful response as sufficient evidence.

A public proof of concept, with a blunt boundary

The source and install-tested v0.2.1 build are public. I want Billie Flow to be usable should you want to, but I do not really care if anyone does.

Nothing large starts until Install is chosen; the disclosure also limits the proof of concept to English speech on Apple Silicon and macOS 26.

The boundary is explicit: Apple Silicon, macOS 26, English recognition and about 3.5 GB of consented runtime and model setup. Inference is local after setup, but setup still downloads Python dependencies and fixed models. The app is ad-hoc signed and unnotarised, with no updater, App Store release, compatibility promise or support plan. I am definitely not getting an Apple Developer Programme membership for it at this point.

Billie Flow does not establish that everybody should rebuild the software they already use. It shows that a surprisingly good personal version of this loop can now be made quickly. What I am going to remember is still the oddest part: lying there while Codex decided it needed to test the microphone and speech started coming out of my speakers. It was really really impressive.

Want to talk about something I’ve written or built? Get in touch.

This article was adapted with AI assistance from an original article on billiem.uk. The original article was reviewed before publication.

A rotation changed my embedding plot, not its neighbours

Billie M — Tue, 14 Jul 2026 15:18:33 +0000

One switch in my embedding animation makes the plot look substantially different while preserving every top-five cosine neighbour.

I did not start with that as a teaching point. I had been playing with embeddings while learning about AI, first by using them to modulate sound and then by embedding code and looking for similar code. PCA and UMAP projections seemed pretty cool. At some point the next thought just popped into my head: “Huh, it'd be cool to try and animate embeddings rather than just flatten them.”

I also thought, “Oh, this is just not going to work.”

It worked, which surprised me.

The changing picture

The working Embedding Tours instrument has a deterministic eight-dimensional synthetic dataset and 180 short phrases embedded into 384 dimensions with MiniLM. You can move through raw coordinate pairs, pairs of principal components, or a seeded grand tour through projection planes that are not aligned to the raw axes.

Those are not three names for the same thing. The raw tour cycles through coordinate pairs. PCA puts the highest-variance directions first, although that does not make them the true semantic axes. The grand tour moves through more general planes.

Every frame is still a two-dimensional projection. Motion gives me more partial views; it does not recover all the information lost when hundreds of dimensions are put on a flat screen. It is also just really satisfying to watch the structure move.

The raw-coordinate mode adds a more pointed comparison. Its basis switch applies the same orthogonal transformation to the whole dataset. Individual coordinates change and the raw plot can look very different. Dot products, lengths, distances and cosines do not change under that shared rotation.

The raw projection changes after the basis switch; cosine-neighbour geometry remains fixed to floating-point precision.

In this run, the measured similarity drift is 5.55e-16 and all top-five neighbours are retained. That is visible evidence for the narrow claim I wanted the switch to make. It does not prove that every transformation preserves an embedding, and it does not mean individual axes can never be useful. It shows that the cosine-neighbour structure does not uniquely privilege the raw coordinates I happened to receive from the model.

That limitation matters when interpreting embedding coordinates. A very different-looking coordinate tour can represent the same neighbour geometry.

I had walked into an existing idea

My first version just moved from dimensions 1 and 2 towards dimensions 2 and 3, then kept going. I later learned that this belongs to an established family of visualisations called tours.

Daniel Asimov described the grand tour in 1985. The tourr project gives the useful distinction: cycling through axis-aligned views is a little tour, while a grand tour moves through general projection planes. Distill used a grand tour for neural-network activations, and the recent dtour project has a much more complete browser interface for steering through high-dimensional data.

I had not invented a new visualisation technique. I had kind of independently arrived at a known idea, or at least the entrance to one. That was probably the coolest part.

The instrument stayed brief: two datasets, three projection modes, the basis comparison, selection and playback controls. I do not expect a researcher to discover a new technique in it. Maybe it helps someone like me who is still learning and finds embeddings confusing. Maybe it does not. Maybe it is just a cool experiment.

This article was adapted with AI assistance from an original article on billiem.uk. The original article was reviewed before publication.