I wanted a small research database to grow from one subreddit to ten communities where people talk about building with Claude Code and automation. Doing it properly was estimated at about $118. It cost about $34, and the corpus went from roughly 1,000 chunks to 5,846. Overnight, while I slept.
The scraping ran through a paid provider (Apify) that charges per delivered record. The cleaning, tagging and indexing were my own Python. Around all of it sat one PowerShell script, which Claude wrote for me. That script is what this post is about, because the agent isn't the hard part. Anyone can make an agent do something impressive while they're watching. The hard part is the wrapper that makes it safe to stop watching.
Three rules.
1. Every phase is resumable
The script keeps its own state: chunks before and after, spend at the start and end, what it recovered and what it couldn't. All of it is set before the work starts and read again at the end.
Scraping is the phase that fails. Reddit blocks a scraper that hits too many communities in one session, with a 403. My first version asked for all ten subreddits at once. Four listings came back empty, including every Claude community, and the run still said it succeeded.
The fix wasn't a smarter agent. It was batching: three subreddits at a time, and anything that comes up short goes into a retry queue and comes back on its own after a pause. All four blocked listings came back that way. Because the provider only bills for delivered records, the failed attempts cost nothing.
Everything after scraping (cleaning, tagging, indexing) can run again on the same input without doubling anything, and the paid tagging step skips whatever it has already done. That's what lets the script retry without asking me. If it needed my permission to re-run, I'd have to be awake.
2. Judge success by the provider's number, not yours
The obvious check: count your rows before the scrape, count them after, and if the number went up, it worked.
It's wrong, and it's wrong in a way that looks exactly like a failure. The pipeline deduplicates on the way in. A second pass over a community pulls the day's top posts, and many of those are already stored from the first pass. So the provider can deliver hundreds of good records while your database barely moves. A check on your side calls that a dead run and pays to scrape it again.
So success is the provider's delivered-record count, checked against a minimum I set for each community. My side can legitimately show no change and still be fine.
Money works the same way. Before every paid attempt, the script reads the account's actual spend for the month from the provider and stops at a hard cap. It doesn't keep its own estimate. If something in my pipeline miscounts, the budget check can't be fooled, because it isn't counting anything. It asks the account.
3. Write the report even if it crashes
The end of the script is a finally block. Whatever goes wrong above it (an error, a failed scrape, the provider going down), it still builds a coverage table, writes a status banner, saves the report and opens it in Notepad. One caveat: finally covers errors inside the script. If the whole process is killed, nothing runs.
The first real report said PARTIAL: one community, one listing not recovered, named, with the command to re-run it. Not a failure, a to-do. The mop-up the next day brought back 47 records, which was everything that community had that day.
When you're asleep for the whole run, silence is the worst output. Silence can mean everything worked, or that it died in the first 30 seconds, and from the couch those look the same. The report is what tells them apart.
What went wrong anyway
Two things, both the same lesson.
Headless Claude Code holds back its output. Launched from a script, it prints nothing until it exits unless you tell it to stream. So the log file sits at zero bytes for the whole run, and a healthy agent looks exactly like a dead one. I now judge an agent by the timestamps on the files it's writing and its CPU time climbing, never by the log.
My summary tables were one row long. The script that builds them had a nested database cursor that reset the outer loop after the first row. Every table came out formatted and plausible, and wrong. Now I count the rows I expect instead of eyeballing the top of the file.
A run that reports success and is wrong is scarier than a run that crashes. The crash tells you. The plausible report doesn't.
Get the script
The script is free on GitHub, with a header marking which lines to adapt:
https://github.com/lgcreativestudios/buildwithaihub/blob/main/scripts/v1/phaseC_overnight_orchestrator.ps1
It runs locally in PowerShell on Windows, reads your own Apify and OpenRouter keys from your user environment variables, calls those two services with them, and writes only local files. Read it before you run it.
Here's the 7-minute walkthrough, with the real report on screen:
Tools I used to make the video
I'm an independent affiliate of ElevenLabs, Higgsfield and vidIQ. If you sign up through these links I may earn a commission, at no extra cost to you.
- ElevenLabs voices the narration: https://try.elevenlabs.io/n5brzq80coed
- Higgsfield made the presenter images and thumbnail options: https://higgsfield.ai?fpr=buildwithaihub
- vidIQ scored the title options and rendered the animated title cards: https://vidiq.com/buildwithaihub
I also turned these rules into a short book, It Ran While I Slept: five rules instead of three, the whole wrapper annotated, and a one-page pre-run checklist. Every store in one link: https://books2read.com/u/4DLP87
I'm building the whole system in public. If you want the next pieces as they land, there's a waitlist: https://lgcreativestudios.github.io/buildwithaihub/
Drafted with Claude's help from the video script. The run, the code and the numbers are real.
ElevenLabs and the ElevenLabs logo are registered trademarks or trademarks of ElevenLabs, Inc. Used with permission. buildwithaihub is not sponsored by, endorsed by, or affiliated with ElevenLabs except as an independent participant in the ElevenLabs Creator Affiliate Program.
Top comments (5)
"Judge success by the provider's number, not yours" generalizes further than scraping — it's the same reason I've stopped trusting an unattended agent's own log for liveness at all, for the exact failure mode you hit with headless Claude Code holding back output. A log that's silent because the run is healthy and one that's silent because it died at second 30 are indistinguishable from the outside, so I now write an external heartbeat file with a plain timestamp after each real step completes, independent of anything the agent itself decides to print. A supervising process (or a human) can poll that file's mtime and know within one interval whether it's stuck, without parsing stdout or trusting the agent's self-report — same principle as reading spend from the provider's account instead of your own running total: the check has to live outside the thing being checked, or a bug in that thing can corrupt the check too.
Your nested-cursor bug is the same story from the other direction: a plausible-looking one-row table is a report the pipeline trusted about itself. The fix you landed on — count the rows you expect instead of eyeballing — is basically a cheap invariant assertion, and I'd bake that pattern in everywhere a "finally" block writes a summary: assert shape/count before formatting, not just after, so a formatting bug can't silently launder a data bug into something that looks like success.
Timestamps-on-files-plus-climbing-CPU-time is a good liveness proxy — it's checking that work is actually happening, not just that a process exists or a log line was written recently (which a stuck loop can fake). And yes, the row-count assert belongs at the very top of the finally block specifically because that's the one place a formatting bug can't have run yet to mask it. Good luck building the rest of this in public — "owning your agents" as a category is exactly the gap between demo and overnight-run reliability.
Thanks, Hamid. That's the distinction I'd underline too: a stuck loop can keep writing log lines, but it can't fake the output files growing or the provider's counter moving.
Agreed — output-growth and provider-counter movement are both hard to fake by accident, which is exactly what makes them good liveness proxies versus a log line that just proves the loop is still executing code, not that it's making progress. Nice wrapper.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.