I've spent the last year building small AI automations that actually run unattended — on a Raspberry Pi, a cheap VPS, and my laptop. Most AI automation content shows you the fun part (a prompt, a demo, a wow moment). Almost none of it shows you the part that decides whether the thing survives contact with reality.
Here are the tips and tricks that made the difference between scripts that die in a week and automations I haven't touched in months. Every one of these is boring. That's the point.
1. Never let the model be the only thing holding state
The #1 reason agent scripts fail: they keep state in memory or in the conversation, and when the process dies, everything is gone. Write every step to disk — SQLite is perfect for this. If your automation can answer "what was I doing, and what's left?", it can survive restarts, crashes, and reboots. On my Pi, every agent writes a checkpoint before and after each tool call. If it dies mid-run, the next invocation resumes instead of starting over.
2. Wrap every LLM call like it's an unreliable API
Because it is. Your wrapper needs: retries with exponential backoff, a timeout, token/usage accounting, and a fallback. I keep a tiny LLMClient class that does all four. Ten lines of retry logic has saved me more times than any clever prompt. If you're rolling agents by hand, this is also exactly the kind of plumbing I packaged up in the AI Agent Toolkit — the client, the agent loop, logging, and config as clean Python modules, $9 one-time. But you can (and should) build your own first; you'll understand it better.
3. Structure your outputs, don't parse prose
Never ask the model to "write a report" and then regex the result. Ask for JSON with an explicit schema, validate it, and retry on failure. I use a small validation loop: request JSON, attempt to parse, if it fails feed the error back to the model once. Two tries is almost always enough. Structured output turns a flaky text generator into a reliable data source.
4. Give tools tight contracts and narrow permissions
An agent that can "run any shell command" is a liability, not a feature. Give it specific tools: list_subdomains, fetch_url, write_report. Each tool has a defined input and output. Narrow tools are easier to test, easier to log, and much safer. This is the core idea behind the recon pipeline in my Bug Bounty Automation Kit — enumeration, scanning, and triage as discrete, auditable steps rather than one black-box "agent."
5. Log like you'll be debugging at 3 AM
You will be. Log the prompt, the raw response, the tool calls, and the timing — but log them to structured files (JSONL), not just stdout. Then a quick jq query tells you exactly where things went sideways. Timestamp everything. On a headless Pi with no monitor, your logs are your eyes.
6. Schedule with cron, not cleverness
People reach for Kubernetes and message queues for things that cron handles perfectly. A single crontab line plus a systemd timer for retries covers 95% of "run this every night" needs. Keep the scheduler dumb and the script robust. Complexity is a cost; spend it only where it earns its keep.
7. Make the last step a human-readable report
The best trick I learned: end every automation with a step that produces a short, readable summary — a markdown file or an email. Not a dashboard. Not a metric. One paragraph of "here's what I did and what I found." It turns a background process into something you actually check, which is how you notice problems early. My recon runs end with a markdown report that's genuinely pleasant to read.
8. Run it on weak hardware on purpose
If your automation runs on a Raspberry Pi with 2 GB of RAM, it runs anywhere. I test everything on the Pi first. It forces small memory footprints, honest about resource use, and it catches assumptions about speed and availability that would hide on a beefy machine.
9. Version your prompts and configs like code
Put prompts, model names, and thresholds in a config file, and commit it to git. When output quality changes, you can bisect. "It worked last Tuesday" is a solvable problem when your prompts are versioned; it's a mystery when they're inline strings.
10. Ship boring, fix clever
Get the dumb-but-reliable version running end to end before you optimize. A pipeline that produces mediocre results every single night is infinitely more valuable than a brilliant one that runs when it feels like it. You can improve quality later; you can't improve a thing that isn't running.
That's the whole playbook. None of it is exciting, and all of it compounds: state on disk, retries, structured output, tight tools, honest logging, dumb scheduling, readable reports, weak-hardware testing, versioned configs, and boring-first shipping.
Stack those ten habits and your AI automations stop being demos and start being infrastructure. Happy building.
Top comments (0)