Hey, it's oji_ai_dev here. I'm a 38-year-old developer building AI agents and automated trading bots on the side.
Today, I want to talk about a super subtle, yet critically important, infrastructure setting. My home server PC crashed overnight, and when I woke up, some of my bots were completely dead. I initially thought it was just a typical PC hiccup, but after digging in, I found it was a silent failure caused by my own human error. These kinds of silent failures are truly brutal.
What Happened: My Dashboard Was Too Quiet This Morning
It all started when I woke up and checked my custom monitoring dashboard. Several bot logs, which should have been running, had stopped dead around 3 AM.
"Ah, the PC crashed again."
This happens occasionally. Sure enough, the Event Viewer showed an unexpected shutdown record. Probably a power flicker or something. But here's the kicker: I have the PC set to auto-restart, yet out of my 11 running bots, 4 simply hadn't started back up.
No error logs. They just weren't executing. This is the worst kind of "silent failure." It leads to data loss and missed opportunities. What was the difference between the running bots and the dead ones?
The Cause: One Tiny Checkbox
I started comparing the Task Scheduler settings for the 7 bots that restarted successfully and the 4 that remained silent.
I found it almost immediately. The culprit was this setting in the "Conditions" tab:
"Start the task only if the computer is on AC power"
The working bots had this box unchecked, meaning they would start even if the PC was on battery. The 4 dead bots, however, all had this box checked.
When the PC shut down overnight and restarted, for some fleeting moment, the OS must have detected it as "on battery power." Any tasks scheduled to start at that precise moment were skipped because they weren't on AC power. That was the truth.
Why did I miss this setting? Tracing back my memory, a few months ago, I had wanted to ensure all bots would reliably restart after a power outage. I went through and updated the settings for all my active bots. I changed 7 of them correctly, but completely forgot the remaining 4. A classic manual rollout mistake. It really drove home how dangerous assumptions can be.
Fix and Prevention: Auditing Settings with Code
Once I knew the cause, the fix was simple: open the settings for the 4 problematic tasks and uncheck the box.
But that's not a fundamental solution. I could make the same mistake again. So, I decided to build a system to verify and audit settings using commands.
First, individual task settings can be exported as XML using the schtasks command:
schtasks /Query /TN "MyBotTask" /XML > task.xml
Then, I could open this XML file and check for <DisallowStartIfOnBatteries>true</DisallowStartIfOnBatteries>.
But doing this for all 11 bots is tedious, and I'd surely forget again. This is where PowerShell comes in handy. I wrote a simple script to iterate through all tasks in a specific folder and list any that don't have the desired setting.
# Detect tasks under the '\MyBots\' path that are configured to NOT start when on battery power (true)
Get-ScheduledTask | Where-Object { $_.Settings.DisallowStartIfOnBatteries -eq $true -and $_.TaskPath -like '\MyBots\*' }
Running this periodically, or whenever I deploy a new bot, will prevent this specific configuration oversight. Ideally, I'd use something like Ansible for configuration management, but for personal side projects, even a simple auditing script like this is incredibly effective.
Lesson Learned: Manual Configuration Rollouts Always Lead to Accidents
The lesson from this incident is simple:
Manual rollouts of configurations will inevitably lead to accidents.
This is a given for large-scale systems in my day job, but with personal projects, I get complacent, thinking "only I touch this." However, as the number of bots grows, my memory becomes unreliable. I learned firsthand how dangerous it is to assume "I've done everything."
System robustness isn't just about fancy algorithms or the latest AI models. It's supported by every single, mundane infrastructure setting. A single checkbox can bring your system's availability to zero.
Time is limited for side projects. That's why I should have built a system to manage and audit these "set-and-forget" parts of the infrastructure with code from the beginning. Spending half a day troubleshooting is far less productive than spending that time building new features.
Personal development offers freedom, but it also means you're solely responsible for your infrastructure. This failure was a good opportunity to re-emphasize that responsibility.
I build and run small Python systems — trading bots, RAG APIs, scheduled automation — and write up whatever breaks along the way.
If a provider-agnostic RAG Q&A API is useful to you, mine is MIT-licensed on GitHub: rag-faq-api. It runs and passes its full test suite **with no API key* (offline stub LLM + hashing embedder), swaps to Claude / Gemini / OpenAI via one env var, and ships a retrieval-quality harness (Hit@k / MRR / Recall@k) with a chunking sweep.*
Top comments (0)