My oldest actor is also my most profitable one - it's caught more paying clients than anything else I've shipped. Which made it worse when I found out it had been silently failing on a large share of its runs, for a while, and I only caught it by accident.
The actor watches CBOE options data on a schedule and flags contracts trading at volumes far above their normal baseline - the kind of activity that sometimes precedes real news. Every step of the actual detection logic was already wrapped in error handling. Except one: the very last line, the one that sends the results out.
That line, Actor.push_data(results), had no try/except around it. If a single row in the batch was malformed - one bad character, one field that didn't serialize cleanly - the whole call could fail, and with it the entire run, even though every ticker had already been checked correctly. All that work, thrown away by one line at the very end.
Apify doesn't show actor owners the logs of runs triggered by clients, only your own test runs. So this had been failing quietly, on other people's schedules, with no way for me to see why - just a success rate that didn't look right.
The fix: wrap the push in a fallback - try the whole batch first, and only if that fails, retry row by row so one bad row gets dropped instead of taking the rest down with it. Only raise for real if literally nothing could be sent. Tested against a forced bulk failure, a single bad row, and a total outage before shipping it - each one now behaves the way it's supposed to.
I also added a last-resort email alert: if a run ever fails end to end now, for any reason, I get a message instead of finding out weeks later from a success-rate graph.
Top comments (0)