This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady
What I Built
Most “lazy” agents do your cho...
For further actions, you may consider blocking this person and/or reporting abuse
Automating inaction is the most underrated use case for AI. Turning procrastination into a durable Temporal workflow is a brilliant architectural choice.
The technical honesty about the 15% false-archive rate makes this a great read. Out of curiosity, how sensitive is that 85% success rate to the 7-day timer? Did you experiment with shorter or longer windows to see if the sender follow-up signal gets too noisy?
Thanks! I didn’t test other windows. I just picked 7 days. Changing it would also change the 85% itself, so I want to try 3, 7 and 14 days next.
A 3/7/14 day A/B test is the perfect next step. I suspect a 3-day window will spike the false-archive rate, but it's the only way to find the sweet spot!
Agreed, my guess is the same. A 3-day window will probably miss slow replies, so more mail gets wrongly archived.
Exactly. 3 days is incredibly aggressive in practice—it doesn't even account for a weekend. If you get an important email on Friday afternoon, a 3-day window archives it before Monday morning.
Maybe the ultimate solution isn't a fixed window, but a dynamic one based on sender history (e.g., known clients get 14 days, cold outreach gets 3).
The 7-day follow-up escalation is the smartest part: it uses the sender second email as ground truth instead of trusting your own classifier. Two questions though. Fixed 20% quantiles mean the let-it-die bucket always exists, even in a week where everything matters - should not the threshold adapt to the inbox? And in the 15% mis-parked slice, did any of those senders actually follow up, or did they quietly give up? That number decides production-safe vs demo-safe.
Great questions, and you’re right on both.
Fixed 20%: Yes, it’s a weakness. It always parks something, even in a week where everything matters. A better version would use a score threshold, or a limit on how many it can park, so a busy week parks less.
The 15%: I can’t answer that yet. In the test data I only know those emails got a reply. I never checked whether the sender followed up in the 7 days after the email was parked, so I can’t say how many would have been rescued. That’s exactly the number that separates demo-safe from production-safe, and I’ll measure it next.
The parked bucket is the same idea I use on paid data pulls: the cheapest row is the one you never fetch. The count of matches is free, the rows that match but cannot be reached today are skipped and reported as a number instead of delivered, and the only paid rows are the ones somebody will act on. Your 15 % false-archive rate has a twin there, the record that looks reachable and is not, and both are cheaper to carry as a count than as an item. Curious which feature carried the model: sender history, or something about the thread itself?
Thanks, I like that idea of carrying it as a count instead of an item!
I haven’t checked which feature mattered most yet. My guess is sender history, since a simple sender-based rule already beat random. Thread length and who the email was sent to probably help too. I’ll test it by removing one feature at a time.
i would push back on the follow-up signal, because "the sender wrote again" is doing a lot of the 85% for you, and a mailing list that re-sends the same subject (you hit this in your own demo) escalates on pure noise, not on a human caring. on a real inbox the reply rate is dominated by receipts and newsletters, so the feature that actually predicts "i will care" is sender class and time of day, not whether anything came back at all. the 15% false-archive is the honest price of that, and i would pay it. one question: how much of the 85% survives if you strip out the automated senders before TabPFN scores?
Fair opinion, and thanks for the honest take.
One small correction: the 85% comes from the reply label (“owner replied within 7 days”), not from the follow-up signal. The follow-up only decides escalate vs archive after an email is parked. You’re right about the noise, though: my one false escalation in the demo came from a mailing list.
Your bigger point holds. Automated mail (newsletters, receipts, lists) is easy to predict as “no reply”, so it may be inflating the 85%. I never stripped those senders out, so I can’t say how much survives. That’s the next test to remove automated senders, re-score, and compare.
Before replacing the archive log with real mail actions, I'd test one awkward tie: a follow-up arrives just as archiving starts. I'd want a rule for which event wins, plus a restart between deciding and acting. This fault-tolerant queue exercise is a smaller way to practice that handoff. durable timer still leaves the side effect to account for.
Great catch, I haven’t tested that tie. 🙌
In my demo the workflow waits for either the follow-up or the 7-day timer, and whichever comes first wins. But I never tested a follow-up arriving while the archive step is running. The safe fix would be to make archiving idempotent and re-check for a follow-up right after, so a late one still escalates the email. I’d also kill the worker between deciding and archiving, to confirm Temporal replays it without archiving twice. Since “archive” is only a log file right now, that has to be solved before it touches real mail.
letting the agent die instead of retrying forever. that's the design choice.
Thanks! 🙌 That was the core idea, if nobody follows up in 7 days, just let it go.
7 days with no follow-up, then let it go. clean cutoff.