I was automating my job search when my pipeline kept crashing. Fixing it became a whole project.

Herbert Yeboah — Tue, 17 Mar 2026 13:54:26 +0000

You might be wondering why someone would spend weeks building a reliability layer for a pipeline instead of just doing the job search manually in the first place. Fair question. I asked myself the same thing more than once. You will get it soon. Let me paint you the picture first.

Finding a job in Ghana after university is a sport. Not the fun kind. You can have the degree, the certifications, show up every day, and still be invisible because you don't know the person doing the hiring. I watched people send out application after application and hear nothing back. Not a rejection, just silence. That silence does something to you.
I finished national service at a bank and stayed on an informal contract while I figured out the next move. Couldn't start a business without capital I didn't have. So I thought, fine. I'll go around the problem instead of through it.
My plan was to build something I called Pathfinder. Just a guy with a laptop, automating the entire job search process with AI models, all for free, which was very necessary because I had no other option. The engineering team was me. The product manager was me. The CEO was also me, unfortunately. Research the role, match it to my background, write the cover letter and draft a CV tailored to that role. I review everything and submit myself, but at least the grunt work gets handled.
The pipeline kept crashing.
Timeout. Rate limit. Bad response from the model at step 4 after you've already spent your free tier quota on steps 1, 2, and 3. Everything gone. Start from the beginning. again.
I tried wrapping things in try/except. It just failed quietly instead of loudly. Still restarting from scratch every time.
For a few weeks I genuinely considered just doing it manually. That felt like the sane path honestly.
I am not a seasoned developer. I want to be clear about that, so when I went looking for solutions I had no idea LangChain or LangGraph even existed. Found them during research. Spent real time going through both. They are solid, genuinely well built. Just not built for what I needed. too many layers, too much to learn, just to solve one problem. I needed something that wouldn't lose my work when it crashed. That's it.
Then I started finding posts from other developers hitting the same wall. different projects, different goals, same frustration. Pipelines crashing mid run and losing everything, no clean way to resume. Enough people had this problem that something shifted for me, fixing it properly might actually matter to someone other than just me. It was worth the detour.
So I stopped working on Pathfinder completely and spent the next several weeks figuring out how to build a pipeline that simply couldn't lose its progress.
The solution was embarrassingly simple. Every step saves its output before the next one starts. Crash happens, re-run it, finished steps skip themselves, you continue from exactly where it stopped. I added a few things I kept needing along the way. routing simple tasks to free models instead of burning expensive ones on everything. Retrying when the model returns broken output. Checking at startup whether your models still even exist because free tier models get retired without warning and nobody tells you.
February 27 was the first commit.
Pathfinder is still in progress.
For a moment there, shipping something from scratch as a non-professional developer that other people might find useful had me feeling like the new Dario Amodei. I'll be honest about that. Then reality reminded me that it does make an impact, just not quite at that scale yet. But it's real, and that's enough to keep going.
If any of that sounds familiar, the library is called DagPipe. Might be worth a look,it is open source .I have a demo on it on YouTube as well as a breakdown of it Using Notebook lm.Feel free to skip to 7:05 where it gets into the actual technical detail.

DEV Community: Herbert Yeboah

I was automating my job search when my pipeline kept crashing. Fixing it became a whole project.