When people talk about AI work, they usually talk about models, prompts, or agents.
But when I looked back at my last few projects, that wasn’t where my time went.
Where the time really went
Here’s what I kept doing over and over:
- Pulling documents from different sources
- Cleaning and reshaping them so the pipeline wouldn’t break
- Deciding chunk sizes again (and again)
- Fixing JSON outputs that were almost valid
- Re-running pipelines just to make sure nothing silently changed
None of this required creativity or deep thinking.
But skipping any of it caused problems later.
A small but real example
In one project, we updated a document extractor.
The content looked the same. But whitespace and ordering changed slightly.
That was enough to:
- Change embeddings
- Shift retrieval results
- Make comparisons with earlier runs meaningless
Nothing crashed. No errors.
The system just felt off, and debugging took hours.
The turning point
The big improvement came when we stopped treating setup as something flexible.
Instead, we:
- Applied the same ingestion rules every time
- Kept chunking and metadata consistent
- Made IDs stable across runs
- Validated outputs before they flowed downstream
Suddenly, when results changed, we knew why.
What automation is actually good at
Automation works best for:
- Setup steps
- Validation
- Re-runs
- Comparing outputs across time
It does not replace:
- Choosing the right approach
- Making trade-offs
- Interpreting results
That judgment still belongs to humans.
Takeaway
Most AI systems don’t fail because the model is bad.
They fail because the process around the model is fragile.
Making the boring parts predictable turned out to be one of the highest-leverage changes I’ve made.
Question for you:
Which part of your AI workflow feels boring, repetitive, but too risky to skip?
Top comments (0)