Setting you up to fail

I saw the following in a job description:

Design agents that utilise LLMs for reasoning, document extraction, and the end-to-end automation of complex enterprise processes. You will implement AI best practices to ensure our autonomous agents act with precision, security and industrial-grade reliability.

(emphasis mine)

This is setting you up to fail and is a classical XY problem waiting to happen.

If you don't know the XY problem, it is a problem with a perceived solution that is wrong for the original problem.

LLMs are a lot of things, but in the end they are only communicators. What's happening with LLMs now reminds me of the things people do with git, just because they can, not because git is very good at the job. Like a completely new p2p internet over git, CMS systems built around git (yes, it works, but CMS's are meant for people who like WYSIWYG editors) and fun tools like taking a picture each time you do a commit.

The perception with LLMs is that they really can brew your coffee. And LLMs are a lot of things, but they are not precise, secure nor reliable. And so, because the CTO (or more likely the Marketing Manager turned 10x Vibe Coder) has already said the solution is LLMs, you are now going to see the following problems:

Precision

Well, how about counting letters in words or asking for words with 5 letters or painting any picture with text in it. It's really easy to see that LLMs are not precise. If they were precise you wouldn't need instructor, because yes, following instructions to the letter is also about precision. I think anyone working with LLMs for more than a few minutes knows that precision isn't a trait that can be attributed to LLMs.

Secure

There's a ton of anecdotal evidence that LLMs leak data and many more found and reported responsibly.

But the most important evidence is mathematical: guardrails will never be sufficient. This makes it a cat and mouse game and it won't be long till consultant agencies will step up, since they are already skilled in the tax evasion game, the stay compliant game and the aggressive accounting game.

Reliable

Related to precision and following instructions, reliability is often achieved in software by being deterministic: same input yields the same output. This isn't true for LLMs, even at temperature 0, drift occurs. This isn't a bug, it's a feature of the architecture.

Contrary to popular belief, LLMs are ill-equipped for reliable automation of repetitive tasks. There's no telling when it will do something it shouldn't, what the consequences are, if it will do it again and how likely that will be.

When routing your emails and flagging something as urgent, you may find errors not that important. You can laugh about hallucinated fake studies in reports that nobody reads thoroughly anyway.

But when your LLM based invoice reader hallucinates the wrong invoice amount and it's agent processes the payment, you may suddenly have a cash flow problem or have insulted a key supplier with a fraction of the amount due, who then delays shipping.

In Conclusion

Be wary of jobs with these kinds of descriptions. They already bought into the LLM solution and so are unlikely to listen to solutions that actually work. Such as Named Entity Recognition, tailored to invoices you see daily and not all the invoices in whole wide world and also cat pictures.

Like Sieve email filters that access your database to route supplier emails to the procurement department.

Like not giving an "Agentic" LLM an API key to modify your entire AWS environment, but use Principle of Least Privilege. Perhaps reconsider giving an LLM access to your bank account, your cloud drive or sending emails as you. The classic usability versus security tension.

All this makes your job to deliver precise, secure and reliable agentic platforms an impossible task. You will think you succeeded till that 2am phone call. Install "Prime Agentic AI security harness" by Snake Oil Guardrails Ltd and a few weeks later sue them for not stopping to send sensitive documents, because the attacker said it was a test.

You have been warned.