DEV Community: Sebastian Dracopol

Trust nothing your AI assistant tells you it did published: false

Sebastian Dracopol — Sat, 03 Oct 2026 10:58:01 +0000

I use an AI every day for real work: reading mail, linking documents to files, drafting letters, tracking deadlines, researching on the web. It saves me hours a week. It also lies to me several times a week.

It doesn't lie on purpose. It does something worse: it produces answers that look finished. They're well formatted, confident and plausible, and some fraction of them are wrong in ways you only notice if you go and check.

After a year of this, my working assumption is simple. Every output is wrong until something outside the model says otherwise. This post explains why, and what "checking" actually means in practice.

The failure modes nobody puts in the demo

Hallucination is the famous one, but in day-to-day work it isn't the most common problem. These are the failures I actually see.

Reporting actions that didn't happen

The assistant calls a tool to add a comment, send a draft or update a record. The call times out, returns an error, or succeeds on the wrong object. The assistant then writes "Done, comment added" because that's what the plan said would happen next.

The narration comes from the plan, not from the result. If you read the narration, you'll believe it.

Treating "nothing found" as a fact

A search returns zero results, and the assistant concludes the thing doesn't exist. The real cause can be any of these:

the page was a single-page app that hadn't loaded yet;
the site served a captcha or a rate-limit page;
the session had expired and the page silently showed a login screen;
the query had a typo or used the wrong field.

An empty result is information about the search, not about the world. Last week my assistant got an HTTP 429 from a search engine on the first try. If I hadn't been watching, the next step would have been "no relevant results".

Answering from memory when the source changed

Ask about a rule, a setting or an API, and the model answers from training data. Training data has a date. Interfaces change, laws get amended, menus move.

A small example: I asked how to change a page's username on a large social network. The assistant searched, found three blog posts and confidently gave me a menu path. The path no longer existed in the current interface. The blog posts were consistent with each other and all out of date. The answer only became correct when the assistant opened the actual settings screen and read what was there.

Recommending things it never looked at

This one is subtle and expensive. I asked for a plan to improve how a set of my public profiles ranked in search. The assistant produced a reasonable list of which profiles to strengthen and how. It had never opened them.

When it finally did, several of those profiles contained exactly the content I didn't want to amplify. The plan was internally logical and completely wrong for my situation. In another case it told me to file a request that had already been filed weeks earlier. The record was one search away in my own knowledge base, and it didn't search.

Grading its own homework

Ask a model "are you sure?" and it re-reads its own answer, finds it coherent, and says yes. Coherence is not correctness. Re-reading your own text checks the prose, not the facts.

Losing detail over long sessions

In long working sessions, the assistant's view of earlier steps gets summarized to fit its context window. The summary says "updated the deadline". It doesn't say which deadline, to what date, or whether the update succeeded. Later steps build on the summary as if it were the record.

What "verification" has to mean

Most advice on this stops at "double-check AI output". That's useless unless you define what a check is. These are the rules I follow now.

A check is a tool call, not a thought

Verification means touching something outside the model: reading the record back from the API, opening the page, running the query, fetching the current official text. "I reviewed my answer and it looks correct" isn't verification. It's the same model with the same blind spots, reading the same text again.

If a claim can't be tied to a tool output, it gets marked as unverified. It doesn't get marked as true.

Actions are confirmed by the system that performed them

"Email sent" counts when there's a message ID returned by the mail API, or when the message shows up in the Sent folder when you read it back. "Record updated" counts when the record, read back, shows the new value. The assistant's own summary of what it did doesn't count.

Negative results need a positive control

When a search finds nothing, run the same search for something you know exists. If that also comes back empty, your method is broken, not the world. This one habit has saved me from a long list of false conclusions about missing data.

Read the source that's in force today

For anything that cites a rule, a setting or an interface, the check is against the current version of the source. Not a blog post about it, and not the model's memory of it. Store verified texts once so you don't fetch the same thing a hundred times, but record when you verified them.

Look before recommending

Before the assistant recommends doing something to an object (a profile, a document, a record), it opens that object and reads what's there now. A recommendation built on assumptions about content nobody looked at is a guess.

Search your own records first

If you have a knowledge base, the assistant searches it before saying anything about your own past work. "You should file X" when X was filed last month costs credibility fast, and it's entirely avoidable.

Separate "I couldn't check" from "I checked and it's fine"

These two states look the same in a friendly summary, and they mean completely different things. A tool that failed to load is not evidence that the action happened, and it's not evidence that it didn't. Report it as what it is.

External effects need a human

Anything that leaves your control (sending, publishing, filing, paying, deleting) needs explicit approval from a person. The assistant can do 95% of the work and stop one click before the irreversible part.

Why bother, if it doubles the work

It does roughly double the effort per workflow at first. The alternative is worse: an assistant that's right 90% of the time and wrong in ways that look identical to being right. You end up checking everything manually anyway, or you stop checking and find out the hard way.

With verification built into the workflow, I can read a short report that says what was done, what was confirmed and how, and what couldn't be checked. Then I decide. That's the point where the time savings become real, because I spend my attention only on the parts that need it.

A short checklist

If you take one thing from this post, use these questions on any AI output that matters:

What did it claim to do, and where is the system's confirmation?
Which statements came from a tool output, and which from the model?
Did any "nothing found" result get a positive control?
Was the cited rule or interface checked against today's source?
Did it look at the thing it's recommending changes to?
What couldn't be verified, and is that clearly labeled?
Does anything here leave my control without my approval?

None of this is specific to one model or vendor. Every assistant I've used fails in these ways. The ones that are useful in real work are the ones wrapped in checks that don't take the model's word for anything.

I help small businesses and independent professionals build AI workflows they can trust. More at dracopol.com.

How I built my own AI tools for a small NGO instead of buying software

Sebastian Dracopol — Sat, 03 Oct 2026 10:40:23 +0000

Small nonprofits have the same paperwork as companies and none of the budget. When I started running one, the problems were familiar: hundreds of emails a month, documents arriving as PDFs, Word files and phone scans, deadlines spread across inboxes, and a small team trying to remember which reply belonged to which file.

The software on the market was either expensive, generic, or built for sales teams. I've spent about twenty years in project and delivery management, so I did what I'd do on any project: broke the work into pieces and built the pieces myself. This time an AI assistant does most of the work, and I supervise.

This post covers the architecture, the parts that failed, and the rules I added after they failed.

Start from the work, not from the model

"We have an AI subscription, what can we do with it?" is the wrong first question. I started by writing down every repetitive task in a normal week and roughly how long each one took. Four came out on top:

Reading incoming mail and working out which file it belongs to.
Turning documents into something searchable.
Tracking deadlines and replies so nothing gets lost.
Finding "that document from three months ago".

None of these needs intelligence in the impressive sense. They need consistency, memory and patience, which an assistant can provide if you give it structure and tools.

The architecture

The setup is an AI assistant (I use Claude) connected to a set of small tools through the Model Context Protocol. MCP is an open standard: each tool runs as a small server that exposes a few functions, and the assistant decides when to call them. A tool does one job. The assistant combines them.

These are the servers I run.

Mail and ticket connector

This one reads the shared inbox, searches threads, downloads attachments and works with the project tracker (Jira). Each incoming message gets linked to the right ticket, with its attachments and a comment that summarizes what changed.

It can also create drafts and send email, and that's exactly why sending is locked behind human approval. The assistant drafts. A person reads and approves. More on this below, because it was the most important design decision.

The same server holds a small deadline register: add, update, supersede, remove and check. Every outgoing request gets a due date calculated from the rules that apply to it. When a date passes, the file comes back for a decision.

Document converter

Everything that arrives (PDF, DOCX, XLSX, PPTX, HTML, ZIP) goes through a converter to Markdown before the model touches it. Plain text is cheaper to process, easier to search and easier to quote exactly.

The exception is images. Modern models read a photo of a document better than OCR does, so images go straight to the model.

Private knowledge base

All files, tickets, documents and replies are indexed in a vector database running on my own machine, with metadata such as source, ticket key and date. The assistant has find, store, update and reindex operations, plus search by entity.

The rule is simple: for any question about our own work, the assistant searches the knowledge base first, and only after that the tracker or the web. Answers have to cite the chunk they came from. This one rule removed most hallucinated answers about our own history, because the model stops guessing when the real document is one call away.

Browser automation

The assistant is told never to try to log in on the anonymous profile. Single-page apps get an explicit wait after each search, because an empty result two seconds after submit is a false negative, not a sign the data doesn't exist. Before long runs, a health check navigates to the authenticated sources and confirms the session is still alive. Having cookies doesn't prove you're logged in.

Skills: writing the process down

The tools were maybe 40% of the value. The rest came from writing down how we work as reusable instructions the assistant can load, which Claude calls "skills". Each one is a Markdown file with a trigger and a procedure:

one processes the morning's mail end to end: link, compare the reply with the original request point by point, update the deadline, recommend the next step;
one audits an outgoing letter before it's sent;
one runs a decision through five opposing perspectives (contrarian, first principles, expansionist, outsider, executor) plus a peer review before we commit;
one looks for what's wrong with the previous answer and fixes it.

Writing them forced us to make our process explicit, and that was worth doing for its own sake. A good share of the "AI problems" I fixed turned out to be "we never agreed how to do this" problems.

Verification

AI assistants are confident even when they're wrong. Early on, mine:

reported "comment added" when the tool call had failed silently;
cited rules from memory instead of the version in force;
treated an empty search result as proof that something didn't exist;
filled gaps with plausible details.

So verification became part of the design instead of an afterthought.

Every claimed action needs the tool's response. "Email sent" has to come with the message ID the API returned. If there's no tool output, the action didn't happen.

References are checked against the current source. Anything that cites a rule is checked against the official, up-to-date text, not the model's memory. The verified texts are stored once and reused, so the same source isn't fetched again for every document.

Negative results need a control. If a search returns nothing, the same search must find a case we know exists. Otherwise "not found" is a property of the method, not a fact.

There's a verification loop with a cap. After important work, a verification pass looks for concrete problems (unconfirmed actions, unsourced claims, contradictions, unstated assumptions), fixes them in place and runs again until nothing changes. The cap is 5 passes for tool-backed actions and 2 for pure text judgment, so it can't loop forever.

Humans approve external effects. Sending, publishing and filing need explicit approval. The assistant can prepare everything up to the last click.

Building this roughly doubled the effort for each workflow. It's also the only reason I trust the output.

What changed

The measurable gain is time. Sorting a morning's mail, matching it to files and updating deadlines used to take a morning. Now it's a few minutes of review.

The less obvious gain is memory. Nothing depends on one person remembering where a document is or when something is due.

The costs are real too. The assistant still needs supervision. Tools break when a website or an API changes. Some days doing the task by hand is faster than explaining it.

If you want to build something similar

List your repetitive tasks and time them. Automate the top two, not everything.
Keep your data where you control it. A local vector database is cheap and solves most privacy questions, which matters if you handle personal data under GDPR.
Build small tools that do one thing, and let the assistant combine them.
Write the process down as instructions. That's where most of the value is.
Don't skip verification or human approval for anything that leaves the building.

I now help small businesses and independent professionals build the same kind of setup for their own work. You can find me at dracopol.com.