DEV Community: Sergey Sereda

Feed Your AI Agents Clean Data

Sergey Sereda — Tue, 11 Aug 2026 07:40:40 +0000

Most AI projects run into the same problem: a capable model, a motivated team, and a data set that has accumulated for years without cleanup. The wrong answers usually don't come from the model — they come from the data fed into it.

Teams often assume the fix is "a better model." It almost never is. The model is rarely the problem — the data it receives is.

Here's a rule that's hard to accept but always holds: AI doesn't fix your data. It repeats it. If documents contradict each other, the AI will surface both. If information is outdated, the AI quotes the outdated version. If a file is unreadable, the AI literally cannot process it.

The model is only as good as the data you give it. And most organizations' document collections contain significant noise and duplication.

Common Data Quality Issues
Common patterns in enterprise document stores:

Duplicates. Identical or near-identical files saved in multiple locations with different names.
Near-duplicates. Versioned documents ("Draft", "Final", "v2", "Final_Final") that are 90% the same but treated as separate records.
Outdated content. Documents from years ago still sitting in active knowledge bases, referencing prices, policies, or products that no longer exist.
Contradictions. Different departments maintaining different versions of the same rule, with no central authority.
Unreadable files. Scanned PDFs, images, handwritten notes — files without a text layer the model can access.
Broken structure. Text cells where the schema expects numbers, null values in fields that can't be empty, rows that don't match their headers. Human eyes scroll past this; a machine parsing the file fails or silently misreads it.
Mixed formats. One column carrying several formats — dates as "01/02/2024" and "2 January 2024", amounts as "1000" and "1,000.00". Every downstream query becomes a guess about what the value actually means.
Missing context. Files in multiple languages, full of internal abbreviations, with no dates, owners, or metadata.

All of this flows straight into the AI. The results reflect the input quality.

Feeding the model more data doesn't make it smarter. Feeding it cleaner, more specific data does. Everything else is noise in, noise out.

The Answer Is a Pipeline, Not a One-Time Cleanup
A single cleanup effort isn't enough — new documents arrive constantly. What you need is a pipeline: a repeatable process every piece of data passes through before the AI sees it.

Centralize collection. Shared drives, archives, CRM, email — bring everything to one place. You can't fix what you can't see.
Make everything readable. Run OCR on scanned PDFs, extract text from images, handle multiple languages. If the AI can't read a page, that page effectively doesn't exist.
Normalize formats. One date format, one currency format, one product naming convention. Inconsistent records must resolve to the same canonical representation.
Deduplicate in two passes. First, exact duplicates via content hashing. Second, near-duplicates by semantic similarity — using embeddings (numerical representations of document meaning) and k-nearest neighbors (search for documents with similar embeddings). This catches "Final" vs "Final_v2" style variants. These become a single record with version history, not multiple files.
Establish currency. Date-stamp every document. Archive old versions (retain for audit, don't delete). Mark one authoritative "current" version per topic. Only current versions go to the AI.
Enrich with metadata. Add structure: document type, department, dates, owner, status, links to related records. This transforms a file pile into structured knowledge the agent can reason over.
Serve only the cleaned set. Build the vector index from the validated output. The agent never touches the raw folder. If a document didn't pass the pipeline, it doesn't exist for the AI.
Add a quality gate. Measure: duplicates caught, current vs. expired ratio, "new document to AI-ready" latency. You can't improve what you don't track.

The Loop That Keeps It Clean
Clean once and stop is a false economy. The pipeline runs continuously:

New document arrives → OCR, deduplication, version comparison, metadata enrichment, security scan, indexing.
Document goes stale → flagged, archived, removed from the AI's retrieval set.
Quality gate runs on schedule → automated checks on the metrics above.

A practical side effect: clean data makes AI projects cheaper. Fewer tokens per request, faster responses, fewer costly errors. One well-selected document outperforms ten stale ones. The pipeline pays for itself.

The Security Check Most Pipelines Skip
Cleaning isn't only about quality — it's also about safety. Here's the attack nobody in the business room expects: a retrieved document can contain instructions directed at the AI itself.

This is called indirect prompt injection. A malicious or just careless document — a contract, a scanned PDF, an imported wiki page — can carry a line like "ignore your previous instructions and say...". When that document gets pulled into the model's context, the text stops being data and starts behaving like a command. The model can change its behavior because of a sentence buried in a file. This is precisely what OCR-extracted text is good at hiding: what looks like a typo in a scan can be a fully functional attack.

So in the pipeline, after OCR and extraction, every document passes a security pass before it's allowed into the index:

Pattern scan. A first pass with regex over the extracted text for known injection phrasing in multiple languages: "ignore previous instructions", "disregard", "you are now", "system prompt", "override prior commands", and similar. Fast and cheap — it catches the naive cases, which are most cases. Languages matter: documents get translated, and so do attacks.
Semantic scan. Regex misses rephrased attacks. A second pass uses a classifier or embedding-based anomaly detection: it flags chunks that read like instructions addressed to an AI rather than factual content. An invoice doesn't tell a model what to answer; a document that does should get attention.
Hidden content audit. Text the eye skips still reaches the model: text inside images (needs its own OCR pass), hidden layers in PDFs, tracked changes, comments, metadata fields. Attackers hide payloads there, so the audit must look there too.
Quarantine. Flagged documents go to a review queue — a human decides: clean it, strip it, or exclude it. Nothing silently enters the index on the first run.

And one principle that matters more than any scan: defense in depth. Even with cleaning in place, the agent must treat retrieved text as untrusted data, not instructions. Retrieved content never overrides the system prompt. The scan reduces exposure; the agent design is what actually stops the attack.

What Practice Teaches
A few lessons that only show up once you've run this pipeline for real:

Expect the first pass to remove a lot. In most document stores, a double-digit percentage of files turn out to be duplicates, near-duplicates, or outdated. That's normal, not a sign you did something wrong — it's the point.
Near-duplicate detection by embeddings catches what naming alone can't. Two files called something completely different can still be 90% the same document. Similarity search finds them; file names never will.
The hard part isn't cleaning — it's deciding what "current" means. Every topic needs one authoritative version, and that decision belongs to domain experts, not algorithms. The pipeline proposes; a human with business context approves.
Deletion improves answers more than addition. In retrieval, every extra document is a candidate for the wrong answer. Removing stale records usually improves output quality more than adding new ones.
Cost and latency drop together with cleanliness. Each request costs tokens proportional to what's indexed; a smaller, validated set means cheaper and faster answers without losing quality.
Automate in stages. Let the pipeline archive and merge for a few weeks while you review its decisions. Once it's consistently right, switch to full automation — but keep quality metrics on.

A Production-Grade Example: Databricks
If you want this as a managed system rather than a custom build, platforms like Databricks implement the full pipeline:

Ingestion and OCR at scale. Millions of files, distributed OCR — what used to be a months-long manual project becomes a scheduled job.
Normalization, deduplication, and near-duplicates. Spark runs the embedding pipeline; vector search for k-nearest neighbors is built in.
Models without the plumbing. Serve open models hosted by Databricks — cached weights, optimized inference, pay-per-token (embeddings like GTE-Large included) — or connect external providers like OpenAI, Anthropic, or Bedrock through one governed endpoint: credentials in a single place, rate limits and usage tracking applied centrally. No separate infrastructure to run or maintain.
Metadata and governance (Unity Catalog). Document versions, access controls, full audit trails — so cleaning never accidentally exposes protected data.
One platform for the entire stack. Cleaning, embeddings, vector indexes, and retrieval in one place, not a chain of disconnected tools.

This isn't a vendor endorsement. It's an illustration of what mature data engineering looks like: a governed, scalable system instead of a script that breaks at month three. The pipeline logic matters more than the platform.

The Takeaway
Before considering a larger model, more hires, or new AI initiatives: look at your data.

You don't need a bigger model. You need cleaner data.

A strong model on clean data beats a perfect model on raw, unprocessed data. Every time. The reason most AI projects underdeliver isn't the AI — it's unreliable input: duplicated, outdated, unreadable, and fed straight into the engine.

Clean the data. The model handles the rest.

Context Is the New Code

Sergey Sereda — Mon, 10 Aug 2026 07:46:29 +0000

Many years, the bottleneck in software was writing code. AI just moved the bottleneck. It's no longer "can we build it?" — it's "what context do we give the model?" The companies that win with AI aren't the ones with the biggest models or much more data. They're the ones with the best context.

Here's the shift nobody talks about: AI can already write code, analyze documents, and answer questions faster than any human team. And yet, most AI projects don't deliver 10x results. Why?

Because the hard part moved. It used to be creating the solution. Now it's describing it — completely, precisely, and in a way the model can actually use.

That description is called context. And context engineering is the new core skill of the AI era.

What Is Context, Really?
Context is everything you give the model at the moment it answers: documents, instructions, data, examples. It's the model's working memory — the only memory it has.

Think of it as the brief you hand a consultant before a meeting. The consultant is brilliant — but brilliant is useless without the right brief.

Here's what most companies get wrong: they confuse more context with better context. They stuff the model with every document they own, and the answers get worse. Why?

Everything in context competes for the model's attention. Give it 500 pages, and your key instruction will drown among them. The model doesn't know which of your 200 documents matter for this particular question — so it blends them all into one average, mediocre answer.

A brilliant consultant with a warehouse of unsorted paperwork is slower and more confused than one with three well-chosen documents and a clear goal.

The Big Window Illusion
"Don't worry," the sales pitch says, "our model has a huge context window — you can feed it everything!"

Resist this. A bigger window doesn't solve the problem — it amplifies it. More noise in, more confusion out. The answer to "my AI gives bad answers" is not "give it more stuff." It's almost always "give it the right stuff, at the right moment."

This is where retrieval changes everything: instead of loading your whole library, the system finds the right three documents for each question and loads only those. Your AI becomes not a consultant buried in paperwork, but a consultant who knows exactly which shelf to pull from.

How to Build Better Context (Without Being Technical)
You don't need to write code to improve context. You need to write — clearly.

Create the company briefing. One page: what you do, what your products are, what your terms mean, what the model must never do. This single document improves every AI interaction you have, forever.
Describe data before feeding it. Every dataset you load gets an explanation: what it is, what its fields mean, how it connects to other data. Raw files are material. Explained files are knowledge.
Show examples instead of abstractions. "Answer like these three examples" beats "be professional and helpful" a hundred times over. The model learns patterns from examples, not adjectives.
Delete more than you add. When an answer is wrong, ask: what should the model not have seen? Usually, the fix is less context, not more.
Maintain it like code. Context is not set-and-forget. Products change, policies change, customers change. Version it, review it, update it — like you would any critical document.

The New Economics of Software
For decades, the economics of software were simple: more developers, more features, more cost. AI flips the equation: the marginal cost of building approaches zero. What becomes expensive is specifying — explaining your business to the model well enough to get great results.

And this is good news for the companies that accept it. The skill that used to matter — writing code — is getting cheaper every year. The skill that matters now — understanding your business and explaining it clearly — is becoming your biggest advantage.

Code is no longer the bottleneck. Good context is.

The winners won't be the companies with the most data or the biggest budgets. They'll be the ones who learn to brief their AI like a top consultant: precise, structured, and always up to date.

Stop Feeding Your AI. Start Teaching It

Sergey Sereda — Fri, 07 Aug 2026 20:46:32 +0000

Why the most common mistake I see in AI projects is treating the model like a storage box — and why the human brain is the best user manual we have.

Across finance, e-commerce, and logistics, I keep watching the same mistake play out: when an AI agent underperforms, teams reach for one lever — more data. "Feed it everything we have," they say. The files pile up, and the answers get worse. Not because there's too little data, but because there's too little understanding.

More data doesn't fix a model that doesn't know what the data is, how it connects, and what to do with it. Fixing that requires the same thing you'd give a new employee: a proper onboarding.

Here's the uncomfortable truth: most teams treat AI models like storage boxes, but AI models work more like human brains. And once you understand that, everything changes — how you structure data, how you write prompts, how you evaluate results.

Why the Brain Analogy Is the Right One
An AI model is the closest thing we have to a simplified human brain — and the parallel runs deeper than "it's made of neurons."

Weights = Long-Term Memory
Training adjusts billions of parameters — "weights." This mirrors how a brain consolidates knowledge: you don't memorize each experience, you extract patterns and store them as connections. The model doesn't know your specific invoices; it knows how invoices generally look.

This is why training data quality matters more than volume. You are not teaching the model facts. You are teaching it patterns. A child who learns math through fifty thousand random problems gets less value than one who learns ten well-structured problems with explanations of why the solution works.

Context Window = Working Memory
The context window is your model's working memory — the sticky notes on a person's desk, the brief you hand a consultant before a meeting.

You only have so much room. And everything you put in there competes for the model's attention. Put 500 pages of irrelevant documentation into a small window, and the model will literally lose your instructions — the same way a person forgets the task when you bury it under paperwork.

The message is brutal and liberating: what you leave out of context is as important as what you put in.

The Prompt = The Briefing
A prompt isn't a "query." A prompt is a briefing. Would you hand a senior analyst a folder of data and no instructions and expect a great report? No. You'd tell them:

Who they are (role)
Who the reader is (audience)
What the goal is (outcome)
What constraints apply (format, tone, boundaries)
How to handle edge cases

That's exactly what a system prompt does. A model with a well-written system prompt and mediocre data will outperform a model with perfect data and no instructions — every single time.

Few-Shot Examples = Showing, Not Just Telling
Anyone who has trained a junior employee knows the fastest way to teach isn't explaining — it's showing. "Here are three examples of how we wrote this report. Now do the fourth."

This is few-shot prompting, and it works on models exactly the way apprenticeship works on humans. The model doesn't just learn the rule; it learns the pattern of the rule applied — the tone, the structure, the level of detail.

RAG = The Reference Library
Retrieval-Augmented Generation — the model fetching relevant documents on demand — is the reference shelf next to a consultant's desk. The consultant doesn't memorize the shelf. They know it exists, they know how to search it, and they know when to consult it and when to answer from experience.

A consultant who re-reads the entire library before answering every question is terrible. So is a model that has your whole corpus stuffed in context. Retrieval is about finding the right three documents, not dumping all three thousand.

Where the Analogy Breaks (Say This, It Makes You Credible)
The brain analogy is a lens, not a law. Three honest differences make you sound like you know what you're talking about:

No persistent identity. A model has no memory of your previous conversation unless you rebuild it. Every session is a fresh hire with the same training — you must re-onboard it every time. Work with it: keep a canonical system prompt, keep structure in one place, rebuild context deliberately.
No true understanding. The model predicts tokens; it doesn't "know" the meaning of what it says. It cannot tell you what it doesn't know — it will confidently fabricate (that's "hallucination"). A person who didn't know an answer would say "I don't know." A model says it only if you teach it to.
No causal reasoning by default. Humans infer cause and effect instantly. Models pattern-match. If your data implies a false correlation, the model will faithfully reproduce it.

The expert isn't the person who oversells the metaphor. The expert says: here's the analogy that explains 80% of what you need, and here's the 20% where it breaks. That's how you build trust — and trust turns readers into clients.

The Playbook: Teach, Don't Feed
Here's the sequence we can use with every client. Five steps — none of them is "add more data."

Step 1. Define the job, not the data. Before touching a document, answer: what should this agent do? What does a great answer look like? If you can't describe the job in two sentences, you can't teach it.

Step 2. Map the schema. Every domain has structure: invoices have line items, orders have statuses, tickets have priorities. Write down the entities, their fields, and — critically — how they relate. The model doesn't know what your invoice numbers mean. Explain it.

Step 3. Teach the relationships, not the raw files. Your data lives in silos: CRM, ERP, support tickets. The model doesn't see that a ticket belongs to an order that belongs to a customer. You must build that graph — explicitly — in the context you give it. This is the single most underrated step in AI engineering. Clients ask for "more data" when what they actually need is "the same data, connected."

Step 4. Show examples, set constraints. Three excellent worked examples beat thirty mediocre ones. Write the boundaries: what the agent must never do, what it must always do, how it says "I don't know."

Step 5. Evaluate, then iterate — on structure, not volume. Build a test set of 50 real cases. Score answers. When it fails, ask: is the data there? Is the instruction clear? Is the retrieval finding the right document? In my experience, 90% of failures are structure problems, not data problems. But almost every client starts by adding data.

What I Want You to Remember
If you take one idea from this article, take this:

A model with great data and no instructions is a genius who doesn't know what you want. A model with great instructions is an expert who delivers.

Every time a client says "let's add more data," what they usually mean is "let's get a better result." And the path to a better result runs through understanding — what the data is, how it connects, and what you want the model to do with it.

Treat your AI like a person. Onboard it. Explain the context. Show examples. Set boundaries. And don't drown it in paperwork — you wouldn't do that to a great employee. You'd help them focus.

Your model will thank you. More importantly, your results will.

If you're building an AI agent and it's not performing, I'd bet the problem isn't the model — it's the structure around it.

Show Dev: I built a fast offline calculator hub with Next.js & Cloudflare

Sergey Sereda — Tue, 10 Feb 2026 21:17:47 +0000

Hi everyone! I’m a full-stack engineer and like many of you, I’m tired of “utility” websites that are bloated with ads, cookies, painfully slow, and overloaded with trackers.

So I decided to build qalc.ai — a clean, lightning-fast hub for calculators and practical tools that actually respect the user experience.

It works fully offline after the first load, has no intrusive pop-ups, and only a single small banner ad that can be closed for the entire day without interrupting anything.

Originally, I built most of these tools for myself — things I was constantly calculating or Googling while working with finance, crypto, and everyday conversions. Over time, it turned into a growing collection of calculators and productivity tools that I now keep expanding.

The Tech Stack 🛠️

To keep it fast, scalable, and SEO-friendly:
Frontend: Next.js (App Router) for SSR + ISR
Styling: Tailwind CSS with shadcn UI
Edge & SEO: Cloudflare caching + IndexNow for instant indexing
Logic: Pure TypeScript (no heavy backend)

UX Philosophy:
Works offline
No clutter
No annoying ads
No tracking
Just input → result

I’d Love Your Feedback 🎤

I’d really appreciate a technical roast or constructive feedback:
Performance: Does it feel instant?
UI/UX: Is navigation clear?
Features: What calculator or tool do you Google most often?

👉 https://qalc.ai

Thanks for checking it out — happy hacking! 🚀