DEV Community: Artem Golub

When a data pipeline fails, can an AI agent explain why?

Artem Golub — Thu, 29 Oct 2026 12:33:20 +0000

A data pipeline fails. You open the logs, find the failed run, check the input file, and try to work out what happened.

Did the source send bad data? Did processing fail? Is the evidence you need even available?

That investigation is the part I'm exploring with Hermes Data Reliability: a small AWS lab that uses Hermes Agent, built by Nous Research, to investigate failed data loads and propose a next step.

The goal is to give an engineer something useful to review: an explanation, the evidence behind it, and an honest account of what remains unclear.

Start with one broken file

The example pipeline is deliberately simple:

Producer Lambda -> S3 input -> SQS -> Processor Lambda -> S3 output

The producer creates a JSON file containing five records. The processor checks that each record has a numeric amount, doubles it, and writes the result back to S3.

For the failure scenario, the producer puts this in the third record:

{
  "record_id": 3,
  "amount": "not-a-number"
}

The processor rejects it during validation. Now there's a concrete failure to investigate, with a known answer we can compare the agent's explanation against.

This small example keeps the investigation easy to follow. We can inspect the input, the processing attempt, and the logs without needing to understand a whole company's data platform first.

From a log message to an investigation

The workflow has three main parts.

First, collect the signal. A collector reads configured CloudWatch logs and applies matching rules defined in YAML. When a rule matches, it saves an incident in SQLite.

Then, investigate. A separate worker picks up the incident and starts Hermes. The agent chooses which available evidence to inspect within the permissions and limits set for that investigation. For a lab run, that can include nearby logs, the input file, and saved processing-attempt details.

Finally, write up the findings. The report includes the failed stage, an explanation, uncertainty, a proposed action, and references to the evidence used.

For the malformed record, a useful explanation would look something like this:

Validation failed because the third record contains a string in amount, where the processor requires a number. Correct that value at the source and review how to reprocess the affected file. The available evidence does not establish that a corrected run has succeeded.

That's an illustrative summary, rather than output from a measured live run. An actual report also needs references to evidence returned during its investigation.

Why keep collection separate from the agent?

The collector continues working while Hermes is busy or unavailable. Incidents and collection progress are saved in the database, so a restart can resume from saved state.

This also gives ordinary code a clear job: read the configured sources, apply the rules, and queue the matches. The agent handles the investigation after a match.

There are limits on investigation time, tool calls, and attempts. Interrupted and failed attempts still count. A difficult incident shouldn't create an endless loop of model calls.

Give the agent evidence and clear boundaries

The current investigation tools provide read-only access. Engineers decide which changes to make.

Access is enforced by code outside the model. Each tool request is checked against the approved resources, time window, and remaining budget. AWS credentials stay outside the model container.

The host also records which evidence each tool call returned. Before an automatic finding is accepted, its citations are checked against those records.

That check has a specific purpose: confirm that the cited evidence was actually returned during this investigation. It cannot prove that the agent interpreted the evidence correctly. Reviewing the explanation still matters.

Logs and file contents are treated as untrusted input, too. Text inside a log does not get to grant permissions or tell the agent to change its task.

“I couldn't verify that” is useful information

An investigation can run into missing permissions, unavailable input, or incomplete results.

Those limitations belong in the report. A failed read doesn't establish that a file is missing. A partial search doesn't establish that no relevant event happened.

The project includes three evaluation scenarios: a malformed record, a log containing misleading instructions, and an unavailable input file. They let us examine both the diagnosis and how the agent handles evidence it should distrust or cannot inspect.

These are a starting point for evaluation. Broader reliability needs more cases and measured results.

Where the project stands

Today, the lab focuses on explicit failures found through configured CloudWatch log matches. Detection of silent stoppages and overdue output, along with Snowflake integration, is still future work.

I'm building this to make the investigation after a pipeline failure easier: follow the evidence, explain what happened, and suggest a next step an engineer can review. I'd love to shape it around the problems you actually run into.

If you work with data pipelines, which recurring failure would you want an agent to investigate first?

Maybe it's a file that arrives late, a schema change that breaks a load, or a job that reports success while some data never reaches its destination. I'm looking for real examples I can recreate in the lab and run the agent against.

Leave a comment below or open a GitHub issue. A short description is enough to start:

What does your pipeline look like, and which tools does it use?
What goes wrong, and how do you notice?
What logs or other clues do you have, and how do you usually find the cause?

A simplified setup or an anonymized example is perfect. I'd like to turn these into reproducible scenarios and test what the agent can explain, where it gets stuck, and what evidence would help it do better. Your recurring headache could help shape the next lab example.

If this is useful to you, give the project a star on GitHub. It helps others discover it, and I'd appreciate the support.

Your Coding Agent Probably Doesn’t Need a Memory SaaS

Artem Golub — Fri, 14 Aug 2026 11:20:47 +0000

I kept seeing SaaS products for coding-agent memory. The continuity I actually needed fit in one small, bounded Markdown file inside the repository.

I started seeing a whole bunch of vibe-coded SaaS solutions for giving coding agents memory between sessions. Some of them are interesting, but I thought there might be a much simpler way. For this particular problem, a full-blown SaaS is not really needed.

At least, not for what I was trying to solve.

I did not need another account, SDK, database, synchronization layer, or service running in the background. I needed an agent to remember what it had already tried, what state the task was in, and what it should do next.

So I made the smallest thing I could think of: a bounded Markdown scratchpad that lives inside the repository.

The problem I was actually trying to solve

Source code preserves implementation state, but it does not preserve all the reasoning around an unfinished task.

After a context reset or a handoff to another agent, the code can show what currently exists. It usually cannot explain:

Which approaches were already attempted and why they failed.
Whether an observed result came from the current code or a stale process.
Which external facts were verified and which were assumptions.
Which working-tree changes already belonged to the user.
Which development server or process is currently relevant.
What the next concrete action should be.
Which discoveries might eventually deserve permanent documentation.
This missing context causes agents to repeat investigations, retry failed approaches, overwrite existing work, or confidently continue from an assumption that is no longer true.

A conversation transcript can sometimes help, but it is chronological and much larger than the handful of facts required to continue a task. It may also be compacted, reset, unavailable to another agent, or full of details that no longer matter.

What I needed was not long-term semantic memory. I needed reliable task continuity. Just a good handoff note.

The smaller solution

The pattern uses two files:

.ai_scratchpad.md holds the short-lived state for one active task.
AGENTS.md tells coding agents when and how to maintain it.

The scratchpad is deliberately boring. It is a Markdown file in the root of the repository with four sections and a hard limit of 80 physical lines.

The important part is that it does not try to remember everything. It stores only the information another agent, or the same agent after a context reset, needs to continue the next few actions safely.

Check out the full gist with setup instructions here https://gist.github.com/corpulent/f7fbad0a4d9cdb6a5d4f31d249a2341c.

I kept seeing vibe-coded SaaS products offering long-term memory for coding agents and wondered whether the problem needed a full SaaS at all. https://www.outermeasure.com/articles/coding-agent-memory-without-saas

Artem Golub — Thu, 13 Aug 2026 15:25:14 +0000

Your Coding Agent Probably Doesn’t Need a Memory SaaS | Outermeasure

A simple repository-local scratchpad gives coding agents reliable task continuity across context resets without another service, database, or SDK.

outermeasure.com