DEV Community: Glendel Joubert Fyne Acosta

Evidence Beats Claims: Why AI Agents Need Runtime Proof

Glendel Joubert Fyne Acosta — Tue, 26 May 2026 01:49:17 +0000

An AI agent saying "I did it" is not proof that anything happened.

"I sent the email."

"I updated the database."

"I escalated the issue."

"I published the post."

Those are claims.

In a real production system, claims are not enough.

If an AI Agent performs work that affects users, data, money, operations, or another system, the runtime must be able to prove what actually happened.

The Problem

Language models are very good at producing confident completion statements.

That confidence can be useful in conversation, but dangerous in infrastructure.

A model may say:

"Done, I sent the email."

But what actually happened ?

Maybe the email tool succeeded.

Maybe the permission check failed.

Maybe the API timed out.

Maybe the retry limit was reached.

Maybe the tool was never called.

Maybe the model only assumed the action happened because that was the most natural response in the conversation.

This is one of the most important differences between a demo and a production AI system.

In a demo, the agent saying "done" feels impressive.

In production, "done" needs evidence.

Model Claims vs Runtime Evidence

A model claim is what the AI says happened.

Runtime evidence is what the system can prove happened.

Those are not the same thing.

A serious AI Agent system should separate them clearly.

For example:

const response = await agent.run("Send the customer follow-up email");

// This is only a model-generated claim
console.log(response.message);
// "Done, I sent the email."

That message is not enough.

A production system should also have a runtime record:

{
  "actor": "support-agent-01",
  "tool": "send_email",
  "permission": "granted",
  "input": {
    "to": "customer@example.com",
    "template": "follow_up"
  },
  "status": "success",
  "providerMessageId": "msg_abc123",
  "timestamp": "2026-05-25T14:32:10Z",
  "auditId": "audit_789"
}

Now the system can answer:

who requested the action
which tool executed
whether permission was granted
what input was used
what result came back
when it happened
what audit record proves it

That is the difference between trusting text and trusting infrastructure.

Why This Matters

AI agents are moving from chat interfaces into real workflows.

They are not just answering questions anymore.

They are:

sending messages
creating tickets
updating records
calling APIs
reading customer data
triggering workflows
escalating incidents
generating reports

Once agents do real work, organizations need more than fluent responses.

They need accountability.

If an agent says it updated a record, the system must prove the record was updated.

If an agent says it escalated a complaint, the system must prove the escalation happened.

If an agent says it sent a message, the system must prove the message was sent.

Otherwise, the organization is not operating on evidence.

It is operating on model confidence.

The Dangerous Failure Mode

The dangerous failure mode is not always a loud crash.

Sometimes the agent simply says:

"Done."

And everyone believes it.

But behind the scenes:

the tool failed
the permission was denied
the payload was invalid
the API returned an error
the action was never executed
the workflow stopped halfway

This creates a false sense of completion.

The user thinks the task is finished.

The agent thinks the task is finished.

The organization acts as if the task is finished.

But the runtime has no proof that the task ever happened.

That is a serious reliability problem.

Multi-Agent Systems Make This Worse

This problem becomes even more dangerous in Multi-Agent Systems (MAS).

Imagine this flow:

Agent A says it collected the customer data.
Agent B uses that claim to draft a response.
Agent C sends the response.
Agent D summarizes the case as resolved.

If Agent A's claim was unsupported, the entire chain becomes unreliable.

One unsupported claim becomes another agent's input.

The error propagates across the system.

By the end, the final result may look coherent, but the foundation is wrong.

This is why Multi-Agent Systems need runtime evidence at every important boundary.

Agents should not pass around unsupported claims as if they were facts.

They should pass around claims connected to evidence.

The Architecture Pattern

A better architecture separates three things:

Reasoning
Execution
Evidence

The AI agent reasons about what should happen.

The runtime executes the action if it is allowed.

The system records evidence of what actually happened.

const request = await agent.decideNextAction(context);

const permission = runtime.permissions.verify(request);

if (!permission.allowed) {
  return runtime.recordDeniedAction(request, permission.reason);
}

const result = await runtime.execute(request);

const evidence = await runtime.recordEvidence({
  request,
  permission,
  result
});

return agent.summarizeResult({
  result,
  evidenceId: evidence.id
});

The model can still explain the result to the user.

But the explanation is now grounded in runtime evidence.

The agent is no longer saying:

"Trust me."

It is saying:

"Here is what happened, and here is the evidence."

What Runtime Evidence Should Include

At minimum, an evidence record should capture:

actor identity
requested action
permission result
tool or workflow used
input payload
execution result
timestamps
failure reason, if any
retry attempts
audit/reference ID

For sensitive systems, it may also include:

approval record
policy version
resource identifier
provider response metadata
verification result
human review state

The goal is not to create bureaucracy.

The goal is to make AI work inspectable, debuggable, and trustworthy.

The Rule

A simple rule for production AI systems:

If the agent claims an external action happened, the runtime should have evidence.

No evidence means the claim is unsupported.

Not necessarily false.

But unsupported.

That distinction matters.

An unsupported claim should not be treated as completed work.

It should trigger one of three outcomes:

retry
verify
escalate

That is how AI Systems become operationally reliable.

From Chatbots To Organizational AI Systems

Chatbots can get away with claims.

Organizational AI Systems cannot.

When AI agents operate inside real organizations, they need:

permissions
execution boundaries
audit trails
verification gates
runtime evidence
human escalation paths

The more responsibility we give agents, the more important evidence becomes.

A confident answer is not enough.

A fluent summary is not enough.

A completed-looking workflow is not enough.

The system must be able to prove what happened.

Conclusion

AI Agents should reason.

Runtimes should execute.

Evidence should prove.

That separation is what turns agent behavior from conversation into infrastructure.

If we want AI Agents to operate inside real organizations, we need to stop treating model-generated claims as proof of completed work.

Evidence beats claims.

AI Agents Don't Have Permissions — Runtimes Do

Glendel Joubert Fyne Acosta — Thu, 21 May 2026 00:45:50 +0000

Right now, many Multi-Agent Systems are implementing permissions inside prompts.

"You may access the CRM."

"You are allowed to send emails."

"Do not modify billing records."

This is becoming one of the biggest architectural mistakes in modern AI systems.

A prompt is not a security boundary.

Language models are probabilistic reasoning engines. They are excellent at planning, summarizing, reasoning, and interpreting context. But they are not deterministic authorization systems.

If your application's security model depends on the LLM consistently obeying natural-language instructions, your system does not actually have runtime governance.

It has probabilistic behavior shaping.

The Problem

I keep seeing architectures where the agent itself is expected to decide whether an action is allowed:

const prompt = `
You are an AI Agent.

The user wants to delete a customer record.
The user's permissions are: ${permissions}.

Should you allow this action?
`;

const decision = await llm.generate(prompt);

This looks flexible.

It also creates several major problems immediately:

prompts can conflict
context windows drift
instructions can be overridden
reasoning can hallucinate
behavior changes across models
authorization becomes non-auditable

And once you move into multi-agent systems, the situation becomes even worse.

One agent may interpret permissions differently from another. Handoffs may lose constraints. Context summarization may remove critical security instructions entirely.

Now your governance model depends on whether probabilistic agents correctly preserve natural-language policy across multiple reasoning steps.

That is not enterprise architecture.

The Runtime Must Enforce Boundaries

The AI should reason about what needs to happen.

The runtime should determine whether it is allowed to happen.

This distinction is critical.

A governed architecture should look more like this:

if (!runtime.permissions.verify({
  agent: agentId,
  action: "delete_customer",
  resource: customerId
})) {
  throw new UnauthorizedError();
}

const result = await executor.deleteCustomer(customerId);

The LLM may request the action.

The deterministic runtime decides whether execution is permitted.

That is a real security boundary.

The Cognitive Layer vs The Deterministic Layer

I think a lot of confusion in the current AI ecosystem comes from mixing these two responsibilities together.

The Cognitive Layer:

reasoning
planning
interpretation
summarization
decision support

The Deterministic Layer:

permissions
schema validation
execution
workflows
retries
state transitions
audit logs
policy enforcement

The AI should not govern itself.

The framework must govern the AI.

Why This Matters More In Multi-Agent Systems

Single-agent systems are already difficult to debug.

Multi-agent systems amplify the problem dramatically:

context drift compounds
handoff failures appear
responsibilities blur
state becomes harder to trace
authorization assumptions leak between agents

Without deterministic runtime enforcement, governance becomes almost impossible to reason about operationally.

And when systems fail, the incident report becomes:

"The model ignored the instruction."

No serious infrastructure team will accept that as a security architecture.

Organizational AI Systems Need Runtime Authority

As AI systems move into real organizations, governance stops being optional.

Enterprises need:

auditability
traceability
deterministic enforcement
runtime evidence
policy validation
observability

Natural-language instructions alone cannot provide these guarantees.

The future of Organizational AI Systems will depend on separating:

probabilistic reasoning from
deterministic governance.

AI Agents should reason.

Runtimes should govern.

The AI FOMO Trap: Why your Multi-Agent System is brittle (and how to fix it)

Glendel Joubert Fyne Acosta — Thu, 14 May 2026 00:45:30 +0000

A developer on Reddit recently told me: "Companies right now are risking the LLM-led parts of their architecture due to FOMO. We'll see how far they get".

He is absolutely right. Fear Of Missing Out is driving engineering teams to ship "Autonomous Agents" at breakneck speed. But in the rush to production, we are abandoning 20 years of established software engineering principles.

We are letting probabilistic models control deterministic runtimes.

If you are routing network traffic, validating data schemas, or checking user permissions using an LLM prompt, you are not building a resilient system. You are building a fragile prompt-chain wrapped in hope. When it fails (and it will), it will be slow, expensive, and completely un-auditable. InfoSec won't accept "the model hallucinated the auth check" as a valid incident report.

The Cure: The Manager-Executor Pattern

To build enterprise-grade Multi-Agent Systems, we must separate the Cognitive from the Deterministic.

1. The Manager (Probabilistic) This is the LLM. Its only job is to reason, plan, and analyze context. It decides what needs to be done. It does not execute code. It does not manage its own memory. It requests actions via strict JSON schemas.
2. The Executor (Deterministic) This is your runtime framework. It acts as the boundary. When the Manager requests an action, the Executor:

Verifies the agent's permissions.
Validates the payload against a strict schema.
Checks the token/cost budget.
Executes the code (API call, DB write).
Returns the exact result to the Manager.

The Framework Controls the AI

The fundamental shift required in MAS architecture is understanding that the framework must control the LLM; the LLM must never control the framework.

Right now, developers are having to build these custom state machines and validation layers from scratch because popular frameworks default to LLM-routing. It's time we standardize this. We need "A Real Framework" for Multi-Agent Systems—a framework that enforces the Manager-Executor pattern by default.

Stop relying on vibes-based engineering. Let's get back to rigorous software architecture.

The Token Waste Problem: Why your AI Agents shouldn't evaluate permissions

Glendel Joubert Fyne Acosta — Sat, 09 May 2026 00:47:02 +0000

We are burning millions of API tokens on problems that if statements solved 20 years ago.

I speak with developers building Multi-Agent Systems (MAS) every day, and I keep seeing the same massive architectural anti-pattern: Routing everything through the AI model.

Need to check an agent's permissions? "Ask the LLM."
Need to route a message? "Ask the LLM."
Need to validate a data schema? "Ask the LLM."

Language models are extraordinary reasoning engines. But they are also expensive, probabilistic, and relatively slow. If a problem has a deterministic, correct answer (like checking an access policy), it should be evaluated by runtime code, not guessed by a neural network.

The Anti-Pattern

Instead of doing this (Probabilistic):

// BAD: Asking the LLM to check permissions
const prompt = `You are an agent. The user wants to delete a file. 
Here are their permissions: ${user.permissions}. 
Should you allow it?`;

const decision = await llm.generate(prompt);

The Solution

We need to get back to doing this (Deterministic):

// GOOD: Let code handle policy, let AI handle reasoning
if (!user.hasPermission('delete_file')) {
  throw new Error("Unauthorized"); 
}

// Only call the LLM for actual cognitive tasks
const plan = await agent.reasonAboutFile(file);

AI should decide what to do. Deterministic code should execute it and enforce the boundaries.

Are we forgetting basic software engineering principles just because AI is exciting? The MAS space doesn't need more wrappers; we need standardized frameworks that enforce these boundaries. Let's get back to building solid infrastructure.