DEV Community: Ricardo Feldhaus Krisanoski

What I log from AI workflows before I trust the automation

Ricardo Feldhaus Krisanoski — Sun, 16 Aug 2026 18:58:02 +0000

Most AI workflow posts focus on prompts, tools or output quality.

Those matter, but they are not the first thing I want to trust.

The first thing I want to trust is the audit trail.

If an AI workflow suggests a commercial follow-up, changes a queue, routes a conversation or asks for human review, I want to know:

what surface executed the decision;
what system would be affected;
whether human approval was required;
what context or evidence shaped the suggestion;
what it likely cost;
what happened after a human reviewed it.

That is why I started writing an observability contract for AI workflows before expanding the automation surface.

The Real Problem

In operational systems, the hardest bug is often not "the model answered badly."

It is "nobody can explain the path from context to action."

That creates a few predictable failures:

follow-up suggestions with no clean handoff reason;
invisible cost growth;
logs spread across workflow history, scripts and chat tools;
human edits that never become quality signals;
no reliable way to compare one workflow against another.

What I Log

I now prefer a small contract that every AI-assisted workflow can emit, whether it runs inside n8n or through a wrapper around an agent script.

At minimum, I want:

a stable workflow name;
a trace or execution ID;
the execution surface, like n8n or an agent wrapper;
the business unit and target system;
a risk level;
a flag for human approval;
the decision itself;
the outcome;
the model and estimated cost when applicable.

Then I add the fields that make review actually useful:

prompt or skill version;
retrieved sources or evidence references;
handoff reason;
human review outcome;
latency and token estimates.

That structure is much more useful than a raw dump of prompts and responses.

Why This Matters For Commercial Workflows

Imagine a workflow that suggests a commercial follow-up.

If the trace only says "AI replied" or "task created", the review loop is weak.

I want the system to answer higher-value questions:

Was this a suggestion or an action?
Which system would have been touched next?
Was approval required before any outbound step?
If it went to human review, why?
If the human edited it, was the edit approved, rejected or escalated?

That is the difference between using AI as a black box and using it as an operational component.

The Bridge Pattern

One practical detail mattered a lot in my implementation work:

I did not want workflow JSON carrying observability secrets everywhere.

So instead of wiring keys directly into each workflow, I prefer a small internal bridge:

workflow -> trace payload -> internal bridge -> observability backend

That keeps the trace contract shared while reducing secret sprawl.

It also makes it easier to instrument different surfaces with the same schema.

Human Review Is Part Of Observability

The most useful signal is often not the first model decision.

It is what happened after a person saw it.

Did they approve it?
Did they edit it?
Did they reject it?
Did they mark it as needing review?

Those outcomes turn observability into the start of an eval loop instead of a passive log archive.

Public-Safe By Design

For operational AI, public proof needs constraints.

So the public version of this work avoids:

raw customer messages;
phone numbers or candidate data;
private workflow URLs;
webhook paths;
internal secrets or auth headers;
unsupported metric claims.

Where a number is not approved, I keep it as metrics to collect.

The Practical Lesson

Before expanding an AI workflow, I want to make its decisions inspectable.

Not just its prompt.
Not just its output.
Its actual operational path.

That is why I now treat observability as part of workflow design:

trace the decision;
trace the risk;
trace the human review;
trace the cost;
keep the action path explainable.

Public case study:

https://github.com/rkrisa/portfolio-ai-ops/tree/main/cases/ai-workflow-observability-contract

AI Coding Agents Need a Decision Gate Before the Implementation Loop

Ricardo Feldhaus Krisanoski — Thu, 13 Aug 2026 13:00:00 +0000

AI-assisted development has changed the economics of implementation.

A plausible architecture can become a working prototype before the team has finished asking whether it solves the right problem.

That is useful — and dangerous.

The failure mode I kept seeing was not poor execution. It was premature commitment: a request arrived with a tool, repository, architecture, or output format already selected, and the implementation loop started before the decision had been challenged.

That is why I built Decision Path Challenger.

It is an open-source Agent Skill for OpenAI Codex and Claude Code. Its job is to create a decision gate before meaningful implementation work begins.

The solution is often baked into the request

Consider a request such as:

Build a weekly sales dashboard in Notion.

It sounds concrete. It also contains several decisions that may never have been made consciously:

the output must be a dashboard;
the dashboard must be interactive;
the work must live in Notion;
storage and delivery should happen in the same system;
someone will maintain that workspace every week.

The actual outcome may be much simpler:

Deliver a consistent weekly decision brief that managers can review, archive, and forward to the sales team.

Once the outcome is clear, the solution space changes. A collaborative dashboard remains possible, but so do a controlled HTML report, a PDF snapshot, a spreadsheet template, a concise automated briefing, or a hybrid workflow with human approval.

The point is not that Notion is wrong. The point is that the tool should not silently define the outcome.

Four layers before technology

Decision Path Challenger separates a request into four layers.

1. Outcome

What must become true for the user, team, or business?

This should be stated without inheriting the proposed tool. A weekly decision brief is an outcome. A Notion dashboard is a possible mechanism and technology choice.

2. Operating experience

How will the result be consumed, reviewed, shared, maintained, and repeated?

This layer surfaces practical questions that architecture discussions often miss. Does the result need to be editable? Must it be forwarded? Do historical copies matter? Who corrects errors? Is the user opening a workspace, receiving an alert, or approving a fixed artifact?

3. Mechanism

What is the simplest reliable way to produce that operating experience?

The mechanism may be a template, a script, a native feature, a scheduled workflow, a human review step, or a custom service. It should earn its complexity.

4. Technology

Which tools, repositories, models, APIs, and frameworks implement the chosen mechanism?

Technology comes last because it is easier to evaluate once the outcome and operating model are explicit.

Why exactly three questions

An effective decision gate cannot become another long intake form.

The skill asks exactly three high-value questions. Together they resolve three categories:

Success and consumption — what the result must enable and how people will use it.
Operating constraints — frequency, maintenance, acceptable manual work, control, cost, and reliability.
Hard boundaries and hidden flexibility — what is truly mandatory and what is merely inherited from the first proposed solution.

Three is a deliberate constraint. One question rarely exposes enough. Ten questions create friction and encourage generic answers. Three forces the skill to select the decisions that materially change the path.

If the answers already exist in the current context, the skill does not ask them again. It states the inferred answers and continues.

Compare solution classes, not cosmetic alternatives

Tool comparison often means evaluating three products from the same category. That is useful only after the category itself has been justified.

Decision Path Challenger compares materially different solution classes:

Native or no-code: built-in features, templates, exports, or scheduled functions in tools already in use.
Lightweight composition: HTML, spreadsheets, scripts, low-code workflows, serverless functions, or simple API orchestration.
Custom product: a bespoke application, agent, service, dashboard, or data pipeline.
Hybrid: native storage combined with custom rendering, or automation combined with human approval.
Process redesign: changing the workflow or output so that less technology is needed.

At least one option must challenge the original framing. Otherwise the comparison only optimizes a decision that has already been made.

Build cost is not total cost

Implementation speed is only one part of the decision.

The skill distinguishes four costs:

Build cost: what it takes to create the first working version.
Run cost: infrastructure, licenses, manual operations, monitoring, and recurring attention.
Change cost: how difficult it is to modify the system when the process evolves.
Failure cost: what happens when the workflow is wrong, unavailable, stale, or trusted too much.

It also considers reversibility. An uncertain workflow should not begin with a hard-to-exit architecture unless there is evidence that the extra commitment is justified.

This is especially important in AI-assisted development. Generating code quickly can make build cost look small while run, change, and failure costs remain hidden.

Possible is not the same as appropriate.

A decisive output

The skill is not designed to end with “it depends.”

It returns one of seven decision labels:

Proceed as proposed
Proceed with simplification
Adapt the proposed path
Use a hybrid
Run a small proof of concept first
Choose another path
Do not implement yet

Every recommendation includes the reason, the most tempting rejected alternative, what was intentionally excluded, and the smallest reversible next step.

For the recurring-report example, the decision may be Proceed with simplification: use a controlled HTML template rendered to PDF, populate it from existing sources, and preserve a human approval step before distribution.

That recommendation is not universally better than a dashboard. It wins only when consistency, portability, archival snapshots, and low weekly maintenance matter more than collaborative editing and interactive exploration.

Cross-agent by design

The canonical behavior lives in a platform-neutral SKILL.md.

The repository includes installation paths for both Codex and Claude Code, Codex-specific interface metadata, examples, evaluation references, a static validator, and a GitHub Actions workflow.

For Codex:

$decision-path-challenger

I found a repository that may help me orchestrate multiple coding agents.
Evaluate the approach before installing or modifying my project.

For Claude Code:

/decision-path-challenger

I want to build a weekly operations dashboard in Notion.
Challenge the implementation path before we start.

The skill does not replace project governance, security review, or human approval. It adds a structured pre-implementation review layer.

What the skill intentionally does not do

Decision Path Challenger does not make every task slower. It should not trigger for a small copy edit, a routine command, an emergency restoration, or a narrowly scoped bug with an already-defined fix.

It also does not treat complexity as a problem by itself. A custom product, agent, or integration can be the right answer. The requirement is that the extra burden is connected to a confirmed operating need rather than to excitement about the technology.

Finally, the skill does not make the decision on hidden assumptions. When a critical preference cannot be discovered from the environment, it asks the user. The goal is a decision-complete path, not autonomous certainty theater.

Try it on a real decision

Clone the repository:

git clone https://github.com/rkrisa/decision-path-challenger.git
cd decision-path-challenger

Install for Codex:

./scripts/install.sh codex

Install for Claude Code:

./scripts/install.sh claude

Or install for both:

./scripts/install.sh all

PowerShell users can run:

./scripts/install.ps1 -Target all

The repository is MIT licensed. Issues and contributions are welcome.

AI agents do not only need more implementation capability.

They also need better judgment about what deserves to be implemented, how the result will operate, and whether a simpler path would work better.

Decision Path Challenger is my attempt to make that judgment explicit, repeatable, and inspectable.

Try it on a real decision. If it changes or confirms your original path, I would like to hear what happened.

Why my first RAG layer starts in Postgres, not in a standalone vector database

Ricardo Feldhaus Krisanoski — Sat, 13 Jun 2026 12:30:37 +0000

When people say they are "adding RAG" to a workflow, the conversation often jumps too quickly to infrastructure choices.

Should this use a vector database?
Should there be a reranker?
Should everything go into a knowledge graph?

Those are valid questions, but they are usually not the first question.

The first question is narrower:

What approved knowledge should the workflow be allowed to retrieve before an AI decision happens?

That is why my first retrieval layer for operational AI workflows starts in Postgres, not in a standalone vector database.

The Workflow Problem

In operations-heavy systems, the model usually should not answer from raw memory or from a giant prompt dump.

The useful context already exists somewhere else:

approved response rules;
handoff criteria;
product or service notes;
source or campaign guidance;
operational decisions that were already made by humans.

The hard part is not generating fluent text.

The hard part is retrieving the right approved context, showing which source influenced the decision and refusing when no safe source exists.

Why Postgres First

For this kind of workflow, most of the surrounding data is already relational:

leads or conversations;
workflow names;
stages and owners;
human review outcomes;
source metadata;
trace logs;
document versions.

So the first technical choice is not "where do vectors live in the abstract?"

It is:

Where can I keep retrieval close to the operational data model?
Where can I log the retrieval path and the final decision together?
Where can I evolve the schema without creating a second system too early?

Postgres plus pgvector is a good first answer to that set of questions.

It lets me keep:

documents and chunks;
metadata such as allowed use and approval requirements;
retrieval traces;
cost estimates;
human review outcomes

in one place.

What The First Version Needs

The first version does not need to be broad.

It needs to be inspectable.

My narrow retrieval scope looks like this:

approved response rules;
product/service notes;
handoff and escalation criteria;
campaign/source guidance;
commercial playbooks.

Each retrieved chunk should carry more than text. It should also carry metadata such as:

source name;
document version;
business domain;
allowed use;
whether human approval is required.

That metadata matters because a workflow may be allowed to use one chunk as internal reasoning support, but not as customer-facing language.

The Eval Mindset

I do not think a retrieval layer is real until it has failure criteria.

So the public prototype includes a small golden-question set:

Does the expected source appear in the top results?
Does the workflow return no-answer or handoff when the source is missing?
Does customer-facing language come only from allowed chunks?
Can I review which chunks influenced the decision later?

That matters more than announcing that the system has embeddings.

Without retrieval checks, a RAG layer can look sophisticated while still pulling the wrong context.

Observability Is Part Of The Design

The retrieval step and the AI decision step should be traceable together.

I want the same review surface to show:

retrieved chunk IDs;
similarity or retrieval score;
model name;
token and cost estimates;
final decision;
handoff reason;
human review outcome.

That is the bridge between "the system answered" and "the system answered for a defensible reason."

When I Would Add More Infrastructure

I am not against standalone vector databases.

I would add one later if the system needed:

more search traffic;
more complex filtering boundaries;
separate deployment requirements;
recall/latency needs that justify the extra moving parts.

But before that point, I prefer a smaller stack that makes retrieval, evaluation and auditability visible.

The Practical Lesson

For AI workflows in revenue or operations contexts, the first retrieval layer should optimize for control, review and schema clarity.

Not for maximum architectural novelty.

That is why my first RAG layer starts in Postgres:

closer to operational data;
easier to trace;
easier to evaluate;
easier to keep human-in-the-loop.

The public case study is here:

https://github.com/rkrisa/portfolio-ai-ops/tree/main/cases/operational-knowledge-retrieval-layer

I added a context resolver before an AI sales agent replies

Ricardo Feldhaus Krisanoski — Thu, 28 May 2026 05:38:41 +0000

Most AI sales agents fail before the model writes a single word.

The failure is not always the prompt. It is usually the context.

In a real chat-commerce workflow, a lead can arrive with several competing signals:

the latest customer message;
CRM stage and owner;
previous conversation history;
campaign or source data;
product/category assumptions;
approved response rules;
handoff and support policies.

If all of that gets dumped into a prompt, the model may produce a fluent answer based on the wrong clue.

That is not an AI problem in the abstract. It is an operating-system problem.

The Problem

I was designing an AI reception agent for a chat-driven commerce operation.

The goal was simple: help the business reply faster and more consistently without losing the commercial context behind each lead.

But the first automated reply had a hidden risk.

A customer might send a short message like "hi" or "I want more information." On its own, that message is weak. The stronger signal may be the campaign, source, CRM stage, product page, previous conversation or approved sales rule.

If the AI agent receives all possible context at once, it still has to decide what matters.

That decision should not be left entirely to generation.

The Design Choice

I added a context-resolution step before the AI response.

Instead of asking the model to inspect every clue and improvise, the workflow first resolves a smaller object:

{
  "source_priority": "campaign_or_crm_or_message_or_fallback",
  "category": "resolved_commercial_category",
  "confidence": "high_or_medium_or_low",
  "selected_directive": "one_approved_response_rule"
}

The important part is not the exact schema. It is the order of decisions.

The system first decides which commercial context is most trustworthy. Only then does the AI agent write the reply.

The Architecture

The workflow looks like this:

Incoming chat lead.
CRM and conversation lookup.
Campaign or source-context lookup.
Context resolver.
Resolved commercial context.
AI response agent.
Structured response and routing decision.
Customer reply or human handoff.

The resolver is intentionally boring.

It is a control layer, not a creativity layer.

It exists to answer questions like:

What is the lead probably asking about?
Which source should win when signals conflict?
Is confidence high enough to answer directly?
Which approved sales rule should be used?
Should the system reply, ask a clarifying question or hand off to a human?

Why This Matters

In business workflows, "more context" is not always better.

More context can mean more ambiguity:

old messages compete with new ones;
generic playbooks compete with campaign-specific offers;
product assumptions compete with what the customer actually asked;
internal rules compete with customer-facing language.

The context resolver reduces that ambiguity before the model responds.

The AI layer becomes easier to debug because every reply can be traced back to a chosen context, confidence level and directive.

Guardrails

The workflow keeps several guardrails around the AI response:

low confidence triggers a consultative fallback;
sensitive commercial cases can be routed to human review;
approved response rules are selected before generation;
the agent receives a compact context package instead of a noisy dump;
the system logs what context was used.

This is not about making the AI sound more impressive.

It is about making the operational decision safer.

What I Would Measure Next

The current public version keeps metrics as metrics to collect, because I do not want to publish numbers that are not validated.

The useful metrics would be:

lead volume handled by the context resolver;
reduction in wrong-context replies;
human handoff rate by risk category;
response-time impact;
manual review time saved.

The Lesson

For AI agents in revenue workflows, the prompt is only one part of the system.

The harder design question is:

What should the model be allowed to know, trust and act on?

That is why I prefer designing AI agents as operating workflows: context resolution, retrieval, guardrails, structured outputs, human review and observability.

The public case study is here:

https://github.com/rkrisa/portfolio-ai-ops/tree/main/cases/context-aware-ai-reception-agent