DEV Community: lamingsrb

4 Systems I Built: Architecture Decisions That Held Up

lamingsrb — Thu, 20 Aug 2026 07:25:44 +0000

4 Systems I Built: Architecture Decisions That Held Up

Every architecture decision looks smart in a slide deck. The real test is what happens six months in, when the bill arrives, a model deprecates, or a client asks you to add a feature nobody planned for. I want to walk through four systems I built, the choices I made at the start, and which of those choices I would make again today.

I'll skip the tutorial framing. This is the honest version: what I picked, why, what broke, and what the numbers looked like once the thing was running in production.

System 1: ContentStudio, a multi-agent content and SEO machine

ContentStudio is the autonomous engine behind BizFlowAI. It researches topics, drafts, edits, optimizes for search and AEO, and publishes across multiple sites without me in the loop. It has been running for months and publishes on schedule.

Core decisions:

Decision	Choice	Why
Orchestration	Custom Node/TypeScript loop	LangGraph felt heavy for a linear pipeline with conditional edges
LLM	Claude API (Sonnet + Opus mix)	Best long-form writing quality; cost model works at my volume
Storage	Supabase (Postgres) + pgvector	One database for content, embeddings, and run history
Scheduling	Node worker on a small VPS + cron	Cheaper and simpler than serverless for a steady, predictable workload
Guardrails	Eval agent + hard checks (word count, links, factuality prompts)	The system will happily publish garbage without a gatekeeper

The interesting call was custom orchestration over a framework. I looked at LangGraph, CrewAI, and a few others. For a pipeline where I know the stages (research, outline, draft, edit, SEO pass, publish), a framework mostly adds abstraction I have to debug. A 400-line TypeScript state machine with typed transitions was clearer and cheaper to maintain. When I need branching, I add a conditional. When something breaks, the stack trace points at my code.

The choice I would revisit: I used Claude Sonnet for drafting and Opus for the final polish pass. That two-model split saved roughly 40% on inference versus running Opus end-to-end, with output quality my human review couldn't distinguish. With Opus 5 pricing where it is now, I would run more of the pipeline on the top model and drop a stage.

System 2: Sovereign AI POC with local LLMs and RAG

A different flavor entirely. This one is a proof of concept for scenarios where data cannot leave a private environment: regulated industries, government-adjacent workloads, anyone with a "no US-cloud LLM" rule. Everything runs local.

Stack:

Ollama serving Llama and Qwen variants on a workstation with a single consumer GPU
Postgres with pgvector for embeddings
Full-text search (Postgres FTS) alongside vector search
Reciprocal Rank Fusion to combine both retrievers
A thin FastAPI layer for the app

The hard decision was hybrid search over pure vector. Pure semantic retrieval looks magical in demos and then fails on the queries that matter: exact identifiers, product codes, names with unusual spellings, acronyms. FTS catches those. Vector catches paraphrased intent. RRF (a simple rank-based fusion, no tuning weights) combines them in a way that consistently beat either retriever alone on my test set.

Rough numbers on a 12k-document corpus:

Vector only: ~68% top-5 recall on my eval set
FTS only: ~61%
RRF hybrid: ~84%

That gap is the difference between "cute demo" and "the user trusts it."

The trade-off with local LLMs is honest: a 70B model on consumer hardware is slower than an API call to Claude, and the quality gap on hard reasoning is real. For extraction, summarization, and grounded Q&A over retrieved chunks, it is entirely workable. For open-ended reasoning, it isn't. I tell clients this upfront. A sovereign deployment is a constraint, not a feature.

System 3: Serverless AWS + Zendesk integration for SLA compliance

This one is older but the architecture lessons still apply. The problem: a ticketing system that was missing SLAs because the routing and escalation logic lived in a manual process. The solution: an event-driven serverless pipeline that watched Zendesk events, applied business rules, and triggered escalations before SLAs breached.

Architecture:

Zendesk webhook -> API Gateway -> Lambda (validate + normalize)
                                      |
                                      v
                                EventBridge (rules per event type)
                                      |
                    +-----------------+-----------------+
                    v                 v                 v
              Lambda (route)    Lambda (escalate)  Lambda (audit log)
                    |                 |                 |
                    v                 v                 v
                DynamoDB          Zendesk API        S3 + Athena

Why serverless and not a container on ECS or a small EC2:

Traffic is bursty and unpredictable. Ticket volume spikes when something breaks upstream. Lambda scales to that automatically. A container would be either overprovisioned or under.
The compute per event is tiny. Milliseconds of logic. Paying for an idle container 23 hours a day is silly.
EventBridge as the backbone meant I could add new consumers (a new escalation channel, a new dashboard feed) without touching existing code. Publish an event, subscribe a new Lambda. That decoupling saved me multiple times when requirements shifted.

The result was first-ever SLA compliance for that workflow. The lesson I keep coming back to: event-driven serverless is a genuine unlock for integration work, where the code is thin, the events are external, and the load is spiky. It is a bad fit when you have long-running processes, heavy dependencies, or per-request latency budgets that cold starts blow through.

Cold start was the one thing I fought with. For the routing Lambda that had to respond within a webhook timeout, I moved to provisioned concurrency for the two most-hit functions. That is essentially a paid warm pool. Adds cost, kills cold starts. Worth it for the hot path.

System 4: Analytics migration that cut annual cost

This wasn't a "build an AI thing" project, but the architecture reasoning is the same and it is the one that put the biggest number on the board: $30 to $60k a year in savings, depending on how you count.

The old system: a proprietary analytics platform with per-seat and per-event pricing that grew linearly with usage. The new system: a straightforward pipeline into a warehouse the client already owned, with dashboards built on top.

Decisions that mattered:

Own the warehouse, rent the compute. Storage in the client's warehouse was already paid for. What we replaced was the marked-up ingestion and query layer.
Batch where possible, stream where necessary. Most analytics questions do not need sub-minute freshness. Batching hourly cut compute cost by roughly an order of magnitude versus streaming everything.
Pick a query engine that matches the shape of the questions. For this workload, a columnar warehouse with materialized views for the hot dashboards was faster and cheaper than the general-purpose engine we replaced.

The generalizable lesson: the biggest wins in cloud architecture usually come from removing a per-usage tax, not from writing better code. If your unit economics get worse as you scale, no amount of clever caching fixes it. Change the pricing surface.

The pattern behind the four choices

Looking across all four systems, the decisions that held up were the ones I made against a specific constraint, not against a generic best practice.

Local LLM vs cloud API

Cloud API wins by default. Use a local LLM when one of these is true:

Data cannot leave a controlled environment (regulatory, contractual, sovereignty)
You need predictable per-token cost at very high volume and you have the ops capacity
Latency to a specific region matters more than model quality

Otherwise, Claude or OpenAI, every time. The engineering effort you save is worth more than the inference bill for almost every project under a certain scale.

Serverless vs containers

Serverless wins when:

Traffic is spiky or unpredictable
Per-request compute is small
You are integrating with external event sources
You want to add consumers without redeploying producers

Containers win when:

The workload is steady and known
You have long-running processes, heavy dependencies, or GPU needs
Cold start latency is unacceptable and provisioned concurrency doesn't cover it
Your team already runs Kubernetes well and adding serverless is cognitive overhead

For ContentStudio I picked a plain worker on a VPS. The workload runs on a predictable schedule, the container is warm, and I don't pay for cold starts I don't need. Serverless would have been the wrong answer even though I know AWS well.

Framework vs custom orchestration

I default to custom for anything I can hold in my head. LangGraph and similar frameworks earn their keep when you have genuinely complex graphs, dynamic agent spawning, or a team that needs a shared vocabulary. For a linear pipeline with a handful of conditionals, a typed state machine in code you wrote is easier to debug at 2 a.m. than a framework you learned from a README.

Vector vs hybrid search

Always hybrid. I have not built a production RAG system where pure vector search beat RRF hybrid on a real eval set. If you are shipping vector-only, you are leaving recall on the table for queries with exact terms, and those are often the queries that matter most.

What I'd do differently today

Three things:

Invest in evals earlier. On ContentStudio I built the eval harness after the first bad publish. It should have existed on day one. For any agentic system, the eval is the product's spine. Without it you cannot tell whether a prompt change made things better or worse, you only have vibes.
Pick the pricing surface before the tech. The analytics migration taught me that architecture decisions are often really pricing decisions in disguise. Before I write a line of code now, I ask: what is the unit cost, and how does it scale with success? If the answer is bad, no amount of engineering will save it.
Default to boring infrastructure. Postgres for state and vectors. A worker on a VPS or a Lambda behind API Gateway. A queue if I need one. Every time I have reached for something more exotic (a specialized vector DB, an orchestration framework, a bespoke event bus) I have paid for it in operational surface area. Boring wins.

The four systems above are not glamorous. They work, they run unattended, and they solve problems that had a dollar figure attached. That is the standard I hold my architecture decisions against.

If you are benchmarking your own stack choices or thinking through a build like one of these, I am always happy to talk shop. You can reach me at lazar-milicevic.com/#contact or dig through more of the blog for the deeper writeups on each system.

How I Scope AI Automation Work in 30 Days

lamingsrb — Thu, 20 Aug 2026 07:25:40 +0000

How I Scope AI Automation Work in 30 Days

Most AI automation projects do not fail because the model is weak. They fail because nobody decided what the system is allowed to do, what a correct result looks like, or who owns the edge cases before development starts. I have learned to treat scoping as engineering work, not a sales step.

A 30-day delivery window can produce a real, useful AI system, but only when the first call turns a vague request into a bounded workflow with measurable acceptance criteria.

I begin by finding the expensive manual decision

The first call is not about choosing Claude, OpenAI, LangGraph, or an agent framework. It is about identifying one recurring decision or handoff that costs time, creates delay, or causes errors, then defining what must happen before and after the AI touches it.

Clients often arrive with a request like, “We need an AI agent for operations,” or “Can you automate our support workflow?” Those are directions, not requirements. “Agent” is usually shorthand for a frustrating process that has grown beyond what a spreadsheet, Zapier flow, or a person checking inboxes can handle.

My first goal is to isolate the real unit of work.

For example, a workflow may actually be:

A customer submits a request through a form, email, or ticket.
An employee reads it and identifies the category.
They look up information in two or three systems.
They decide whether the request is valid, urgent, or complete.
They create a record, draft a reply, or route it to the right team.
Someone checks the result because a wrong decision is expensive.

That is much more useful than “build an agent.” It tells me where the AI should classify, retrieve information, draft text, trigger an action, or stop and ask a human.

On an initial call, I ask questions that force the workflow into the open:

What happens today, step by step, from trigger to completion?
How many times per week or month does this happen?
Which part takes the most human time?
What is the cost of a bad output?
What systems hold the source of truth?
Is the input structured, such as a form, or messy, such as email attachments?
Can the system take action automatically, or should it only recommend an action?
Who can make decisions when I need an answer within 24 hours?
What would make you say the project worked after 30 days?

The last question is the most important. If the answer is “we will know it when we see it,” I do not yet have a project I can responsibly quote.

I have built unattended systems, including a four-system automation ecosystem that saved more than 73 hours per month. The work was not valuable because it contained AI. It was valuable because the systems had clear triggers, reliable handoffs, and an agreed definition of done.

I map the workflow before I choose an AI architecture

I scope AI automation by drawing the current workflow and proposed workflow side by side. This exposes whether the problem needs an LLM, retrieval, deterministic rules, an integration, or a process change that should happen before any model call.

A useful discovery artifact is not a long strategy deck. It is a one-page workflow map that makes ownership, data movement, and failure paths visible.

Here is the level of detail I want before I recommend an architecture:

Workflow element	Question I need answered	Example decision
Trigger	What starts the process?	New Zendesk ticket, inbound email, scheduled job
Input	What data arrives and in what format?	Ticket text, customer ID, PDF attachment
Source of truth	Which system is authoritative?	PostgreSQL, CRM, help desk, internal knowledge base
AI task	What judgment is being delegated?	Classify, extract, summarize, draft, rank
Action	What can the system change?	Create ticket, update CRM field, send draft for approval
Human review	When must a person intervene?	Low confidence, missing data, financial or legal impact
Evidence	How do we prove it worked?	Audit log, before-and-after sample, SLA report

This is where I separate AI assistance from autonomous action.

A system that drafts a support response and waits for approval has a very different risk profile from one that changes account status or sends a customer-facing message on its own. Both can be useful. The mistake is pretending they deserve the same architecture, testing plan, and delivery estimate.

For a narrow internal workflow, I often start with deterministic orchestration around one LLM step:

Event arrives
  -> validate required fields
  -> retrieve approved context
  -> ask model for structured output
  -> validate output against schema and business rules
  -> either take allowed action or route to human review
  -> log input, decision, action, and outcome

That is an agentic workflow in the practical sense. It may include planning or tool use, but it does not need a free-running autonomous agent that can invent its own sequence of actions.

I use RAG only when the answer depends on a body of internal knowledge that cannot fit reliably in a prompt. If the workflow only needs a customer record and a handful of fields from a database, direct retrieval through an API is usually better than building a vector database.

For knowledge-heavy tasks, I define the retrieval problem explicitly:

What documents are approved for use?
How often do they change?
What is the document-level permission model?
Does the answer need citations?
What should happen when retrieval returns weak or conflicting evidence?

In production RAG work, hybrid search is often more reliable than vector search alone. Combining PostgreSQL full-text search with pgvector and reranking results through reciprocal rank fusion can prevent exact product names, ticket IDs, and policy terms from disappearing behind semantically similar but wrong passages.

I turn 30 days into four acceptance checkpoints

A 30-day AI automation engagement is realistic for one bounded workflow with accessible systems, a responsive owner, and a decision maker who can resolve trade-offs quickly. It is not realistic for replacing an entire operations department, cleaning five years of data, and integrating every system at once.

I organize delivery around weekly evidence, not weekly status updates. At the end of every week, the client should be able to see something concrete, test it, and change direction while the change is still cheap.

Week	Delivery checkpoint	What I need from the client	Evidence of progress
1	Workflow specification and technical design	Access path, sample inputs, business owner decisions	Approved scope, test cases, architecture diagram
2	Working vertical slice	Sandbox credentials, representative data	One input travels through the full proposed path
3	Integration, controls, and evaluation	Feedback on outputs and exceptions	Measured test set, logs, human-review path
4	Production release or controlled pilot	Deployment approval, owner for operations	Runbook, monitoring, handover, acceptance review

Week 1: Define the narrowest useful system

The output from week one is a short scoping document. Mine usually contains:

The business outcome and baseline process
In-scope and explicitly out-of-scope work
System diagram and integration boundaries
Data classification and access assumptions
Acceptance criteria
Known risks and unresolved decisions
Delivery checkpoints
Operating ownership after launch

I write acceptance criteria as observable behavior. “AI should classify tickets accurately” is weak. “For the agreed test set, the system must return one of six approved categories, include a reason and source references where applicable, route uncertain cases to review, and never update a ticket without a valid category” is testable.

Week 2: Build the vertical slice

I want one real input to travel through the entire system early. That means trigger, retrieval, model call, validation, output, and logging, even if the interface is ugly and the dataset is small.

This catches the problems that diagrams hide: an API does not expose the field we expected, source documents have inconsistent permissions, tickets contain unexpected formats, or the organization does not agree on the categories it wants the AI to use.

Week 3: Test the failure modes, not just the happy path

By week three, I stop judging the system by impressive examples. I test missing fields, duplicate events, ambiguous requests, stale documents, API failures, retries, long inputs, and model outputs that are syntactically valid but operationally wrong.

AWS frames one of its reliability design principles as “Automatically recover from failure.” I agree with that principle, but for AI systems I add a condition: recover automatically only when the action is reversible and the system has enough evidence to proceed. For everything else, fail visibly and route the case to a person. The AWS Well-Architected Reliability Pillar is a useful reference for designing those operational controls.

Week 4: Ship with ownership, not just credentials

A deployed endpoint is not a shipped system. I consider the work shipped when there is a deployment path, monitoring, a runbook, access ownership, and a clear answer to “what happens when this fails at 2 a.m.?”

For serverless AI architecture, that often means an event trigger, queue or retry strategy, Lambda workers, managed secrets, structured logs, and alerts for failure rate or backlog growth. In an AWS and Zendesk integration I built, operational reliability was central to delivering the first SLA compliance for that workflow. The integration mattered, but the visible state, retries, and escalation path mattered just as much.

I quote the boundary, the risk, and the operating cost

A useful AI automation quote describes a system boundary and a delivery outcome. It should not sell an unlimited promise to “build AI agents” without defining the data, integrations, evaluation method, and support model.

I avoid fixed pricing against an undefined problem. That does not protect the client or the engineer. It rewards ambiguity at the beginning, then creates tension when real constraints emerge.

Before I quote, I need answers in five areas:

1. Workflow value

I estimate time saved from the current process, not from optimistic AI claims. If three people spend 15 minutes each on 200 requests per month, that is 150 hours of manual effort before considering rework or delay.

The actual savings may be lower if the workflow still needs human review. That is fine. A system that safely removes 60% of repetitive work is often more valuable than a fully autonomous system nobody trusts.

2. Integration reality

Every external system changes the estimate. API quality, authentication, rate limits, sandbox availability, webhooks, and the ability to write data back all affect delivery risk.

I have seen simple-looking integrations become difficult because the source system had no reliable event trigger, no stable identifier, or no way to distinguish a retry from a new request. Those details are scoping facts, not implementation trivia.

3. Data and security

I need to know whether the AI will process customer information, internal documents, regulated data, credentials, or financial records. I do not assume that because a team can paste data into a chatbot, the same data can be sent through a production API.

I document where data is stored, which providers process it, retention requirements, access roles, and whether prompts or outputs must be redacted. When the policy is unclear, that decision belongs with the organization’s security and legal owners before I build.

4. Evaluation

LLM application development needs an evaluation plan before launch. I ask for a representative test set, including normal cases and bad cases. For a classification workflow, I want known labels. For a RAG assistant, I want questions with expected supporting sources. For a drafting workflow, I want a reviewer rubric.

Without this, teams tend to evaluate generative AI by reading a few impressive outputs. That is how a demo becomes a production surprise.

5. Operations after delivery

I clarify whether I am delivering a managed system, handing over infrastructure, or supporting a limited stabilization period. The system needs an owner on the client side regardless. Someone must decide when a prompt change is acceptable, when a knowledge source is outdated, and when a failed action requires manual cleanup.

I decline work when the conditions make a good result unlikely

I decline AI implementation work when the project is framed as a shortcut around missing ownership, poor source data, or an unresolved business decision. An LLM can help process information, but it cannot make an organization agree on its own rules.

These are the red flags I take seriously:

No process owner. If nobody owns the workflow, nobody can resolve edge cases or accept the result.
No access to representative data. I cannot validate an AI proof of concept using only invented examples.
The desired action is high-risk and irreversible. Automatically sending legal, financial, employment, or account-changing decisions without an approved control model is not a reasonable first engagement.
“It needs to be fully autonomous” is a requirement before the workflow is understood. Autonomy is an architectural choice, not a maturity badge.
The project has no measurable outcome. “Use AI more” is not an outcome. Reduced handling time, faster response, fewer routing errors, or improved SLA compliance are.
A buyer wants a fixed promise but cannot provide a decision maker. A 30-day project cannot spend 10 days waiting for access approval or basic process answers.
The team expects the model to compensate for bad data without changing the process. Sometimes it can tolerate messy inputs. It cannot create a trustworthy source of truth from conflicting records.

Declining a project is not about being difficult. It is often the most useful advice an AI automation consultant can give. I would rather define a smaller pilot that has a chance of working than deliver an expensive system that becomes another unused dashboard.

What I would do before starting an AI automation project

If I were hiring an AI engineer, a fractional AI engineer, or an AI integration consultant for a 30-day engagement, I would insist on five things before signing:

One workflow, one owner, one measurable outcome.
A written list of systems and data required to build it.
A decision on human review and allowed automated actions.
A representative test set, not just a few ideal examples.
Weekly checkpoints where working software is demonstrated.

I would also start narrower than feels comfortable. The first custom AI agent for a business should earn trust by handling one repeatable job well. Once it has reliable inputs, observable outputs, and a clear owner, expanding to adjacent workflows is much easier.

That is how I have approached my own work on BizFlowAI ContentStudio and other autonomous systems. The useful part is not that a model can generate content, classify requests, or call tools. The useful part is the operating loop around it: input controls, evaluation, feedback, retries, and a way to improve based on real outcomes.

A good AI MVP is not a miniature version of every future feature. It is the smallest system that proves a valuable workflow can run safely in the real environment.

The first call should leave both sides with more than enthusiasm for AI agents. It should produce a shared view of the workflow, the risks, the delivery boundary, and the evidence required to call the project successful. If you are planning an AI automation or LLM application and want to compare notes on the scope, you can reach me through my contact page or explore more of my writing on the blog.

@Claude In Slack Is Four Components. Here's The Whole Build.

lamingsrb — Thu, 20 Aug 2026 06:12:44 +0000

@claude In Slack Is Four Components. Here's The Whole Build.

Anthropic shipped @claude as a first-class Slack participant and LinkedIn spent a week calling it a paradigm shift. It's four components: a webhook, a history fetch, a context assembler, and an async job runner. If you're staring at a $30-per-seat quote wondering whether you actually need it — or you're the builder whose clients keep forwarding you that quote — this is the teardown.

The four components, drawn end-to-end

The mention-your-bot pattern has one novel piece and three pieces of plumbing that have shipped in production since GitHub open-sourced Hubot in 2015. Here they are in order of execution:

Mention webhook. Slack, Telegram, Discord, WhatsApp Business — every one of them POSTs to an HTTPS endpoint the instant a user tags your bot. ~20 lines of code.
Channel history fetch. On mention, you call the platform's conversations.history (Slack) or getUpdates (Telegram) endpoint and pull the last N messages. That's your immediate context window.
Context assembly. Take that history, optionally join it with a jobs table or vector store for anything older than the channel window, and build a single prompt.
Async job runner with callback. Claude can take 20–40 seconds. Slack expects a 200 within 3. So you ack fast, enqueue the real work in Redis, and a worker posts the reply back into the thread when it finishes.

That's the system. The "remembers across days" magic that everyone got excited about? A jobs table with a thread_id column and a SELECT. Not new technology.

What each component actually costs to run

Webhook receiver: FastAPI or Express, one route, ~$0 marginal.
History fetch: 1 API call per mention, rate-limited but free.
Context assembler: pure Python/TS, no external cost.
Worker + Redis: 1 process, ~150 MB RAM on a $6 VPS.

Component 1 & 2: webhook and history in ~40 lines

Here's a stripped Slack handler. It verifies the signature, acks in under a second, pulls the last 20 messages, and drops the job on a queue.

from fastapi import FastAPI, Request, BackgroundTasks
from slack_sdk import WebClient
import os, hmac, hashlib, time, redis, json

app = FastAPI()
slack = WebClient(token=os.environ["SLACK_BOT_TOKEN"])
q = redis.Redis.from_url(os.environ["REDIS_URL"])

@app.post("/slack/events")
async def events(req: Request, bg: BackgroundTasks):
    body = await req.body()
    # Signature verification omitted for brevity — do NOT ship without it
    payload = json.loads(body)

    if payload.get("type") == "url_verification":
        return {"challenge": payload["challenge"]}

    event = payload["event"]
    if event["type"] == "app_mention":
        history = slack.conversations_history(
            channel=event["channel"], limit=20
        )["messages"]
        job = {
            "channel": event["channel"],
            "thread_ts": event.get("thread_ts") or event["ts"],
            "user": event["user"],
            "text": event["text"],
            "history": history,
        }
        q.lpush("claude_jobs", json.dumps(job))
    return {"ok": True}

Two things worth calling out. First, always verify the X-Slack-Signature header — Slack's docs walk through the HMAC. Skip it and anyone can POST fake mentions to your endpoint. Second, that conversations_history call is your entire "context window" for zero-effort setups. Twenty messages is usually plenty; if a thread runs long, switch to conversations_replies with the thread_ts.

Component 3: context assembly is where the product actually lives

This is the part the launch video skipped, and it's the only piece where you have real leverage. Fetching 20 messages and shoving them at Claude gets you a demo. Joining those messages against your CRM, your invoice history, and last week's on-call notes gets you a product.

A reasonable prompt assembler looks like this:

def build_prompt(job, db):
    thread_id = job["thread_ts"]
    prior = db.execute(
        "SELECT question, answer FROM jobs "
        "WHERE thread_id=%s AND status='done' "
        "ORDER BY created_at ASC LIMIT 10",
        (thread_id,),
    ).fetchall()

    recent = "\n".join(
        f"{m.get('user','?')}: {m['text']}" for m in reversed(job["history"])
    )
    memory = "\n".join(f"Q: {p.question}\nA: {p.answer}" for p in prior)

    return f"""You are the ops bot for Acme Inc.
Prior thread context:
{memory or '(none)'}

Recent channel messages:
{recent}

The user just asked: {job['text']}
Answer concisely. If you need CRM or invoice data, call the appropriate tool."""

The jobs table is the "persistence across days" that got framed as a feature. Schema:

column	type	purpose
id	uuid	primary key
thread_id	text	Slack `thread_ts` / Telegram `message_thread_id`
channel	text	routing
question	text	user's tagged message
answer	text	worker output
status	enum	queued / running / done / failed
created_at	timestamp	ordering + TTL

That's it. That is the memory system.

Component 4: async worker with callback

Slack's 3-second ack rule is the reason you cannot answer inline. Every production build of this pattern uses a queue. Redis + one worker process handles thousands of mentions a day on a $6 box.

# worker.py
import redis, json, os, anthropic, time
from slack_sdk import WebClient

r = redis.Redis.from_url(os.environ["REDIS_URL"])
slack = WebClient(token=os.environ["SLACK_BOT_TOKEN"])
claude = anthropic.Anthropic()

while True:
    _, raw = r.brpop("claude_jobs")
    job = json.loads(raw)
    prompt = build_prompt(job, db)

    resp = claude.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}],
    )
    answer = resp.content[0].text

    slack.chat_postMessage(
        channel=job["channel"],
        thread_ts=job["thread_ts"],
        text=answer,
    )
    db.execute(
        "INSERT INTO jobs(thread_id, question, answer, status) "
        "VALUES (%s, %s, %s, 'done')",
        (job["thread_ts"], job["text"], answer),
    )

Real numbers from a client bot I run on Telegram with this exact shape: median mention-to-reply is 8.2 seconds, p95 is 19 seconds, and the box (a $6/mo VPS with 1 vCPU / 1 GB RAM) sits at 4% CPU handling roughly 400 mentions/day.

Failure modes you actually hit in production

Claude times out or 529s — retry twice with exponential backoff, then post an apology into the thread instead of leaving the user hanging.
Slack rate-limits conversations.history at Tier 3 (~50/min) — cache per-channel for 30 seconds.
Worker dies mid-job — mark jobs running with a heartbeat; re-queue anything running >60s.
Duplicate mention deliveries — Slack retries on non-200; dedupe on event_id.

The $3,600 vs $150 math

Let's price the exact same capability two ways for a 10-person team.

Line item	Anthropic Team plan	Self-hosted
Seats (10 × $30/mo)	$3,600/yr	—
VPS ($6/mo)	—	$72/yr
Anthropic API usage (~400 msgs/day, Sonnet)	included	~$60–90/yr
Redis (bundled on VPS)	—	$0
Domain + TLS (Let's Encrypt)	—	~$12/yr
Total year one	$3,600	~$150

That's a 24× gap for a pattern that a competent operator wires up in a weekend. And the self-hosted version isn't locked to Slack — the same four components drop into Telegram for teams that already live there, WhatsApp Business for field ops, or Discord for community businesses. "Meet users where they work" stops being a marketing line and becomes an architecture decision.

Two honest caveats on that math. If your team already pays for Anthropic Team for the Claude web app, the Slack integration is a free add-on and the calculation flips. And if your ops person bills at $150/hr, a 16-hour build is $2,400 — still cheaper than year one, break-even happens fast, but it's not zero.

Who should pay Anthropic and who should build

Pay Anthropic if:

You're a regulated shop with SOC 2 / HIPAA obligations and a security team that won't approve a custom bot with chat:write on your workspace.
Procurement is a 6-week gate and you don't have infra ownership.
Nobody on your team wants to own a worker process at 2 AM.

Build it if:

You have a dev-friendly operator (or you are one).
Your workflows need to touch your CRM, invoicing system, or internal database — off-the-shelf can't do that.
You want the same bot on two or more channels (Slack + WhatsApp is the most common combo I see).

The primitive is not the moat. The integration into your specific workflow is the moat. A bot that knows your customers, has read/write access to your pipeline, and can actually close a loop — that's worth building. A bot that summarizes the last 20 messages in a channel is worth $0 because in twelve months every messaging platform will ship a first-party version.

Why bizflowai.io helps with this

Most of what we ship for clients is exactly this pattern, extended: a mention-driven bot on Slack, Telegram or WhatsApp that reads a specific channel, joins the message against the client's CRM or invoicing system, drafts a reply or triggers a workflow (send quote, update status, schedule follow-up), and logs everything to a jobs table the operator can audit. The four-component skeleton in this post is the starting point. What clients pay for is the integration into their stack — the part Anthropic's $30/seat plan doesn't touch.

Your one action this week

Open your team's messaging tool. Find the one question that gets asked three times a week where someone always answers by pulling context from two other places. That's your first tag-your-AI target. Write the four components against it: which webhook, which history call, what context sources, what callback action. If you can name all four in a paragraph, you have a spec. A spec is 2–3 days of work for a builder who's done it before.

Want more like this?

I publish practical AI automation, GenAI engineering, and faceless content workflows on YouTube every week.

Subscribe to bizflowai.io on YouTube — never miss a new tutorial.

Planning an AI automation project or need a second opinion on your architecture?

Connect with me on LinkedIn — Lazar Milicevic, GenAI Engineer & bizflowai.io Founder.

Visit bizflowai.io for our services, case studies, and AI consulting.

I/O 2026 Put An Agent In Chrome. 2 Of 3 Users Broke It In 4

lamingsrb — Thu, 20 Aug 2026 06:12:41 +0000

Chrome's Agent Ships Without A Seatbelt. Here's The Server-Side Fix.

Google I/O 2026 dropped an AI agent into every Chrome tab. I ran it across a shared invoicing tool with four teammates for one afternoon. Two of them fired actions they never approved, one auto-sent an invoice to a real client, and none of it was logged anywhere I could pull. If your ops team shares browser profiles to do actual work, the governance hole is bigger than the productivity win.

The three I/O 2026 updates, ranked for an ops team (not a developer)

Every dev recap ranks these by API surface. Here's the ranking that matters if your bookkeeper gets sued when the agent misfires: Skills is the highest-risk update, WebMCP is second, Built-in AI is third. Skills lets a non-technical user install a packaged workflow from a blog post and trigger it from the address bar — same threat model as installing random Chrome extensions on the accounting laptop, except the install friction is one click and the permissions UI is unreadable.

Here's what each one actually does:

Update	What it is	SMB risk
WebMCP	Sites expose tools an agent can call directly from the browser tab	Agent gets write access to SaaS tools using the logged-in session
Built-in AI	Gemini Nano runs locally, inference is free and offline	Agent acts without a network round-trip you can log at the edge
Skills	One-click packaged workflows installed from any URL	Non-technical user grants permissions they can't read; no rollback

Every dev recap I read this week framed these as productivity. Fine for a solo builder. For a 3-person ops team sharing one Chrome profile against a live invoicing tool, the ranking flips. Skills is where a mistake hits a client. WebMCP is where your own SaaS becomes the weapon. Built-in AI is the reason your normal network logs won't show you what happened.

What broke in 4 hours of real testing

I ran this last week: three shared accounts on the invoicing tool we use daily, four users, one browser profile pattern, real client data. Not a sandbox. Within four hours, two of the three users triggered unintended actions.

The specific failure that still bothers me: one teammate installed a Skill that was supposed to "prep invoice drafts from selected line items." She hovered over a draft to read it. The Skill interpreted the hover as intent-to-send. The invoice went out. To a real client. For the wrong project.

Here's what Chrome did not give me after the fact:

No default event log for agent actions in the browser
No email confirmation before the send
No undo / rollback
No permission-scope UI a non-developer could read to figure out what the Skill was allowed to do
No server-side notification that this request came from an agent vs. a human click

The only reason I know the send happened is the client replied asking why the invoice was for the wrong project. That's the audit trail. A confused email from a customer.

If you're small and shared-browser is your reality, this is the failure mode you inherit by default.

The three things to do this week

Don't wait for Google to ship a governance layer. Do these three things on Monday.

The checklist

Disable Skills on any shared browser profile. Managed policy on the profile, or just turn it off in settings on every machine the accounting/ops team touches. Treat a Skill install like installing a random Chrome extension on the finance laptop — because functionally it is.
If you own the web app your team uses, do not expose write actions through WebMCP. Read-only tools are fine (list invoices, search clients, summarize a report). Anything that sends, charges, deletes, refunds, or notifies a customer needs a human click the agent cannot fake.
Log every agent-originated request server-side with the user, the tool called, the arguments, and a timestamp. Chrome will not do this for you. Your backend has to.

The third one is the boring work nobody showed a slide for at the keynote. It's also the difference between "we caught it in 5 minutes" and "the client emailed us three days later."

How to expose a WebMCP tool safely: a real pattern

Here's the pattern I use when a client asks to expose their SaaS to browser agents. Read-only endpoints are unrestricted. Write endpoints require a confirmation token that only a human interaction can generate.

# server-side: FastAPI example
from fastapi import FastAPI, HTTPException, Header
from pydantic import BaseModel
import secrets, time

app = FastAPI()
PENDING = {}  # token -> {user, action, args, expires}

class SendInvoiceArgs(BaseModel):
    invoice_id: str
    client_email: str

@app.post("/mcp/invoice/prepare_send")
def prepare_send(args: SendInvoiceArgs, user: str = Header(...)):
    # Agent can call this freely. It does NOT send.
    # It returns a token + a preview the user must confirm in-app.
    token = secrets.token_urlsafe(24)
    PENDING[token] = {
        "user": user,
        "action": "invoice.send",
        "args": args.dict(),
        "expires": time.time() + 120,
    }
    return {"confirm_token": token, "preview_url": f"/confirm/{token}"}

@app.post("/invoice/confirm_send/{token}")
def confirm_send(token: str, user: str = Header(...), csrf: str = Header(...)):
    # This endpoint is ONLY callable from a real form submit in the app UI,
    # protected by CSRF. The agent cannot forge the CSRF token.
    pending = PENDING.pop(token, None)
    if not pending or pending["expires"] < time.time():
        raise HTTPException(400, "expired or invalid")
    if pending["user"] != user:
        raise HTTPException(403, "user mismatch")
    # ... actually send the invoice ...
    audit_log(user=user, action="invoice.send",
              args=pending["args"], source="human_confirm")
    return {"ok": True}

Two properties matter here:

The MCP-exposed endpoint (prepare_send) never mutates state. Worst case, the agent spams your pending table, which you rate-limit.
The confirming endpoint requires a CSRF token that only your app's real UI issues. An agent driving the browser can click a button, so add a short-lived one-time code shown in a modal — a human reads it and types it, an agent using DOM automation reads it too. If you need real resistance, require WebAuthn on the confirm step. Now the agent physically can't complete it without a hardware key touch.

This is the "seatbelt" the keynote didn't ship.

The audit log Chrome won't give you

The other thing your backend has to do: mark every request that originated from an agent and log it separately from human traffic. Chrome doesn't set a standard header for this yet, but you can enforce your own convention on any tool you expose via WebMCP.

# middleware: tag agent-originated requests
@app.middleware("http")
async def tag_agent_requests(request, call_next):
    is_agent = (
        request.headers.get("x-mcp-client") is not None
        or request.headers.get("sec-agent-initiated") == "1"
    )
    request.state.is_agent = is_agent
    response = await call_next(request)
    if is_agent:
        await audit_write({
            "ts": time.time(),
            "user": request.headers.get("x-user-id"),
            "path": request.url.path,
            "method": request.method,
            "agent_client": request.headers.get("x-mcp-client"),
            "body_hash": request.state.body_hash,
            "response_status": response.status_code,
        })
    return response

Ship this before you expose a single write tool. When something misfires — and it will — you need to answer three questions in under 60 seconds: which user, which agent, what did it call, with what arguments. If your log can't answer those, you're the person emailing the client an apology while grep-ing nginx access logs at 11pm.

Two more practical rules I enforce on client projects:

Rate-limit per-user, per-tool. An agent that fires 40 prepare_send calls in 10 seconds is not a workflow, it's a bug. Cap it.
Reversibility window on destructive actions. Sends, deletes, charges — hold them in a 30-second "sent, but revocable" state and show a banner in the app. This is what Gmail's undo-send taught us. The agent can trigger, but the human still has a physical window to catch it.

The gap that decides who wins the SMB market

Here's the hot take. The agentic web is a governance problem before it's a productivity win. Whoever ships the audit layer, the permission-scope UI a non-developer can actually read, and a real undo button for browser-native agents will own the SMB segment for the next five years. Not the model vendors. Not Google. The person who makes it safe for a bookkeeper to install a Skill without hosing a client relationship.

Google's own Agent Safety guidance talks about human-in-the-loop for consequential actions. Fine as a principle. The Chrome shipping surface as of I/O 2026 does not enforce it — that's on you to add at the app layer and the profile-management layer.

If you're a small team, the near-term move is defensive: turn Skills off on shared profiles, expose only read tools via WebMCP, log everything server-side, and put human-confirmation gates on anything that touches a customer. If you're building a SaaS product, the opportunity is offensive: be the vendor that ships the confirmation flow, the per-tool audit stream, and the undo window. That's the market gap I see every week in real SMB deployments.

Where this lands in practice

I ship browser and workflow automation into small US ops teams every week — invoicing, CRM writes, email triage, lead follow-up. The pattern in this post (read-only MCP, human-confirmed writes, server-side audit, per-tool rate limits, revocable-send windows) is roughly the default checklist we apply before any agent touches a client's live SaaS. Most of the "agent went rogue" stories I hear from SMB owners resolve to two missing pieces: no confirmation gate on write actions, and no server-side log tagged with which agent called what. Both are a weekend of backend work, not a rewrite.

The keynote sold you the agent. Nobody sold you the seatbelt. Build the seatbelt before you let the agent drive.

Want more like this?

I publish practical AI automation, GenAI engineering, and faceless content workflows on YouTube every week.

Subscribe to bizflowai.io on YouTube — never miss a new tutorial.

Planning an AI automation project or need a second opinion on your architecture?

Connect with me on LinkedIn — Lazar Milicevic, GenAI Engineer & bizflowai.io Founder.

Visit bizflowai.io for our services, case studies, and AI consulting.

17 Hires, 0 Pre-Start Ghosts, $1.62: The Post-Signature

lamingsrb — Mon, 17 Aug 2026 06:12:40 +0000

17 Hires, 0 Pre-Start Ghosts, $1.62: The Post-Signature Agent

Every ATS and monday.com recruiter agent stops the moment the offer is countersigned. That silence is where I was losing 23% of hires on the last cohort — counteroffers landed in the seven-day gap between signature and start date. Here's the agent I built to own that window, the exact architecture, and the token receipts.

The gap every recruiting stack ignores

The post-signature to day-one window is where hires quietly evaporate — 23% of my last cohort updated LinkedIn within 72 hours of signing and fielded a counteroffer before they ever received a laptop. Every tool in the recruiting stack optimizes the funnel up to signature. Sourced, screened, interviewed, offered, signed. Green dashboard. Then nothing. No equipment ticket, no Slack invite, no payroll form, no manager intro, no welcome doc.

The old employer, meanwhile, sees the LinkedIn update within a day and starts a counter. You're not competing with the market anymore — you're competing with a manager who has a week of unopposed access to someone you already spent about $4,000 closing (recruiter time, interview loops, offer negotiations). If one in four walks, your true cost-per-hire is 33% higher than your ATS reports.

The lesson from running this end-to-end: the bottleneck isn't screening 500 resumes. It's the 168 hours between signed and started. Nobody automates that window because it doesn't feel like a "recruiting" problem — it sits between HR, IT, finance, and the hiring manager, and everyone assumes someone else is running it.

Trigger and state: DocuSign → n8n → Supabase

The whole system runs off a single source of truth: one row in a Supabase table called onboarding_state, written the moment the offer is countersigned in DocuSign. Every subsequent agent run reads and writes to that row, which kills the two problems that break most multi-step agent workflows — lost context and duplicated messages.

DocuSign fires a Connect webhook on envelope completion. n8n receives it, extracts candidate fields, and inserts:

create table onboarding_state (
  id uuid primary key default gen_random_uuid(),
  candidate_name text not null,
  candidate_email text not null,
  start_date date not null,
  role text not null,
  location text not null,
  manager_email text not null,
  equipment_profile text,
  interview_notes text,
  day_count int default 0,
  status text default 'day_zero',
  sentiment_score jsonb,
  last_touchpoint_at timestamptz,
  created_at timestamptz default now()
);

The status field only ever has four values: day_zero, active, flagged, started. The day_count gets incremented by whichever cron ran last. sentiment_score stores the JSON from the day-three sensor. That's it — no separate tables for messages, no external state store. If an agent crashes mid-run, the next cron just reads the row again and picks up.

Why Supabase and not a Google Sheet? Two reasons. Row-level locking so two crons can't collide on the same candidate, and a real SQL layer so the day-N query is a one-liner instead of a full-sheet scan.

The six-day cron sequence

Six n8n cron triggers, one per day from day one through day six. Each cron runs at 9 a.m. local, queries Supabase for day_count = N AND status = 'active', and fires a Claude agent per matching row with a day-specific job. Splitting into six discrete crons instead of one long workflow means a failure on day three doesn't block day four for other candidates.

Here's what each day actually does:

Day	Job	Systems touched	Avg tokens/hire
1	Equipment order + Google Workspace + Slack	IT ticketing API, Google Admin, Slack	~1,200
2	Payroll forms email + pre-filled DocuSign	Email, DocuSign, finance inbox	~900
3	Warm check-in from hiring manager (sensor)	Email, sentiment webhook	~4,800
4	(rest day — no touchpoint by design)	—	0
5	Manager 1:1 scheduling	Google Calendar, Calendly-style link	~1,100
6	Personalized welcome doc	Email, Google Docs	~3,200

Day one is where the operational commitments show up. The agent reads role and location, picks the laptop spec from a config file (a senior engineer in Austin gets a different SKU than a support rep in Denver), files the IT ticket with the shipping address, provisions the Google Workspace account, and drops the hire into the right Slack channels with a founder-signed intro. The candidate wakes up on day two knowing their laptop is on the way and sees a welcome from the CEO in #new-hires. That single sequence kills 60% of the counteroffer temptation on its own.

Day two is the boring but critical one — payroll forms specific to their state and employment type (W-2 vs. 1099 vs. international contractor), a pre-filled DocuSign envelope so they're not hunting for their SSN in three tabs, and finance CC'd so nothing falls through.

Day three: the sentiment sensor

Day three is the only touchpoint designed to generate a reply, and the reply is the sensor for the whole seven-day window. The agent writes a short, non-templated note from the hiring manager that references something specific from the interview notes and asks one open question. When the candidate replies, sentiment analysis fires and either lets the sequence continue or triggers a recruiter alert.

The generation prompt is boring on purpose:

You are drafting a short check-in email from {manager_name} to {candidate_name}
who signed 3 days ago and starts on {start_date}. 

Read these interview notes and reference ONE specific thing they said they were
excited about. Do not use the word "excited." Do not use "just checking in."

Ask exactly one open-ended question that invites a real reply, not a yes/no.

Notes: {interview_notes}

Keep it under 80 words. Sign as {manager_name}, no title, no signature block.

Reply comes back into a separate n8n webhook. That webhook runs a second Claude call with a structured output prompt:

Rate this candidate reply on three dimensions, 1-5:
- excitement (5 = clearly energized, 1 = flat/perfunctory)
- hesitation (5 = raising doubts/concerns, 1 = none detectable)
- specificity (5 = engaged with the specific question, 1 = generic response)

Return JSON only: {"excitement": N, "hesitation": N, "specificity": N, "signal": "..."}

Reply: {reply_body}

Rules are hardcoded in the next n8n node: if excitement < 3 OR hesitation > 3, flip status to flagged and fire a Telegram message to the recruiter with the candidate name, the reply, the JSON scores, and a one-paragraph rescue script generated on the fly from what the candidate actually said. The agent doesn't try to save the hire. It hands the recruiter a warm intervention with the exact language to use.

Day three is also the most expensive step by a mile — 91¢ of the total $1.62 cohort spend, because the sentiment call carries the full interview notes as context.

Escalation logic and the human handoff

Automation stops the moment there's a real signal. Two escalation triggers fire independently of the cron schedule: 48-hour silence on any touchpoint, or a sentiment score outside the safe band on day three. Both route to the recruiter's Telegram with everything they need to act in one message.

The silence detector is a separate n8n workflow that runs every 6 hours:

// n8n Function node
const now = new Date();
const stale = items.filter(row => {
  const last = new Date(row.json.last_touchpoint_at);
  const hours = (now - last) / 36e5;
  return hours > 48 && row.json.status === 'active';
});
return stale.map(r => ({ json: r.json }));

Each stale row triggers a Claude call that reads the last message sent, the candidate's role, and their interview notes, then drafts a suggested outreach in the recruiter's voice. Telegram message goes out with candidate name, days since last contact, the last message sent, and the suggested reply. The recruiter taps once, edits if needed, sends.

On the last cohort of 17 hires, four Telegram alerts fired. The recruiter intervened on three of them. All three signed on to day one. The fourth was a false positive — the candidate was on a pre-start vacation and replied warmly two days later. That's the ratio I want: the system errs slightly toward escalation, because a false positive costs 90 seconds of recruiter attention and a false negative costs $4,000.

What the numbers actually looked like

17 hires processed end-to-end
43 automated touchpoints delivered (some hires triggered fewer than 6 based on role/location)
0 pre-start ghosts (down from ~4 expected at the historical 23% rate)
$1.62 total in Claude tokens across the whole cohort
91¢ on day-three sentiment runs, 48¢ on welcome doc generation, 23¢ on everything else
Runs on a Raspberry Pi 4 next to the main server, ~$0 infra beyond electricity

At $1.62 in tokens versus 3 counteroffers saved at ~$4,000 sunk cost each, the ROI math is not close.

Why bizflowai.io helps with this

Post-signature onboarding is exactly the kind of cross-system workflow bizflowai.io already builds for SMB clients — the boring wiring between DocuSign, an ATS, IT ticketing, Google Workspace, Slack, and finance that nobody on a 10-person team has time to own. When we ship an onboarding agent, it looks like the architecture above: one state row per hire, discrete daily jobs, sentiment on the human touchpoints, and Telegram escalation to a real person when something goes off-script. No dashboards to check, no rules engine to maintain — just receipts in a Supabase table and alerts when the recruiter needs to step in.

Want more like this?

I publish practical AI automation, GenAI engineering, and faceless content workflows on YouTube every week.

Subscribe to bizflowai.io on YouTube — never miss a new tutorial.

Planning an AI automation project or need a second opinion on your architecture?

Connect with me on LinkedIn — Lazar Milicevic, GenAI Engineer & bizflowai.io Founder.

Visit bizflowai.io for our services, case studies, and AI consulting.

47 Take-Homes Reviewed for $18: The Hiring Step Nobody

lamingsrb — Mon, 17 Aug 2026 06:12:36 +0000

47 Take-Homes Reviewed for $18: The Hiring Step Nobody Automates

Your ATS vendor brags about sourcing, screening, and scheduling. None of those are where you lose candidates. You lose them in the 11-day silence after they submit the take-home, while your hiring manager stares at a folder of 47 zips and PDFs and picks the one on top.

I built an agent for a client's internship cohort that reviews take-homes for 39 cents each, ranks them into Notion, and pings the hiring manager once a day. Offer-to-accept dropped from 19 days to 8. Here's the exact build.

Why take-home review is the real bottleneck

Every mainstream recruiting tool — Greenhouse, Ashby, monday, Workable — automates the cheap steps and leaves the expensive one alone. Sourcing is cheap. Keyword-screening resumes is cheap. Sending a Calendly link is cheap. Reviewing a take-home requires judgment, so the vendors skip it, and it becomes an 11-day dead zone in the middle of your pipeline where the money leaks out.

The client's numbers, no rounding:

Stage	Before	After
Applicants per cohort	203	203
Take-home submissions	47	47
Days to open the folder	11	<1
Days to first reply	14	2
Manager time per candidate	22 min	4 min
Offer-to-accept cycle	19 days	8 days
Review cost (Claude)	—	$18.33 total

Three of their top five picks from the previous cohort had already accepted elsewhere by the time the manager sat down to review. That's not a sourcing problem. That's a silence problem, and silence is the one thing automation is actually good at killing.

The stack: n8n, Claude, Notion, Slack

Nothing exotic. The trigger is a Gmail label. An earlier step in the pipeline labels candidate submission emails as take-home-submitted. n8n polls that label every 5 minutes and fires the workflow.

The first node classifies the submission format, because candidates don't cooperate:

~60% send a GitHub repo link
~30% send a PDF or Google Doc
~10% send a Loom video

Each format needs a different extractor before you burn tokens on Claude:

// n8n Function node - route by submission type
const body = $input.first().json.text;
const githubMatch = body.match(/github\.com\/[\w-]+\/[\w-]+/);
const loomMatch = body.match(/loom\.com\/share\/[\w]+/);
const pdfAttachment = $input.first().json.attachments?.find(a => a.mimeType === 'application/pdf');

if (githubMatch) return [{ json: { type: 'github', url: `https://${githubMatch[0]}` }}];
if (loomMatch)   return [{ json: { type: 'loom',   url: `https://${loomMatch[0]}` }}];
if (pdfAttachment) return [{ json: { type: 'pdf', file: pdfAttachment }}];
return [{ json: { type: 'unknown', raw: body }}];

For GitHub, I call the API and pull three things: the README, the three largest source files, and the last 20 commits. Not the whole repo — you'll blow the context window and pay for boilerplate. For PDFs, I convert to markdown first (I use pdftotext piped through a cleanup script) because raw PDF parsing burns tokens and destroys structure. Loom links go through Whisper for a transcript before Claude ever sees them.

That preprocessing is the difference between 39 cents per submission and $2+.

The scoring rubric (this is where hand-waving fails)

Every AI hiring demo I've watched shows a vague prompt like "evaluate this candidate submission." That gives you vague scores. The rubric has to be sharp, and it has to force a written justification, or the model will just regress to 3-out-of-5 on everything.

Six criteria, scored 1-5, each with a mandatory one-sentence justification:

Problem understanding — did they solve the actual problem or a nearby one?
Technical execution — does it work, and is the approach reasonable?
Code quality — naming, structure, obvious code smells
Communication clarity — README, comments, PR description
Edge case handling — what happens on bad input, empty state, failure?
Scope discipline — did they overbuild or underbuild?

Max score: 30. Here's the actual system prompt (trimmed):

You are a skeptical senior engineer reviewing an internship take-home.
You have seen hundreds of these. You are NOT here to be encouraging.

Score the submission on six criteria, 1-5 each. For each score, write
one sentence justifying it with a specific reference to the code or doc.

Then write a "concern" field: exactly two sentences on the single biggest
issue a hiring manager should look at before an interview. If there is no
real concern, say so - do not invent one.

Finally, flag one of:
  - CLEAN
  - MINOR_FLAG (small issues, worth noting)
  - MAJOR_FLAG (undisclosed AI use, plagiarism, non-working code)

Return JSON only. No preamble.

The concern field is the whole reason this is usable. A hiring manager doesn't want a score — they want to know what to look at. Two sentences of "the auth handler swallows all exceptions silently and there are no tests around the token refresh path" is worth more than any number.

The output schema:

{
  "scores": {
    "problem_understanding": 4,
    "technical_execution": 3,
    "code_quality": 4,
    "communication_clarity": 5,
    "edge_case_handling": 2,
    "scope_discipline": 4
  },
  "total": 22,
  "concern": "Error handling in the main request handler is missing entirely and there is no test coverage for the retry logic. Both are red flags for a role that will touch production ingestion.",
  "flag": "MINOR_FLAG"
}

Notion card + the one Slack message that matters

Step three writes a Notion card. Title is <Candidate Name> — <Score>/30. Body has the six sub-scores, the concern paragraph, the flag, and a direct link back to the submission. Traffic-light status property:

Green — score >24 AND flag is CLEAN
Yellow — score 18-24 OR MINOR_FLAG
Red — score <18 OR MAJOR_FLAG

The Notion database has one saved view sorted by score descending, filtered to the current cohort. The hiring manager opens it once and sees a ranked list with greens at the top.

Step four is the daily Slack ping. Once at 9am, if there are new reviewed submissions in the last 24 hours, the workflow posts a single DM:

Take-home review, Wed Aug 12
3 green candidates ready to schedule
2 yellows to skim
1 red to skip
→ Open ranked view

That's it. No per-submission notifications. No email digest. One message, one link, actionable. Notification fatigue is what kills these systems in month two, and I'd rather ship one signal a day than ten.

Live run: 40 seconds, 34 cents

To show it working end-to-end I submitted a fake take-home to the client's test inbox — a small Node project on GitHub. The Gmail trigger fires within 5 minutes. n8n picks it up, hits the GitHub API, pulls the README and the three main source files. Claude runs the rubric prompt. Total wall-clock time: about 40 seconds. Token cost from the log: $0.34.

The Notion card comes back with a score of 22, yellow status, and the concern reads: "The main handler has no try/catch and there is no test coverage on the transform step. Ask them to walk through their error strategy in the interview."

That's a real concern I'd want a human to look at. It's not the model padding to seem thorough — it's pointing at two specific files and two specific gaps. That's the bar.

Across the full cohort:

47 submissions reviewed end-to-end
$18.33 total Claude spend ($0.39 average)
14 hours of manager time saved (22 min → 4 min per candidate)
19 → 8 day offer-to-accept cycle

Two of the top three hires in that cohort said the fast turnaround was why they took the offer over a competing one. That's the number that pays for everything.

Three things I got wrong on v1

Pass these on so you don't repeat them.

Auto-rejecting reds was a mistake. One candidate got flagged MAJOR_FLAG for undisclosed AI use — turned out they'd disclosed it in a section my parser skipped over. Reds now go to a human for a 30-second sanity check before any candidate-facing action fires. The cost of one false rejection on a top candidate is way higher than 30 seconds of manager time.
Ten criteria was too many. My first rubric had ten. Scores blurred together and the justifications got repetitive. Six is the sweet spot — enough coverage to be fair, few enough that each score carries weight.
Never skip the concern field. I ran an early version that only produced numeric scores. Managers ignored it. The one-paragraph concern is what makes the hiring manager trust the ranking and stop re-reading every submission themselves.

One more, quieter lesson: the model is a first reviewer, not a decision maker. Every candidate-facing action — reject, invite, offer — still goes through a human. The agent's job is to make sure that human is looking at the right five submissions on day 1, not the wrong 47 on day 11.

Why bizflowai.io helps with this

The take-home reviewer is one of about a dozen hiring-stage agents I've shipped for clients through bizflowai.io — panel schedulers, reference-call bots, offer-letter agents, ghost-prediction sentiment monitors. The pattern is always the same: find the step where a human is doing 20 minutes of judgment work on every candidate, extract the mechanical 80% of that judgment into a scored + flagged artifact, and leave the human to validate. It's not about replacing hiring managers. It's about making sure they're never the reason a good candidate walked.

Want more like this?

I publish practical AI automation, GenAI engineering, and faceless content workflows on YouTube every week.

Subscribe to bizflowai.io on YouTube — never miss a new tutorial.

Planning an AI automation project or need a second opinion on your architecture?

Connect with me on LinkedIn — Lazar Milicevic, GenAI Engineer & bizflowai.io Founder.

Visit bizflowai.io for our services, case studies, and AI consulting.

A Week as an AI Integration Consultant

lamingsrb — Thu, 13 Aug 2026 07:32:45 +0000

A Week as an AI Integration Consultant

Most weeks I don't write much new code. I read other people's systems, draw arrows on a whiteboard, and try to figure out which of the twelve places customer data lives is the one I should actually trust. That is the honest shape of AI integration work in a B2B company that has been shipping since 2014. The LLM is the easy part. The plumbing is the job.

Here are field notes from a recent week doing exactly this, plus the checklist I wish every founder had in hand before they hire anyone (me included) to bolt an LLM onto their stack.

Monday: Mapping the data flow before touching a prompt

The client wanted "an AI assistant that answers customer questions from our knowledge base and CRM." That is a sentence, not a specification. My first day is almost always the same: I map where data actually lives, who writes to it, and how stale it is by the time anyone reads it.

For this client the map ended up looking like this:

System	Role	Write frequency	Trust level
Salesforce	Accounts, opportunities	Sales reps, daily	High but partial
NetSuite	Invoices, entitlements	Nightly batch	High, delayed
Zendesk	Tickets, macros	Agents, real-time	High for current, weak for history
Confluence	Internal KB	Product team, weekly	Medium, drifty
A 2019 MySQL app	Product config per customer	Nightly cron	Source of truth but ugly

The MySQL app is the one nobody wanted to talk about. It is also the only place that knows which features a given customer is actually entitled to. If the assistant answers "yes, you have access to X" without reading that database, it will hallucinate entitlements and create a support fire.

Rule I now apply on day one: find the ugliest system in the stack. That is almost always the real source of truth. The pretty SaaS on top is a view, not a fact.

I don't touch a prompt until I have this map signed off. If the CTO can't tell me which system wins in a conflict between Salesforce and NetSuite on the same field, we are not ready to add an LLM. We are ready to have a meeting about data governance dressed up as an AI project.

Tuesday: Picking the glue (queues, webhooks, or agents)

By Tuesday I know the systems. Now I have to pick the glue. This is where most integrations quietly fail six months in, because someone reached for the wrong tool on day one and everything after that inherited the mistake.

My rough decision tree, after doing this on and off for a decade:

Synchronous HTTP call from the app to the LLM. Use when the user is waiting for the answer and latency budget is under two seconds. Chat UIs, autocomplete, inline summarization. Fails badly for anything that touches more than two backend systems.
Webhook + queue (SQS, EventBridge, or a Postgres-backed queue). Use when work is triggered by an event (ticket created, invoice posted, form submitted) and the user does not need the answer in the same request. This is where 70% of real B2B integrations live. It is also where the money is, because it maps directly to "hours saved per month."
Scheduled worker. Use for enrichment, backfills, nightly summaries, weekly digests. Cheap, boring, reliable. Underrated.
Agent loop (multi-step, tool-using). Use only when the task genuinely requires branching decisions that a deterministic workflow cannot express. Most tasks people call "agentic" are actually a switch statement with anxiety.

For this client I went with an EventBridge bus in front of Lambda workers, with Zendesk and Salesforce webhooks feeding in, and one small agent loop for a specific case (drafting a reply that needs to check entitlements, then pull the right KB passage, then decide whether to escalate). Everything else is a linear workflow, because linear workflows are debuggable at 2am and agent loops are not.

The brittle trap I avoid: a prompt sitting inside a Zapier or Make step, calling an LLM, and writing back to a CRM with no queue, no retries, no idempotency key, and no audit log. It demos beautifully. It falls over the first time the LLM returns malformed JSON, and nobody can tell you which record got corrupted.

Wednesday: The retrieval layer nobody scopes for

Wednesday is always retrieval. The client had 4,200 Confluence pages, 18,000 closed Zendesk tickets, and a product manual as a 380-page PDF. "Just point the AI at it" is a six-month project disguised as a sentence.

What I actually built this week:

Ingestion workers that pull from each source on its own cadence (Confluence via API on a 6-hour schedule, Zendesk via webhook on ticket close, PDF once with a manual re-ingest hook).
A normalization step that strips boilerplate, chunks by semantic boundary (not fixed token count), and attaches metadata: source system, last modified, author, customer tier visibility.
Storage in Postgres with pgvector for embeddings and a tsvector column for full text search on the same rows.
Hybrid retrieval at query time using Reciprocal Rank Fusion to merge vector and lexical results. This alone fixed a class of failures where pure vector search kept missing exact product names and error codes.
A rerank pass with a small cross-encoder for the top 30 candidates before handing 6-8 to the LLM.

The number that matters here: on this client's evaluation set of 140 real historical questions, pure vector retrieval got 61% top-5 recall. Hybrid + rerank got 89%. That 28 point gap is the difference between "the assistant is useful" and "the assistant is a liability." It is also the reason I no longer take retrieval seriously if it is just a vector database and vibes.

Cost-side note: keeping retrieval in Postgres (instead of a dedicated vector DB) saved this client roughly $600 to $900 per month at their volume, and removed one vendor from the security review. For a mid-market B2B company that matters more than the theoretical benchmarks.

Thursday: Guardrails, evals, and the boring stuff that keeps you employed

Thursday is the day I earn my rate. Anyone can wire an LLM to a CRM. Making it not embarrass the company is the actual skill.

Three things I insist on before anything goes to production:

1. A structured output contract, enforced twice. The prompt asks for a specific JSON schema. The response is validated against a Pydantic or Zod schema before it touches a downstream system. On failure, the worker retries with an error-corrective prompt, up to two times, then drops to a dead-letter queue with the full trace. No silent failures, no half-written records.

2. An eval set that lives in the repo. For this client, 140 questions with expected behaviors, run on every prompt change and every model change. Not just "does it answer correctly," but categorical: did it refuse when it should have refused, did it cite the right document, did it escalate the entitlement question. I run this in CI. A prompt change with a 3-point regression on the eval set does not ship.

3. An audit log, always. Every LLM call gets stored with input, output, model, cost, latency, retrieval context IDs, and the outcome downstream. This costs almost nothing in Postgres and it is the single most valuable artifact you will have when a customer emails asking why the assistant told them something wrong three weeks ago.

The kind of thing that goes into a worker looks roughly like this:

def handle_ticket_event(event):
    ticket = fetch_ticket(event["ticket_id"])
    context = retrieve(ticket.body, customer_id=ticket.customer_id)
    entitlements = get_entitlements(ticket.customer_id)  # from MySQL, not CRM

    draft = llm.generate(
        prompt=REPLY_PROMPT,
        context=context,
        entitlements=entitlements,
        schema=DraftReply,  # enforced
    )

    audit.log(event, context, draft)  # always, before any write

    if draft.confidence < 0.7 or draft.needs_human:
        assign_to_agent(ticket.id, draft=draft.text)
    else:
        post_internal_note(ticket.id, draft.text)  # never auto-send week 1

Note the last comment. Week one, the assistant never sends anything to a customer. It writes internal notes for agents to review. Week four, once the eval set and the audit log show it is behaving, we flip the switch on the low-risk categories. This is how you ship an LLM into a real business without a Slack channel full of angry executives on day two.

Friday: Handover and the parts I refuse to skip

Friday is documentation and handover. This is where independent consultants lose clients (by writing nothing) and where I keep them (by writing too much, honestly).

What I hand over on every engagement:

An architecture diagram that matches what actually got built, not the one from the proposal.
A runbook for the top 8 failure modes: LLM API down, retrieval empty, schema validation failing, webhook backlog, quota hit, downstream write failure, silent degradation, customer complaint triage.
The eval set, with instructions to add to it every time something goes wrong in production. The eval set is a living asset. It is arguably the most valuable artifact of the entire project.
Cost dashboards per workflow, so someone can answer "is this saving us money" without asking me.
A 30-day, 60-day, and 90-day review checklist.

Clients who run the 30-60-90 reviews get compounding value. Clients who don't tend to let the whole thing quietly rot within a year. I now write this into the contract.

The checklist I wish founders used before hiring anyone

If you are a founder or CTO thinking about hiring an AI integration consultant, run through this before you write the brief. It will save you and your consultant a painful week.

Name the outcome in hours or dollars. "Reduce first-response time on tier-2 tickets by 40%" or "eliminate 20 hours per week of manual invoice reconciliation." Not "add AI."
List every system that holds the data the LLM will need. Include the ugly ones. Especially the ugly ones.
Identify the source of truth for each conflicting field. If two systems disagree on customer status, which wins? Write it down.
Decide the failure mode. When the LLM is wrong, what happens? Does it write to a customer, or draft for review? Week one should always be draft for review.
Budget for retrieval and evals, not just prompts. If the proposal is 80% "prompt engineering" and 20% everything else, the ratio is backwards.
Ask who owns the audit log. If nobody, that's the first thing to fix.
Confirm you have 100+ real historical examples the assistant would have handled. If you don't, the first two weeks are collecting them, not building.
Agree on the review cadence. 30, 60, 90 days. In writing.
Ask the consultant what they will not do. If the answer is "anything," walk away.

What I'd do if I were starting fresh today

Start with the smallest workflow that touches real money or real customer time. One ticket type. One report. One approval step. Wire it end to end with a queue, a schema, an eval set, and an audit log. Ship it as an internal draft first, promote it to customer-facing only when the numbers say so. Then, and only then, look at the next workflow.

The teams that get AI integration wrong try to boil the ocean with a chat interface. The teams that get it right pick one boring workflow, instrument it obsessively, and let the ROI compound. Everything I have shipped that survived past a year followed the second pattern.

If you are wrestling with an integration like this and want a second pair of eyes, or you have already tried the prompt-in-a-Zap route and want to do it properly, get in touch at lazar-milicevic.com/#contact. More field notes from production systems live on the blog.

The Honest Numbers From My Last 5 AI Builds

lamingsrb — Thu, 13 Aug 2026 07:32:42 +0000

The Honest Numbers From My Last 5 AI Builds

Most AI automation case studies read like brochures. Big percentages, no denominators, no mention of what broke or what cost more than it saved. I keep a spreadsheet for every system I ship: token spend, infra cost, hours reclaimed, and whether the thing actually earned its keep after six months. Here is that spreadsheet, translated into English, across my last five builds. Two of them underperformed. I will tell you which and why.

The Five Builds At A Glance

Before the deep dive, the summary table. Numbers are monthly averages over the first six months in production, or the full life of the build if it was retired earlier. Dollar figures are USD. "Time saved" is measured against the pre-automation baseline the operator actually tracked, not a guess.

#	Build	Monthly infra + tokens	Time saved / mo	Est. $ value / mo	Verdict
1	Multi-agent content + SEO machine (ContentStudio)	~$180	45 h	~$3,600	Strong win
2	Serverless AWS + Zendesk SLA integration	~$40	18 h + first-ever SLA compliance	~$1,800 + risk removed	Strong win
3	Analytics migration + automated reporting	~$95	~14 h	$2,500, 5,000/mo saved on prior tooling	Strong win
4	Local-LLM RAG for internal docs (Ollama + pgvector)	~$65 (hardware amortized)	~6 h	~$480	Underperformed
5	Autonomous outbound qualifier (LLM + CRM)	~$210	~4 h	~$320	Killed at month 4

Two winners built around clear, repeated workflows. One infra win that paid for itself in software licenses. Two that did not earn their spot and taught me more than the wins did.

Build 1: The Multi-Agent Content Machine

This is BizFlowAI ContentStudio, the system that researches, writes, optimizes, and publishes across multiple sites without me in the loop. It has been running for over six months. The honest breakdown:

Monthly cost: roughly $180. That splits into ~$120 in LLM tokens (Claude for drafting and critique, a cheaper model for classification and metadata), ~$35 in serverless AWS (Lambda, EventBridge, S3, a small RDS instance), and ~$25 in third-party APIs for search data and rank tracking.

Time reclaimed: 45 hours per month against the manual baseline of researching, drafting, editing, formatting, publishing, and internal linking. I timed this over two months before automation. The number is not inflated.

Where the money actually goes: the critique agent. I run a generate-then-critique-then-revise loop, and the critique pass consumes roughly 35% of total tokens. I could cut it. I have not, because the 40% token reduction work I did earlier this year came from prompt compression and caching, not from removing the critique. Killing critique would save ~$40/mo and cost me an unknown amount in quality regressions. Not worth it.

What actually made it work: a self-learning loop that reads GSC data weekly, identifies which posts are gaining impressions, and biases the topic queue toward adjacent long-tails. That single feedback loop is why the system stopped producing vanity content and started producing content that gets impressions. Without it, this build would be in the "underperformed" pile.

Build 2: Serverless AWS + Zendesk For SLA Compliance

Smallest infra bill on the list, biggest strategic impact. The problem was simple: tickets were missing SLA because the routing logic lived in a human's head and that human went on vacation.

Monthly cost: ~$40. Lambda invocations, EventBridge rules, a tiny DynamoDB table for state, CloudWatch for observability. Zero LLM tokens. Yes, this counts as an "AI automation build" because part of the routing uses a classifier, but the classifier runs on cached embeddings and costs pennies.

Result: first-ever month of full SLA compliance for the team it served. That is the number the business cared about. The 18 hours per month of manual triage reclaimed was a side effect.

Lesson: the highest-ROI "AI" builds are often 90% boring event-driven plumbing and 10% intelligence. If you skip the plumbing and lean on the model to do everything, you get a demo, not a system. I wrote more about this in Agentic Workflows That Survive Real Inputs.

Build 3: Analytics Migration And Automated Reporting

This is the one that delivered EUR 30, 60k in annual savings (about $32k, 65k) by replacing a legacy analytics stack with a lighter, event-driven pipeline plus scheduled LLM-generated summaries for stakeholders.

Monthly cost: ~$95 all-in. Warehouse queries, a small orchestrator, and roughly $30 in tokens for the weekly and monthly narrative reports.

Where the savings came from: license consolidation, not the AI. The LLM narrative layer was the wedge that made stakeholders comfortable retiring the old tool ("but who will write the exec summary?"). Once that objection died, the migration paid for itself in the first quarter.

Honest caveat: if you subtract the license savings, the pure AI-generated-reporting value is maybe $400, 600/mo in analyst time. Real, but not the headline number. I try to keep those two things separate when I report to a client. Conflating them is how people lose trust in AI ROI claims.

Build 4: Local-LLM RAG That Underperformed

This one hurt because I liked the architecture. Ollama running a mid-size open model, pgvector for embeddings, hybrid search with reciprocal rank fusion, a clean retrieval API. Technically sound. I wrote the hybrid search post based on this exact build.

Monthly cost: ~$65 amortized (hardware plus power plus a small managed Postgres).

Time saved: only about 6 hours per month. Estimated value ~$480. Barely covers the cost when you include the maintenance I do on it.

Why it underperformed:

Query volume was too low. The internal team it served asked the system maybe 40, 60 questions a week. RAG shines at scale. At low volume, people just Slack a colleague.
The corpus was not the bottleneck. People were not blocked because they could not find docs. They were blocked because the docs did not exist for the questions they were asking. RAG cannot retrieve what was never written.
Trust cold-start. New users tried it twice, got one great answer and one mediocre one, and stopped. The mediocre answer was technically correct but poorly phrased. Retrieval quality was not the problem. Presentation was.

What I would do differently: I would validate with a two-week Slack-bot-with-a-human-in-the-loop before building the full retrieval stack. If the team is not asking questions to a human proxy, they will not ask a bot either. That is a $5 test that would have saved me a $65/mo permanent bill.

Build 5: The Outbound Qualifier I Killed At Month 4

An autonomous agent that ingested inbound leads, enriched them, ran a qualification prompt, and updated the CRM with a score plus a short reasoning paragraph. Sounds useful. Was not.

Monthly cost: ~$210. Higher than expected because enrichment API calls added up fast and the qualification prompt was long (roughly 3.5k input tokens per lead).

Time saved: ~4 hours per month. Value ~$320. Net negative.

Why it failed:

The qualification signal it produced correlated with the sales rep's own gut about 75% of the time. On the 25% where they disagreed, the rep won 8 out of 10. The model was confidently wrong on edge cases that actually mattered.
Volume was too low to justify the ceremony. Roughly 60, 90 leads a month. A human can score that in an hour with a spreadsheet.
The "reasoning paragraph" the model produced became a liability. Reps started trusting it, and when it was wrong, they had already sent an email based on a bad framing.

I killed it at month 4. Refactored the useful parts (the enrichment layer and a much simpler rules-based scorer) into a 30-line script that costs $6/mo and does the job better. That script is still running.

Lesson: if a human can do the task in under an hour a month, do not build an agent for it. The floor cost of any agentic system, in tokens, tokens for retries, monitoring, evals, and your own attention, is higher than people admit.

What The Numbers Actually Say

Three patterns show up across all five:

1. Token cost is rarely the problem. Fit is. The build with the highest token spend (ContentStudio at $120/mo) is the biggest winner. The build with lower token spend (RAG at ~$20/mo in embeddings) underperformed. Fit to a real, repeated, high-volume workflow matters more than efficient prompting. Optimize prompts after you have proven fit, not before.

2. Boring infra beats clever prompts. The Zendesk SLA build has almost no LLM in it and produced the clearest strategic win. If you are pitching an AI project, ask yourself first whether the pain is actually an event routing problem, a data problem, or a workflow problem. If it is, solve that first and add intelligence where it genuinely helps.

3. The kill decision is the skill. Killing build 5 at month 4 was the highest-ROI decision in this whole spreadsheet. It stopped a $210/mo bleed, freed my attention, and produced a better $6/mo replacement. Most teams keep zombie AI systems alive because someone got promoted for launching them. Do not do that.

What I'd Do If I Were Starting Today

Track the baseline before you build. Two weeks of honest time tracking. If you cannot show me the hours you spent on the task pre-automation, we cannot measure ROI post-automation.
Set a kill threshold in writing. Mine is: if a build does not clear 3x its monthly cost in reclaimed time or hard savings by month 3, it goes on a "fix or kill" list. Month 4 is decision time.
Instrument tokens per outcome, not tokens per call. "This agent costs $0.04 per completed useful task" is a number a CFO can price. "The model costs $3 per million tokens" is not.
Prefer event-driven serverless for anything that runs unattended. Scale-to-zero economics change what is worth building. A build that costs $6/mo idle can sit there proving itself for a year without pressure.
Publish the losers. If you only ever hear about the wins, you are being sold to.

Close

These are the actual numbers from actual systems, not slide decks. If you are weighing whether an AI automation build is worth it inside your own company, I am happy to talk through the shape of it honestly, including telling you when the answer is "do not build this." Reach out at lazar-milicevic.com/#contact, or read more on the blog if you want the deeper technical write-ups behind any of these builds.

112 Applicants, 47 Filtered on Salary First: 71¢ Total

lamingsrb — Thu, 13 Aug 2026 06:12:57 +0000

47 of 112 Applicants Filtered on Salary Before Anyone Read a Resume

You post one role. Twenty applications land. You book eight calls. Four of them collapse the moment you say the number. That's an hour of your week gone because the salary conversation happened at minute twenty-eight instead of minute zero — and the fix is a 40-line node most hiring tutorials skip on purpose.

Why salary has to be the first filter, not the last

The default hiring stack scores skills first and asks about money last. That's backwards for small teams. Skill match without comp alignment produces a prettier list of the same wasted calls — you still end the week with candidates who want $140K on a $95K band, or who ghost the moment you name the number.

The bottleneck in solo and small-team hiring isn't finding qualified people. It's finding qualified people whose number fits your number. Every other filter — years of experience, stack fit, culture, portfolio depth — is downstream of that one gate. If comp doesn't overlap, everything else is theater.

Most published n8n and Zapier "AI recruiter" flows optimize for the demo, not the outcome. They read the CV, score against a JD, dump a Kanban. The tools monday and Ashby ship look great in a screenshot. None of them run a comp-first router before scheduling. That's the piece I'm going to show you.

The bad path: intake → resume score → shortlist → intro call → salary reveal → collapse
The right path: intake → salary extraction → three-way router → screening queue → intro call

The stack, and why it's boring on purpose

Boring stacks ship. Here's the wiring I use for clients running burst hires:

Gmail — intake. One label: applications-inbound.
n8n — orchestrator. Self-hosted, polls every 2 minutes.
Claude Haiku — extraction model. Cheap, fast, structured output.
Notion — talent database. Four custom fields (below).
Telegram — human-in-the-loop for the ambiguous middle bucket.

Runs on a small home server. No per-seat SaaS tax, no vendor lock-in, no waiting on a product team to ship the filter you actually need. Total infra cost for the client run below: the electricity to keep n8n up, plus 71¢ in Anthropic tokens across 112 candidates.

One thing that matters more than it sounds: PDFs get converted to plain markdown before the model sees them. Vision on a CV is a waste — you're paying vision-tier tokens to read Arial 11pt. A pdf-parse node or pdftotext shell step drops both latency and cost significantly versus shoving the raw PDF at a multimodal model.

# in an n8n Execute Command node
pdftotext -layout "{{$binary.data.fileName}}" - | head -c 12000

Cap the character count. Nobody's comp signal is on page 7.

The 40-line salary extraction node

The prompt is narrow on purpose. Haiku isn't asked to score the candidate, summarize them, or judge fit. It's asked one thing: find any signal about compensation expectations.

That means:

Explicit ranges — "$85K–$95K", "looking for 110", "current base is 78"
Soft signals — "competitive", "open to discussion", "market rate"
Location inference — SF Bay Area applying to a Kansas City mid-market role is a signal
Employer-based inference — "currently at [named FAANG]" narrows the expected floor

The model returns three fields: extracted range, confidence 0–1, and the exact source snippet it pulled the signal from. That snippet is the whole ballgame — it's your audit trail when a hiring manager asks why a candidate got a polite auto-reply.

// n8n Function node — salary extractor (Anthropic call)
const prompt = `You extract ONLY compensation signals from job applications.

Return JSON:
{
  "comp_low_usd": number | null,
  "comp_high_usd": number | null,
  "confidence": number,        // 0.0 - 1.0
  "signal_type": "explicit" | "soft" | "inferred" | "none",
  "source_snippet": string     // exact text, max 240 chars
}

Rules:
- "competitive" / "open" / "market" => soft, confidence <= 0.5
- Explicit range in cover letter => explicit, confidence >= 0.9
- Location + role level inference only => inferred, confidence <= 0.6
- No signal at all => "none", confidence 1.0, snippet ""
- NEVER guess a number without textual grounding.

APPLICATION:
"""
${$json.email_body}

--- CV ---
${$json.cv_markdown}
"""`;

const res = await this.helpers.httpRequest({
  method: 'POST',
  url: 'https://api.anthropic.com/v1/messages',
  headers: {
    'x-api-key': $env.ANTHROPIC_KEY,
    'anthropic-version': '2023-06-01',
    'content-type': 'application/json'
  },
  body: {
    model: 'claude-haiku-4-5',
    max_tokens: 400,
    messages: [{ role: 'user', content: prompt }]
  },
  json: true
});

return [{ json: JSON.parse(res.content[0].text) }];

That's the whole extraction. ~40 lines with the wrapper. Average tokens per candidate in the real run: ~1,400 in, ~120 out. Per-candidate cost sits well under a cent.

The three-way router (this is where most people ship the wrong thing)

Binary routers — match or reject — quietly send great candidates polite rejection emails because a soft signal got misread. The fix is a third bucket for ambiguity.

Route logic in plain English:

Confidence ≥ 0.75 AND range overlaps band → Notion screening queue
Confidence ≥ 0.75 AND range clearly outside band → polite auto-reply naming the band, inviting future contact
Confidence < 0.75 OR signal_type == "none" → Telegram alert to hiring manager with name, CV link, snippet, approve/reject buttons

// n8n IF chain
const { comp_low_usd, comp_high_usd, confidence } = $json;
const BAND_LOW = 85000;
const BAND_HIGH = 105000;

const overlaps =
  comp_low_usd != null &&
  comp_high_usd != null &&
  comp_low_usd <= BAND_HIGH &&
  comp_high_usd >= BAND_LOW;

if (confidence < 0.75 || $json.signal_type === 'none') {
  return [{ json: { route: 'human_review' } }];
}
return [{ json: { route: overlaps ? 'advance' : 'auto_reject' } }];

The auto-reply is honest, not corporate. "Thanks for applying. Our budget for this role tops out at $105K. If that shifts for you, let us know and we'll pick it back up." That message has produced two hires in the last year from candidates who came back three months later with adjusted expectations. A generic "we'll keep your resume on file" burns that bridge.

The Telegram alert for the ambiguous bucket takes the hiring manager ~10 seconds per tap. Phone, thumb, done. No laptop, no inbox archaeology.

The Notion schema that makes this defensible

Your talent database needs four fields you probably don't have today:

Field	Type	Why it matters
`comp_low_usd`	Number	Filter, sort, band analytics later
`comp_high_usd`	Number	Same
`extraction_confidence`	Number (0–1)	Sort ambiguous records for weekly review
`source_snippet`	Text	Non-negotiable — the receipt

The source snippet is the difference between a black-box filter you can't defend and a system your hiring manager trusts. When someone asks why a candidate got auto-declined, you paste the exact sentence from their cover letter. That single field killed every "the AI is rejecting people unfairly" conversation on the client engagement below.

Add a fifth field — routing_decision — with the value the router produced. Now you have a queryable log: how many auto-rejects last month, how many false positives caught in the Telegram bucket, what the model's confidence distribution looks like. That's how you tune the 0.75 threshold up or down based on actual error rate, not vibes.

The real numbers from one client run

Three-week window, one senior individual-contributor role, US remote, $85K–$105K band.

Metric	Value
Total applicants through the label	112
Auto-filtered on comp mismatch (≥ 0.75 conf)	47
Flagged to Telegram for human review	12
Advanced to screening queue	53
Total Anthropic token spend	$0.71
Avg time from email arrival to routing decision	~90 seconds
Intro calls that collapsed on salary	0

Of the 47 auto-filtered: 31 were explicitly asking for $130K+, 9 were in high-cost metros with inferred floors above $115K, 7 named a current employer where the floor was public. Of the 12 flagged for review: 8 got approved, 4 got rejected — the model correctly abstained.

The hiring manager stopped taking calls with people whose number was going to blow up minute twenty-eight. That's it. That's the whole win. The system didn't hire anyone — it just stopped wasting the calendar on candidates the math already ruled out.

For comparison: monday's recruiting agent, Ashby's AI screening, and most of the "n8n recruiter" YouTube builds do resume scoring first and comp negotiation never. They optimize the wrong variable. A comp-first router in front of any of them would make them meaningfully better.

Where bizflowai.io helps with this

This is the class of workflow I ship for clients weekly at bizflowai.io — the boring, high-leverage filters that sit in front of the expensive human step. Comp-first hiring routers, invoice triage that flags mismatches before they hit AP, lead qualifiers that reject on budget signal before booking discovery calls. The pattern is always the same: one narrow extraction model, a three-way router with a human-in-the-loop bucket for the ambiguous middle, and an audit trail so the decision is defensible. No SaaS seat tax, no vendor lock-in, runs on infra you control.

What to steal from this

Put the comp filter before the resume filter, not after
Convert PDFs to markdown before the model reads them
Return a source_snippet on every extraction — it's your audit trail
Never ship a binary router; the third bucket for ambiguity is where you stop breaking things
Use the auto-reject reply to name the band honestly — you'll get candidates back later
Log the router decision to Notion; tune your confidence threshold on real error rate

The whole thing is ~200 lines of n8n JSON, one Anthropic key, and a Notion database. Total cost to run 112 candidates: 71 cents plus the electricity. Total calls saved: probably 6–8. Do the math on what your hour is worth.

Want more like this?

I publish practical AI automation, GenAI engineering, and faceless content workflows on YouTube every week.

Subscribe to bizflowai.io on YouTube — never miss a new tutorial.

Planning an AI automation project or need a second opinion on your architecture?

Connect with me on LinkedIn — Lazar Milicevic, GenAI Engineer & bizflowai.io Founder.

Visit bizflowai.io for our services, case studies, and AI consulting.

Claude Tag's 'Multi-Day' Mode Is A $4K Billing Trap

lamingsrb — Thu, 13 Aug 2026 06:12:53 +0000

Claude Tag's "Persistence Over Days" Is A Scheduled Loop With Your Credit Card Attached

Anthropic shipped Claude Tag with a phrase that made it go viral: persistence over days. Tag Claude in Slack, walk away, get pinged Monday. Sounds like a teammate. It's a cron job with a wallet. I've built three async agent systems for clients this year, and the exit condition on every viral demo is the same thing: an LLM deciding, in a loop, when the job is done. Here's what breaks and what to build instead.

What "persistence over days" actually is under the hood

Persistence over days is three components stitched together: a state store holding conversation and tool history, a scheduler that re-invokes the agent on cron or event triggers, and a tool loop where the model calls a function, reads the result, calls another function, and keeps going until it decides it's done. Every re-invocation replays context. Every tool call costs tokens. The exit condition is a probability distribution.

That last part is the one that should scare you. You are trusting a language model to know when a job is finished, in a loop, with your billing key attached. There is no while (task.done) you can inspect in a debugger. The model emits a stop token when its own weights say the work looks complete. Sometimes that's after 3 tool calls. Sometimes 47. Sometimes it re-reads the same doc four times because retrieval returned near-duplicates.

Three things fail here in production:

Context replay tax. Each wake-up ships the accumulated conversation back into the prompt. A 40K-token thread that runs six times over a weekend is 240K tokens billed, minimum, before any new work happens.
Tool loop drift. The model tries an API, gets a 429, "reasons" about retry, tries again, calls a different tool, comes back. Real logs from my systems show 8-12x more tool calls than a human would use for the same task.
Silent completion. The model returns a final message that reads plausible. Nobody verifies whether the work was actually done correctly.

The real math: $0.02 per task vs $2-$15 per task

Here are numbers from a system I run for a US-based SMB client, roughly 40 async tasks a day across email triage, CRM updates, and lead enrichment.

Metric	Capped async task (my pattern)	Unbounded persistent run
Avg cost per task	$0.02	$2 – $15
Avg latency	8 seconds	Minutes to hours
Token ceiling	5K – 50K per run	None visible
Tool-call ceiling	8 – 20 per run	None
Daily spend (40 tasks)	~$0.80	$80 – $600
Monthly spend (single user)	~$24	$2,400 – $18,000

Now put that inside a Slack workspace with 30 people casually tagging the bot into threads. Nobody is watching individual runs. Nobody knows the difference between a tag that costs 2 cents and one that spins for six hours. You land the invoice, and it's $4,000 you didn't budget for.

The bill is the loud failure. The quiet one is worse.

The quiet-wrongness problem is the real liability

The agent posts a confident reply in a Slack thread. Someone on your team reads it, trusts it, and acts on it. They send the customer email. They approve the refund. They update the CRM record. There is no audit trail pointing to a checkpoint because there was no checkpoint. The human is now the one who executed the wrong action, and they did it because the bot sounded sure.

This is the pattern I see fail most often in async agent deployments:

Bot replies confidently in a channel where 8 people are watching.
One person reads it while doing something else, treats it as verified work.
Action gets executed downstream in another system.
Two weeks later a customer complains, and there is no log tying the wrong action back to the model version, the prompt, or the tool calls that produced it.

Anthropic's demo skipped who reviews the output before it becomes a message the whole team acts on. That question has exactly one right answer in a real business: a named human, reviewing in the interface they already live in.

The four checkpoints I bake into every async agent

Before the first token is spent on any async task I ship, four hard limits are already in the code. These are non-negotiable, and they are what separates a working system from a launch video.

1. Max-token cap per run. Fails loud, not silent. Between 5,000 and 50,000 tokens depending on the job. The wrapper aborts and pings me if we hit it.

MAX_TOKENS_PER_RUN = 15_000
MAX_TOOL_CALLS = 10

def run_agent(task):
    total_tokens = 0
    tool_calls = 0
    while not task.done:
        if total_tokens >= MAX_TOKENS_PER_RUN:
            raise BudgetExceeded(f"Hit {MAX_TOKENS_PER_RUN} on {task.id}")
        if tool_calls >= MAX_TOOL_CALLS:
            raise LoopExceeded(f"Hit {MAX_TOOL_CALLS} on {task.id}")
        step = model.step(task.state)
        total_tokens += step.usage.total_tokens
        if step.tool_call:
            tool_calls += 1
            step.tool_call.execute()
        task.state = step.state

2. Max-loop counter. The agent gets 8, 12, or 20 tool calls, forced stop, forced report. If it hasn't finished, that's information — a human decides whether to extend, not the model.

3. Mandatory human checkpoint before any write action. Sending an email, updating a CRM record, pushing an invoice, posting to a wide channel. Everything with side effects pauses, pings a human on Telegram or WhatsApp, waits for a thumbs up. The agent proposes. The human approves. Only then does anything hit production.

4. Per-run and per-day spend ceiling. Checkpoint every 60 seconds of runtime or every 10 cents of spend, whichever comes first. If the bot hits $5 in a day for a single user, it stops and asks. Not silently. Loud.

Every run also gets a structured log: model version, prompt hash, tool calls made, human who approved each write action, final token count. If something goes wrong two weeks later, I can reconstruct exactly what happened.

What to actually copy from Claude Tag (and what to skip)

The tag-and-forget UX is genuinely good. Users don't want another dashboard. They want to mention the bot in the tool they already use and get pinged when work is done. That interaction pattern is maybe 50 lines of code on top of the Slack, Telegram, or WhatsApp API.

Copy: the mention-and-notify pattern, threaded replies for context isolation, one clear "done" ping.
Copy: async job queue so the user isn't blocked waiting.
Skip: unbounded runtime. Every job has a hard wall-clock and token ceiling.
Skip: the model deciding when it's done on write actions. A human decides for anything that touches money, customer data, or an outbound message.
Skip: trusting the reply text as verified work. Every action that matters gets a checkpoint ping to a named human before execution.

The hard part is not the UX. It's the guardrail layer underneath. Async, yes. Persistent across days with no checkpoint, no.

Common mistakes I see teams make on their first async agent

No spend alarm at the API-key level. Set a hard monthly limit in the Anthropic console and a daily soft alert to Slack or email. Every provider gives you this. Use it.
One shared API key across all agents. You cannot attribute cost. Give every agent its own key, tag every request with a user_id and task_id in metadata.
Logging only the final response. Log every tool call, every token count, every model version. If you can't reconstruct a run, you can't debug it.
Human approval by email. Approvals need to happen where the person already checks 20 times a day. For most SMB operators, that's Telegram or WhatsApp, not another dashboard.
Treating the agent's reply as an audit trail. The reply is output. The log is truth. Never confuse them.

Where bizflowai.io fits

At bizflowai.io we build these async agents for solopreneurs and small teams with the four checkpoints baked in from day one. Every deployment ships with token caps, loop limits, human approval routing to Telegram or WhatsApp, and structured logging that lets the operator see what was spent, what was decided, and who approved each write action. Clients running our systems handle 40+ async tasks a day for under a dollar, with zero silent failures reaching production.

The prediction

Six months from now, the Twitter threads will show up. The $4K bill screenshots. The Slack channels full of confidently wrong replies that got acted on. Teams quietly turning the feature off. Async agents are the right direction. Unbounded persistent agents without checkpoint gates are a liability the vendor has offloaded onto the customer. If you're a founder, your job is to put the gates back in before you turn it on.

Want more like this?

I publish practical AI automation, GenAI engineering, and faceless content workflows on YouTube every week.

Subscribe to bizflowai.io on YouTube — never miss a new tutorial.

Planning an AI automation project or need a second opinion on your architecture?

Connect with me on LinkedIn — Lazar Milicevic, GenAI Engineer & bizflowai.io Founder.

Visit bizflowai.io for our services, case studies, and AI consulting.

4 Files Into One Human-Reviewed Interview Brief

lamingsrb — Mon, 10 Aug 2026 06:12:29 +0000

4 Files, One Interview Brief: The Bot That Preps the Human

Your hiring manager opens the CV in one tab, hunts through Gmail for the application thread in another, digs up the role scorecard in a shared drive, and joins the call three minutes late still not sure what to ask. That is 15 minutes of tab-switching before a 30-minute interview. This post walks through the exact bot I build for clients that turns those four scattered inputs into a single evidence-linked briefing packet — without letting the model touch a hiring decision.

The one rule that keeps this safe: the bot preps humans, it does not judge them

This bot prepares the interviewer. It does not score, rank, reject, advance, or message the candidate. That boundary is the entire reason the workflow is worth building first. Every automation vendor demo I have seen starts at the riskiest step — resume scoring — which is exactly where bias, EEOC exposure, and bad candidate experience live. Flip the order. Automate the boring information-gathering that your hiring manager is doing badly at 8:57 AM, and leave the judgment where it belongs.

Practically, that means the bot has three hard "no" rules baked into the system prompt and the workflow logic itself:

No numeric fit score, no "recommend hire / no hire" language, no ranked comparison across candidates.
No candidate-facing output. The bot writes to the interviewer only.
No stage changes in the ATS. If you use Greenhouse, Ashby, or a spreadsheet, the bot reads. A human writes.

If you skip this framing, the same code becomes a compliance problem the moment a rejected candidate asks how the decision was made.

The 4 inputs, and the boring identifier problem that breaks most builds

The four inputs are: calendar event, candidate CV, application form response, and role scorecard. The one people skip is the scorecard, and without it the model produces a fluent summary that has nothing to do with what you are hiring for.

Keep the scorecard to five fields so a hiring manager will actually maintain it:

Field	Example (customer support role)
Role title	Senior Support Specialist, SaaS
Must-have	Fluent written English, 2+ yrs ticketing (Zendesk/Intercom), B2B SaaS exposure
Nice-to-have	SQL basics, technical writing samples
Concerns to verify	US timezone overlap, availability start date, examples of angry-customer handling
Interview areas	Written comms sample, live triage exercise, escalation judgment

The second boring problem: one clean identifier per candidate. Email is the only reliable one. Names collide, LinkedIn URLs get edited, phone numbers are missing half the time. If your calendar event only has a name, matching becomes a guessing game the moment you interview two people named David Chen in the same quarter.

The fix is a title convention, not code:

Interview — Jane Doe — jane.doe@gmail.com — Support Specialist

Or put the email in the event description field. Not glamorous. But every reliable automation I have shipped starts with a boring, consistent input format.

The trigger, the extractor, and the "do not guess" rule

The trigger is a scheduled job that runs every 10 minutes and looks for calendar events with an Interview label starting in the next 30-60 minutes. When it finds one, it pulls: candidate email, candidate name, interviewer name, role title, event start time, meeting link.

If any critical field is missing, the bot does not guess. It sends the interviewer a short exception notice and stops.

# pseudo-code, n8n / Make / Python all map to this shape
def handle_event(event):
    ctx = extract_calendar_fields(event)
    missing = [k for k in ("candidate_email", "role_title", "interviewer") if not ctx.get(k)]
    if missing:
        notify_interviewer(
            to=ctx.get("interviewer_email"),
            subject=f"Briefing skipped: {event['summary']}",
            body=f"Missing fields: {', '.join(missing)}. "
                 f"Fix the event and I will regenerate on next run.\n{event['htmlLink']}"
        )
        return
    build_briefing(ctx)

Silent guessing is how automations create embarrassing failures — the wrong candidate summary sent to the wrong interviewer, or a briefing about "David Chen" that mixes two people's CVs. An exception notice takes 20 seconds for a human to fix. A hallucinated brief takes a week of trust to rebuild.

Fields the trigger must always confirm before proceeding

candidate_email — the join key for every downstream lookup
role_title — required to load the correct scorecard
interviewer — required to send the brief to the right person
event_start — required to schedule delivery 30 min before

Gmail search, CV extraction, and keeping the payload small

Once the trigger has the candidate email, the bot searches Gmail with a narrow query. Broad searches ("everything from this person, ever") are how you end up with a 40,000-token context window full of scheduling back-and-forth.

from:jane.doe@gmail.com OR to:jane.doe@gmail.com
label:Applications OR label:Recruiting OR label:Interviews
newer_than:120d
has:attachment OR subject:(application OR interview OR resume OR cv)

Then apply three filters:

Newest CV wins. If the candidate sent three versions, use the most recent. List older files as source links, do not blend them into the extracted text.
Extract with page references. For PDFs, keep page_number metadata alongside every text chunk. pdfplumber or pypdf both work. This is what lets the final brief cite "CV page 2" instead of a vague summary.
Pull only the fields the interviewer needs from the application form. Work history, key open-ended answers, portfolio links, stated availability. Skip the "how did you hear about us" field.

import pdfplumber

def extract_cv(path):
    pages = []
    with pdfplumber.open(path) as pdf:
        for i, page in enumerate(pdf.pages, start=1):
            text = page.extract_text() or ""
            pages.append({"page": i, "text": text.strip()})
    return pages  # keep structure, do not join into one blob

The payload going into the model should be small and labeled. For a typical candidate this is roughly:

Source	Size
Calendar event	~200 tokens
CV (extracted, page-labeled)	1,500-3,000 tokens
Application form (filtered)	400-800 tokens
Selected email threads (max 3)	500-1,500 tokens
Role scorecard	200-400 tokens
Total input	~3,000-6,000 tokens

At current Claude Sonnet or GPT-4-class pricing, one briefing runs about $0.03-$0.08. Dumping every email would push that past $0.50 per brief and would make the citations unverifiable.

The prompt: labels, citations, and no opinions

Every input chunk gets a source label before it hits the model. This is the single change that separates a useful brief from an AI-generated opinion.

[SOURCE: cv_page_2] "Led a support team of 4 at Acme SaaS, 2022-2024..."
[SOURCE: application_q3_dated_2026-03-04] "Available to start within 3 weeks..."
[SOURCE: email_thread_2026-03-06] "Confirmed she is US Pacific time..."
[SOURCE: scorecard_v1.3_must_have] "US timezone overlap required"

The system prompt then enforces the boundary:

You are preparing a hiring manager for an interview. You do NOT
evaluate the candidate. You do NOT recommend hire/no-hire. You do
NOT produce a score.

For every claim you make, cite the source label in brackets, e.g.
[cv_page_2]. If a scorecard requirement has no matching evidence in
the sources, say "no evidence found in provided sources" — do not
infer, do not fill in from general knowledge.

Output sections (in this order):
1. Snapshot (3 bullets, facts only)
2. Scorecard coverage (each must-have + nice-to-have, with evidence
   or "no evidence found")
3. Concerns to verify (from scorecard, phrased as questions)
4. Suggested questions (3-5, grounded in gaps or claims worth probing)
5. Source index (list of files/links used)

That "no evidence found" clause is the anti-hallucination lever. Without it, the model will happily write "Jane has strong SQL skills" because SQL is on the scorecard, even though nothing in her CV mentions it.

Delivery, review, and what a good brief looks like at 8:30 AM

The brief lands in the interviewer's inbox (or Slack DM) 30 minutes before the call, with the meeting link at the top and the source index at the bottom. Every claim has a bracketed citation the interviewer can click.

A real one looks roughly like this:

INTERVIEW BRIEFING — Jane Doe — Senior Support Specialist
Interview: 2026-04-15 09:00 PT with Marcus | [Join call]

SNAPSHOT
• 2 yrs support lead at Acme SaaS (B2B) [cv_page_2]
• Available in 3 weeks, US Pacific [application_q3, email_2026-03-06]
• Portfolio: writing samples + one incident postmortem [application_q7]

SCORECARD COVERAGE
✓ Ticketing 2+ yrs — Zendesk at Acme [cv_page_2]
✓ B2B SaaS exposure — Acme is B2B [cv_page_1]
✓ US timezone — confirmed Pacific [email_2026-03-06]
? Angry-customer example — no evidence found in provided sources

CONCERNS TO VERIFY
• Ask for a specific escalation she handled end-to-end
• Confirm start date given current notice period

SUGGESTED QUESTIONS
1. Walk me through the incident in your postmortem — what did you own?
2. Tell me about a ticket you escalated. What was the trigger?
[...]

The interviewer skims for 3 minutes, spots the "no evidence found" gap, and knows exactly what to probe. That is the whole product.

What breaks in the first two weeks of running this

Calendar events without the email in title or description — fix the convention, do not patch the code.
Gmail label drift ("Recruiting-2026" vs "Recruiting") — normalize labels once, monthly.
CVs sent as image-only PDFs — add an OCR fallback (Tesseract or a vision model) and flag the brief as "OCR used, verify accuracy."
Interviewer forwards the brief to the candidate by accident — put a visible INTERNAL — DO NOT FORWARD header on every brief.

Why bizflowai.io helps with this

This is exactly the class of workflow I ship for small teams at bizflowai.io — the boring, high-leverage pre-work that a human still owns the decision on. For hiring specifically, that means the calendar-triggered briefing bot above, plus adjacent pieces like a scorecard template your managers will actually fill in, exception routing when inputs are missing, and the source-labeled prompt structure that keeps the model from inventing evidence. Nothing in the stack scores or rejects candidates. Every decision stays with your hiring manager, and they walk into the call already prepped.

Want more like this?

I publish practical AI automation, GenAI engineering, and faceless content workflows on YouTube every week.

Subscribe to bizflowai.io on YouTube — never miss a new tutorial.

Planning an AI automation project or need a second opinion on your architecture?

Connect with me on LinkedIn — Lazar Milicevic, GenAI Engineer & bizflowai.io Founder.

Visit bizflowai.io for our services, case studies, and AI consulting.

218 of 312 Applicants Rejected Before a Human Looked

lamingsrb — Mon, 10 Aug 2026 06:12:26 +0000

218 of 312 Applicants Rejected Before a Human Looked

Every recruiting AI demo shows the same three things: source, screen, schedule. Nobody shows you the tier that actually saves the hours — the guilt-driven manual review of obvious no's. Here's the exact n8n flow that killed 218 of 312 applications on one role before a recruiter wasted a minute.

The 9-hour leak nobody automates

One role. Mid-level ops position. 312 applications in 11 days. The recruiter I built this for was spending roughly nine hours per week opening CVs, reading the first paragraph, and closing them again because the candidate was in the wrong country, wanted double the salary band, or didn't hold the license the role legally requires. Zero hires from that pile.

That's the actual leak. And it's the one nobody automates, because every founder-influencer is busy building outreach bots for the top of the funnel where it looks impressive on a demo.

We flipped it. Automated the rejection tier first. Here are the real numbers after 11 days on one role:

Metric	Before	After
Applications	312	312
Auto-rejected with reason code	0	218
Promoted to human review	312	94
Recruiter time	~9 hrs/week	~90 min/week
Token cost (Claude Haiku)	$0	$0.41
False negatives (month 1)	n/a	2 out of ~600

That's the whole pitch. Now the build.

Step 1: Hard-fail criteria, not scoring models

This is where most teams ruin the pipeline before they start. Do not build a scoring model. Start with hard-fail criteria — binary gates that a person could defend in writing to a candidate, a lawyer, or a labor board.

For this ops role, we encoded six:

Work authorization for the country of hire
Physical location within commuting distance of the office
Minimum years of relevant experience (role-specific)
Salary expectation inside the posted band
A specific professional license required by the role
Language fluency at working level

No culture fit. No "communication skills." No vibes. Soft signals belong later, after a human is already in the loop.

The reason this matters is legal and operational. Soft scoring produces disputes. Hard criteria produce receipts. When a candidate emails back asking why they were rejected, the recruiter points at one line: "You indicated Berlin as your base. The role requires on-site presence in Lisbon." That reply writes itself.

In the US, the EEOC's guidance on AI in employment decisions is clear that automated tools can create disparate impact liability under Title VII. Binary, role-relevant criteria with evidence snippets are defensible. Opaque 1-10 scores are not.

Step 2: The n8n trigger and CV extraction

Every application lands in a shared Gmail inbox. The ATS forwards there. A Gmail label — new-application — fires the n8n workflow.

The flow is boring on purpose:

Gmail Trigger (label: new-application)
  → Extract email body + sender
  → Download attachment (PDF/DOCX)
  → PDF-to-text node (pdf-parse)
  → Merge: {email_body, cv_text, applicant_id}
  → HTTP Request → Anthropic Messages API (claude-haiku)
  → Parse JSON response
  → IF any_fail == true → Airtable: reject_queue
  → ELSE → Airtable: human_review + Gmail label: needs-review

Model choice matters. I use Claude Haiku, not Sonnet, not Opus. We're not writing poetry — we're checking six checkboxes. Haiku runs at roughly $0.25 per million input tokens and $1.25 per million output tokens (check Anthropic's current pricing before you scale). A typical CV + cover email + prompt clocks in around 3,500 input tokens and 400 output tokens per call. That's how 312 applications came in at 41 cents total.

Sonnet would have cost roughly 12x more with zero accuracy gain on binary extraction. Save Sonnet for the tasks where reasoning actually matters.

Step 3: The prompt structure (criterion-by-criterion, not holistic)

Do not ask the model for a holistic score from 1 to 10. That's the single biggest failure mode in every hiring AI I've audited. Ask criterion by criterion, demand a quoted evidence snippet, and force a strict JSON schema.

Here's the shape of the prompt:

SYSTEM = """You evaluate a candidate against exactly six binary criteria.
For each criterion return:
  verdict: "pass" | "fail" | "unclear"
  evidence: exact quoted sentence from the CV or cover email
            that led to your decision, or null if none exists
  confidence: "high" | "low"

Do not infer. Do not guess. If the CV does not explicitly state
the fact needed to evaluate a criterion, return "unclear".
Return only valid JSON matching the provided schema."""

USER_TEMPLATE = """
CRITERIA:
1. work_authorization: Candidate has legal right to work in {country}.
2. location: Candidate is based within {radius_km} km of {city}.
3. experience: Candidate has at least {min_years} years in {domain}.
4. salary: Candidate's stated expectation is within {band_low}-{band_high} {currency}.
5. license: Candidate holds a valid {license_name}.
6. language: Candidate is fluent (B2 or higher) in {language}.

CV TEXT:
{cv_text}

COVER EMAIL:
{email_body}

Return JSON: {{"criteria": [{{"name": "...", "verdict": "...", "evidence": "...", "confidence": "..."}}]}}
"""

The routing logic that follows is trivial:

// n8n Function node
const results = JSON.parse($json.claude_response).criteria;
const hard_fail = results.some(c => c.verdict === "fail" && c.confidence === "high");
const has_unclear = results.some(c => c.verdict === "unclear");

return {
  route: hard_fail ? "reject_queue" : "human_review",
  triggered_by: hard_fail
    ? results.find(c => c.verdict === "fail" && c.confidence === "high").name
    : null,
  results
};

Notice what's not there. No "score > 7 means promote." No weighting. If any single hard criterion fails with high confidence, the candidate goes to the silent-reject queue. Unclear or all-pass goes to a human. That's the whole decision tree.

Step 4: The Airtable logging schema that makes it auditable

This is what turns the flow from a black box into a tool a non-technical recruiter can actually trust and tune.

Every decision — pass or fail — writes a row to Airtable. Three columns matter most:

Column	Example
`applicant_id`	`2026-ops-0184`
`triggered_criterion`	`location`
`evidence_snippet`	"Currently based in Berlin, open to remote roles across EU."
`verdict_full_json`	`{...}` (full 6-criteria response)
`route`	`reject_queue`
`timestamp`	`2026-08-05T14:22:11Z`
`model_version`	`claude-haiku-2026-xx`

Once a week, the recruiter opens Airtable, sorts by triggered_criterion, and asks two questions:

Are we rejecting on the right things?
Are the evidence snippets actually saying what the model thinks they're saying?

If the answer is no, they don't touch the prompt. They adjust the criteria list for that role — the role config, in Airtable, in plain English. The prompt stays generic. Criteria are config. That separation is what lets a non-technical recruiter tune this weekly without calling me.

Step 5: The shadow-review layer (don't skip this)

Three times a week, the workflow randomly pulls one CV from the reject pile and forwards it to the recruiter anyway, flagged as a shadow review. The recruiter reads it in 90 seconds and either confirms the rejection or flags a false negative.

// n8n Cron: Mon/Wed/Fri 09:00
const rejects = await airtable.list("reject_queue", {
  filterByFormula: "AND(IS_AFTER(timestamp, DATEADD(NOW(), -2, 'days')), NOT({shadow_reviewed}))"
});
const sample = rejects[Math.floor(Math.random() * rejects.length)];
await gmail.send({
  to: recruiter,
  subject: `[SHADOW REVIEW] ${sample.applicant_id} — rejected on ${sample.triggered_criterion}`,
  body: buildShadowReviewEmail(sample)
});

Over the first month we caught two false negatives out of roughly 600 rejections. Both were candidates whose CVs were poorly formatted — one was a PDF that was actually an image scan, the other used a two-column layout that scrambled during text extraction. The model wasn't wrong. The input was garbage.

We fixed the extraction step (added an OCR fallback for image PDFs, added a layout-aware parser for multi-column), not the model. If you don't have a shadow-review layer, you'll drift and you won't know you're drifting until a great candidate posts a screenshot on LinkedIn.

What to actually measure

False negative rate (target: <1%)
% of rejections triggered by each criterion (spot skew)
Extraction failure rate (CVs where >2 criteria return "unclear")
Time-to-decision per applicant (should be <30 seconds)

Step 6: The bot never sends the rejection email

Hard rule. The bot does not send the rejection email. Ever.

Several jurisdictions have disclosure requirements for automated hiring decisions. New York City's Local Law 144 requires bias audits and candidate notification for automated employment decision tools. Illinois has the AI Video Interview Act. The EU AI Act classifies most hiring AI as high-risk. The last thing a small business needs is a discrimination complaint over an email nobody read before it went out.

What we do instead: the silent-reject queue sits in Airtable. Once a week, the recruiter reviews it in bulk, spot-checks the evidence snippets, and then sends rejection emails from their own inbox using a templated response that quotes the specific criterion. A human clicks send. That's the compliance line, and it's cheap to hold.

Why bizflowai.io helps with this

We build these silent-reject pipelines for recruiter clients as a standard workflow — the Gmail trigger, the Claude Haiku evaluator, the Airtable audit log, the shadow-review sampler, and the role-config sheet that a non-technical recruiter can edit weekly. Most clients go live in under a week and see their manual CV-reading time drop by 70-85% on the first role they migrate. The pattern is the same across ops, sales, and support roles — only the six criteria change.

Want more like this?

I publish practical AI automation, GenAI engineering, and faceless content workflows on YouTube every week.

Subscribe to bizflowai.io on YouTube — never miss a new tutorial.

Planning an AI automation project or need a second opinion on your architecture?

Connect with me on LinkedIn — Lazar Milicevic, GenAI Engineer & bizflowai.io Founder.

Visit bizflowai.io for our services, case studies, and AI consulting.