DEV Community: Pykero

Why Fewer Tools Make a Better AI Agent

Pykero — Tue, 01 Sep 2026 09:03:13 +0000

Give an AI agent ten tools and you usually get worse output, not more capability. Every tool you add is another place the model can pick wrong, pass bad arguments, or silently do the wrong thing while still returning a confident-sounding answer. The agents that actually work in production tend to run on two to five tools, each scoped to one job.

This isn't a theoretical preference. It shows up constantly in how well-built agents get described: a script that reads GitHub issues and dependency manifests and finds real advisories without touching the code, or a scanner that classifies subdomains as wildcard-likely using nothing but DNS lookups. The common thread isn't clever prompting. It's that each agent has a small, fixed toolbox and never has to guess which of fifteen options to reach for.

The tool-count trap

When you're scoping an agentic build, the instinct is to be generous: give the agent a search tool, a database tool, a file tool, an email tool, a calendar tool, "just in case" it needs them. This feels safe. It isn't — and the two examples above show why each failure mode is concrete, not hypothetical:

Selection accuracy drops. The dependency-manifest scanner works because it only ever chooses between "read a manifest" and "read an issue." Add a third option, say, a general web-search tool, and now the model has to decide on every run whether the advisory it needs is in the repo or out on the web, and it will guess wrong often enough to matter.
Argument errors compound. The DNS-based subdomain classifier takes one input: a hostname. That's the whole schema. The moment you bolt on a second tool with its own parameters, you've doubled the surface area for a malformed call, and on a write action the cost of that malformed call is no longer "wrong DNS answer" but "wrong record submitted."
Failures go quiet. Both of those tools are read-only, so a wrong pick just produces a wrong classification you can spot in the output. An agent wired with a write tool alongside them can misuse it, log success, and move on, and nobody notices until a customer complains.
Cost creeps up. Every tool definition rides along in the context window on every call, whether the manifest scanner ever touches its second or third tool or not. A bloated tool list is a bloated prompt on every single turn.

Anthropic's own guidance on agent design makes the same point from the build side: the most reliable agentic systems are composed of simple, well-tested components, not a single agent juggling every capability at once (see Anthropic's engineering write-up on building effective agents). Simplicity isn't a starting point you graduate out of. It's the thing that keeps working at scale.

What narrow scope looks like in practice

We run our own outbound engine this way. It scrapes a prospect's site, extracts the handful of facts that matter, and drafts one tailored email, all in a single focused call rather than a multi-step chain with a search tool, a CRM tool, and a drafting tool stitched together. Fewer moving parts meant fewer places for the process to quietly go wrong, and it was cheaper to run per lead. The lesson generalized past outreach: the fewer tools an agent needs to reach for, the easier it is to know exactly what it did and why. That's a different question from whether to use a single LLM call or an agent chain for a given task, but the same instinct applies at the tool layer: don't hand the model options it doesn't need.

How to audit your agent's tool list

If you're evaluating a build (in-house or from an agency), ask to see the tool list before you ask about the model or the prompt. A few checks:

Can you name the one job this agent does? Our outbound engine passes this test in one sentence: it turns a prospect's site into one drafted email. If the answer for someone else's agent takes a paragraph instead, it's probably two agents wearing one trench coat.
Does every tool get used on a normal run, or are some "just in case"? The scrape-extract-draft call has no unused tool sitting idle in the schema. Unused tools are pure downside: cost and confusion with no benefit.
Are any two tools easy to confuse from their names and descriptions alone? If a human skimming the schema would hesitate, the model will too.
What happens when the agent picks the wrong tool? If there's no logging that surfaces a wrong call, you won't find out until the customer does.
Could this be two narrow agents with a router instead of one broad agent? Routing logic is boring and testable. A single model juggling twelve tools is neither.

When more tools genuinely make sense

None of this means every agent should be a single-tool script. Some jobs really do span multiple systems, a support agent that needs to look up an order, check a shipping carrier, and issue a refund is legitimately doing three things. The fix there usually isn't fewer tools on one agent, it's splitting the job across a workflow of smaller steps with explicit handoffs, rather than one model deciding among a dozen options on every turn. A deterministic workflow that calls three narrow agents in sequence is far easier to debug than one agent with nine tools trying to figure out the right order itself.

What to ask a vendor about this

If you're comparing agencies or in-house proposals for an agentic build, tool scope is a cheap, high-signal question to add to your vendor evaluation checklist. Ask them to walk through the tool list for the proposed agent and justify each one. A team that's thought this through will have a crisp answer for every tool. A team that hasn't will start explaining tools by describing scenarios, which is usually the sound of scope creep in real time.

Tool count is one of the few agent-design decisions you can sanity-check without reading a line of code, and it correlates strongly with how the whole system will behave once it's live and no one's watching it run.

If you're scoping an agentic build and want a second opinion on the tool list before you commit to it, let's talk.

Originally published on the Pykero blog.

Why Your AI Feature Needs Deterministic Guardrails, Not a Smarter Model

Pykero — Mon, 31 Aug 2026 09:02:32 +0000

LLMs are probabilistic, so trying to make one output consistently is a losing battle. The fix isn't a bigger model or a longer prompt: it's a deterministic layer of rules sitting between the model and your users that decides what's fatal and what isn't, every time, the same way.

Founders evaluating an AI feature (or an agency building one for them) usually ask the wrong question first: "which model is most reliable?" Every model, including the newest release from Anthropic or OpenAI, samples from a probability distribution over tokens. Run the same prompt twice and you can get two different answers, even at low temperature, because of batching and floating-point nondeterminism on the inference side. That's not a defect you prompt your way out of. It's how the technology works.

The mistake: chasing determinism in the model layer

Teams that haven't shipped AI to production yet tend to solve "the model is inconsistent" by:

Lowering temperature to 0 and assuming that means deterministic (it doesn't, fully)
Adding more few-shot examples to "pin" the format
Re-running the same prompt and taking a majority vote, which multiplies cost for marginal gain
Escalating to a bigger, more expensive model when the real issue is architectural

We did versions of the first three on our own outreach engine before we rebuilt it: more examples in the extraction prompt to try to pin the format, temperature dropped to 0 on the drafting call, and at one point a re-run-and-pick-the-better-one step that just doubled the token bill without fixing the underlying inconsistency. None of it stuck, because the output was still a judgment call made by a stochastic process at every one of those chained hops, and no amount of prompt engineering changes that category. If your business logic depends on the model always classifying, scoring, or routing the same way, that logic has to live outside the model.

The fix: separate judgment from consequence

The pattern that works: let the LLM do what it's actually good at (open-ended judgment, extraction, drafting) and hand its output to a deterministic rule layer that decides what happens next. Concretely:

The model scores or classifies. It can flip-flop between runs; that's fine because nothing downstream trusts it blindly.
A fixed rule set decides fatal vs. non-fatal. This is plain code: a frozenset of disqualifying conditions, a JSON Schema validator against the JSON Schema spec, or a small table of business rules. It never calls the model and it never changes between requests.
Only the rule layer's decision is logged and acted on. The model's raw score becomes an input, not the verdict.

This means your test suite can be genuinely deterministic even though the thing being tested isn't. You write unit tests against the rule layer (easy, fast, no API calls) and you monitor the model's distribution of outputs over time (drift detection) instead of asserting on any single response. Those are two different problems and conflating them is why so many teams end up with flaky AI test suites they eventually just skip.

Where this shows up in real builds

We've seen the same principle pay off in our own tooling outside of chat features. When we built our outreach engine, we initially chained several LLM calls together: scrape the prospect's site, summarize it, extract facts, then draft the email. Every added step was another place variance could compound, and debugging why a bad email came out meant tracing through four nondeterministic hops. Collapsing it to a single call that both extracted facts and drafted the email, with a deterministic post-check on length, banned phrases, and required personalization fields, beat the multi-step chain on both cost and quality. Fewer stochastic hops meant fewer places for the rule layer to have to catch problems, and the problems it did catch were easier to trace back to a single call. That's the same underlying lesson as the guardrail pattern: push variance to as few places as possible, then wrap what's left in rules you can actually test. If you're deciding between a single well-scoped call and a multi-step agent chain, the tradeoffs are the same ones we cover in single call vs agent chains.

What to ask a vendor or agency about this

If you're evaluating who builds an AI feature for you, "how do you make the output reliable" is a fair question, and "we use the latest model" is not a real answer. Ask instead:

What's the deterministic layer? The answer should sound like the frozenset of disqualifying conditions or the JSON Schema validator described above, not "the model is instructed to be careful." If they can't name the specific rule set in one sentence, they haven't built one.
How do you test it? You should hear about unit tests against the rule layer plus drift monitoring on the model's distribution, not "we ran it a few times and it looked good."
What happens when the rule layer rejects an output? Retry with a different prompt, fall back to a safe default, or escalate to a human. All three are valid; "nothing, we just log it" is not.
Is the rule layer in the contract? If reliability matters enough to pay for, it should be specified alongside the deliverable, not left implicit. This overlaps with what we cover in evals in AI vendor contracts.

This also has a direct cost angle: every retry or majority-vote call to the model is a token bill you're paying to compensate for the absence of a rule layer. Teams that skip the deterministic layer often end up solving the cost problem later with rate limits and caching, which is a valid pattern on its own but treats a symptom the guardrail would have prevented; see rate limiting AI features for cost control if that's already where you are.

The bottom line

Don't hire for "the model that hallucinates least." Hire for the team that can show you, concretely, what happens to a bad output after the model produces it. A frozenset deciding what's fatal is a more honest reliability story than any claim about model quality, because it's testable, it doesn't drift, and you can read it in five minutes during a vendor review.

If you're scoping an AI feature and want a second opinion on where the deterministic layer should sit, let's talk.

Originally published on the Pykero blog.

AI Cold Email Deliverability: What Actually Breaks It

Pykero — Sat, 29 Aug 2026 09:02:56 +0000

AI-personalized cold email fails for the same reason templated cold email fails: bad sending infrastructure, not weak copy. Mailbox providers decide whether your message reaches the inbox before a human ever reads a word of your AI-written first line, based on domain authentication, sending reputation, and volume patterns. Fix those first, then let AI improve the copy.

Why personalization alone doesn't fix deliverability

Founders building or buying an AI cold-email system usually optimize the wrong layer first. They obsess over prompt quality, tailoring every line to the prospect's company, and skip the boring infrastructure that determines whether Gmail and Outlook even queue the message for the inbox instead of spam. We made this mistake with our own outreach engine before we ever looked at DNS records: the personalization layer was already solid, drafts read like someone had actually opened the prospect's site, and none of it moved reply rates until the authentication was fixed underneath it.

Personalization does help deliverability, but indirectly: fewer recipients mark a relevant email as spam, and spam-complaint rate is one of the strongest reputation signals mailbox providers track. What it doesn't do is substitute for the authentication checks that happen before your subject line is scored at all.

The technical baseline: SPF, DKIM, DMARC

Every major mailbox provider now enforces authentication for bulk senders. Google's own guidance for senders is explicit about this, requiring SPF and DKIM alignment plus a DMARC policy for anyone sending meaningful volume (Google's bulk sender guidelines):

SPF tells receiving servers which mail servers are allowed to send on behalf of your domain.
DKIM cryptographically signs each message so it can't be altered in transit without detection.
DMARC ties the two together and tells receivers what to do when a message fails, publish a policy and you're no longer relying on the receiver's goodwill.

DMARC also gives you visibility you didn't have before: aggregate reports show exactly who's sending mail claiming to be your domain, which matters if you're running an AI agent that sends on your behalf (dmarc.org overview). None of this is AI-specific. It's the same checklist a bulk newsletter sender needs, and skipping it is the single most common reason a technically impressive AI outreach agent lands in spam on day one.

Where AI actually helps: content, not infrastructure

Once authentication is solid, AI's real contribution is making each email read like it was written by someone who spent five minutes on the prospect's site, because in our case, it was. We run our own outreach engine that scrapes each prospect's site with a self-hosted Firecrawl instance and a local LLM, then drafts one tailored email per company in a single call rather than a multi-step chain that extracts facts, then plans, then writes. The single-call "extract facts and draft" pattern beat the multi-step version on both cost and output quality, mostly because fewer LLM calls meant fewer places for the draft to drift from what was actually on the page. If you're deciding between these approaches for your own agent, the tradeoffs are the same ones covered in single-call vs. agent chains.

Genuine personalization also reduces the pattern-matching that spam filters use. A blast of a thousand near-identical emails with swapped-in first names is easy for a filter to cluster and flag. A thousand emails referencing each company's actual product, stack, or recent announcement look, mechanically, like a thousand different emails.

Volume ramp and sending cadence

New domains and new subdomains have no reputation yet, and mailbox providers treat sudden volume from an unknown sender as a spam signal by default. This is exactly why we cap sends at the infrastructure layer for our own outreach engine rather than trusting the model to pace itself, the same system described above that drafts one email per company in a single LLM call and could otherwise fire all of them at once. The practical approach:

Start on a dedicated sending subdomain, never your primary company domain, so a reputation problem never touches billing, support, or product email.
Ramp gradually, most senders start around 20-30 emails per day and increase over several weeks rather than firing a full list on day one.
Keep bounce rate and spam-complaint rate low during the ramp; both are weighted more heavily by mailbox providers than open rate ever was.
Spread sending across the day instead of firing a batch job at 9:00am sharp, bursty patterns look automated because they are.

An AI agent makes this worse if you let it, since it's perfectly capable of firing 5,000 personalized emails in a minute. The rate limit needs to live in your sending infrastructure, not in the model's judgment.

The CTA matters more than the subject line

Once a message lands, what you ask for determines whether it converts. In our own cold-email tool, moving the call-to-action from "book a call" (a calendar link) to a soft "reply YES" consistently lifted reply rates. Scheduling from a cold email adds friction that kills otherwise-interested leads, they meant to click through later and never did. A reply, by contrast, is nearly zero effort, and it keeps the conversation inside the channel where the prospect already is. It also happens to be a stronger deliverability signal: replies tell mailbox providers this sender produces wanted mail, more so than opens or clicks. If you're designing the ask for an AI-driven outreach or sales agent, this is worth reading alongside designing the CTA for an AI sales agent.

Build vs. buy for the sending layer

Writing the personalization logic is the fun part. Getting SPF, DKIM, DMARC, subdomain warm-up, bounce handling, and suppression lists right is the unglamorous part that determines whether any of it matters. Several vendors handle the sending infrastructure so you can focus the AI layer purely on research and drafting; others expect you to own the whole stack. We cover the actual tradeoffs, and where a managed sending layer is worth the markup, in AI cold email agent: build vs buy.

If you're scoping an AI-driven outreach system and want a second opinion on where to spend engineering time, let's talk.

Originally published on the Pykero blog.

How to Detect an AI Agent Scheduled Run That Silently Failed

Pykero — Thu, 27 Aug 2026 09:01:54 +0000

A scheduled AI agent that never fires looks, from the outside, exactly like one that ran and did nothing wrong. Nobody gets an error email. Nothing shows up in your exception tracker. The only signal is an absence, and absences are easy to miss until a customer, an auditor, or a lost week of leads points it out first. The fix is not better error handling inside the agent; it's a monitor that lives outside the agent and expects to hear from it.

Why silent failure is the default failure mode for scheduled agents

Most engineering effort goes into making an agent handle bad input gracefully: retries, fallbacks, structured error logs. That's useful, but it assumes the agent's process actually started. In practice, the runs that never happen usually fail for reasons that never touch your application code at all:

A cron entry or scheduler config gets edited (or removed) during a deploy and nobody notices.
The host, container, or serverless function it runs on is down, throttled, or out of quota.
An API key or OAuth token expires and the process exits before your own logging even initializes.
A queue or orchestrator (Airflow, Temporal, a cron-triggered Lambda) silently drops the trigger during a restart.

None of these produce an exception inside your agent's code, because your agent's code never runs. Any alerting you've wired up inside the agent is blind to exactly the failure mode that matters most here. This is the same class of problem covered in our AI agent security checklist: the risk isn't in the logic you tested, it's in the boundary conditions nobody wrote a test for.

The pattern: a dead man's switch, not an error alert

The standard fix is a heartbeat monitor, sometimes called a dead man's switch: a separate, dumb, extremely reliable service that expects to receive a signal ("I ran, I'm alive") on a schedule, and pages you when that signal doesn't show up within a grace window. The agent doesn't need to report success or failure in detail; it just needs to check in.

A minimal implementation looks like this:

# at the start of the scheduled job
curl -fsS https://your-monitor.example/ping/start-token

# ... agent does its work ...

# after the job completes (success or handled failure)
curl -fsS https://your-monitor.example/ping/finish-token

If finish-token doesn't arrive within, say, 15 minutes of the expected schedule, the monitor pages someone. It doesn't matter whether the job crashed, the container never started, or the whole account got suspended: the absence of the ping is the signal. This is the same principle behind Healthchecks.io's cron monitoring docs, one of the more widely used implementations of the pattern, and it works whether your "agent" is a single cron job or a fleet of LLM-driven workers.

What to actually monitor

Don't just monitor "did the process exit 0." For an AI agent specifically, check-in on the steps that matter to the business outcome, not just the runtime:

Did it start on schedule (catches scheduler and infra failures)
Did it produce output of the expected shape (catches a model silently returning empty or malformed responses)
Did the downstream action actually happen (email sent, record written, ticket created), not just "the LLM call returned"

That third one matters more than it sounds. An agent can complete its loop, log success, and still have failed the person waiting on it, if the final write to your CRM or database silently no-ops. Heartbeat on the outcome, not the process.

A real example: monitoring a nightly outreach agent

We run a self-hosted outreach agent that scrapes each prospect's site and drafts one tailored email per company, on a nightly schedule. Early on, we only had application-level logging: if the scraper or the LLM call threw an error, we'd see it. What we didn't catch, for one uncomfortable week, was a scheduler misconfiguration after a server migration that meant the job simply never started at all. No errors, no alerts, just a week of leads that quietly never got emails. Adding a heartbeat check that expected a daily ping, and paged us when one didn't arrive, would have caught that in a day instead of a week. It's a small addition on top of the agent itself, but it's the difference between "the system is degraded" and "the system stopped existing and nobody noticed."

What this means for buy vs. build decisions

If you're evaluating an agency or vendor to build or operate a scheduled AI agent for you, ask directly: how do you detect a run that never happened? A vendor who can answer with a specific monitor, grace period, and escalation path has actually thought about operational reliability. A vendor who says "we'd see it in the logs" is describing a system that requires a human to go looking, which is precisely the failure mode a dead man's switch exists to remove. This ties directly into ongoing AI agent maintenance costs: monitoring infrastructure is cheap to add up front and expensive to retrofit after the first missed run costs you something real. It's also worth clarifying who gets paged and when, which overlaps with how you've defined escalation paths for the agent more broadly.

Practical checklist

Add a start and finish ping to every business-critical scheduled agent, not just error alerting
Set the grace period based on how bad a missed run actually is: minutes for revenue-critical agents, hours for reporting jobs
Monitor the downstream outcome (record written, email sent), not just the process exit code
Make sure the alert reaches a person, not just a dashboard nobody checks
Test the alert path itself occasionally: an alert that nobody notices is the same as no alert

Scheduled agents fail quietly far more often than they fail loudly. The fix isn't more error handling inside the code, it's a separate, boring, reliable check that something outside the agent is watching for its absence.

If you're building or operating agentic systems and want a second set of eyes on how they're monitored, let's talk.

Originally published on the Pykero blog.

How to Calculate ROI on an AI Agent Before You Build It

Pykero — Tue, 25 Aug 2026 09:01:27 +0000

Calculate ROI by comparing what the manual process actually costs today (fully-loaded labor, error correction, missed volume) against the one-time build cost plus ongoing per-run inference cost, then solve for the volume where those lines cross. If that break-even point is inside your realistic 12-month volume, build it. If it isn't, don't.

Most founders skip this and instead ask "is AI worth it," which isn't answerable. The answerable question is narrower: does automating this specific task, at this volume, pay back faster than the cash and attention it costs to build and run. Here's the framework we use before scoping any agent project.

Step 1: Price the manual baseline honestly

Take the task the agent would replace and cost it out the way a CFO would, not the way an optimist would.

Fully-loaded hourly cost of whoever does the task now (salary + benefits + overhead, not just base pay)
Time per unit: minutes to draft one outreach email, triage one support ticket, extract one invoice
Error cost: rework, refunds, or missed follow-ups caused by human inconsistency at volume
Opportunity cost: work that doesn't happen because a person is stuck doing the repetitive task instead

Multiply time-per-unit by your monthly volume and you get a real monthly cost for the status quo. This number is the target the agent has to beat, and it's usually higher than people expect once error cost and opportunity cost are included.

Step 2: Price the agent, not just the model

Two cost lines, and teams routinely forget the second one:

Build cost (one-time): scoping, prompt/eval iteration, integration with your systems, and the first month of monitoring while you tune it. For a well-scoped single-purpose agent this is usually the bulk of what you'd budget in an MVP timeline, not a multi-quarter platform build.

Run cost (ongoing, per unit): inference tokens, any tool/API calls the agent makes, plus maintenance as prompts drift and models get swapped out. Get this number from a real test batch, not a spec sheet. Run 100-200 real inputs through the actual prompt on the actual model and take the average cost per run. Guessing here is where ROI math falls apart, because agent costs compound with volume in a way flat SaaS subscriptions don't.

One thing worth deciding early: whether the task needs a multi-step agent chain or a single well-designed call. We run our own outreach system on a single-call pattern: one model call that reads a scraped page and produces both the extracted facts and the drafted email, instead of a multi-step chain that extracts, then summarizes, then drafts. It came out cheaper per unit and more consistent in output, because there were fewer places for the process to drift or fail silently. If your task can be scoped that tightly, your run cost drops and your break-even point arrives sooner. Worth reading before committing to an architecture: single-call vs agent chains.

Step 3: Find the break-even volume

With both cost lines in hand:

Break-even volume = Build cost / (Manual cost per unit - Agent run cost per unit)

If manual cost per unit is $4.50 (labor + error cost) and agent run cost is $0.15, your savings per unit is $4.35. A $6,000 build cost breaks even at roughly 1,380 units. If you're doing 2,000 of those units a month, you're profitable inside month one. If you're doing 200 a month, you're looking at seven months, which changes the calculus, especially if the process might change before then.

This is also where you catch the projects that shouldn't be built. Low-volume, highly variable tasks (the exception-handling, judgment-heavy 20%) rarely clear break-even and are usually cheaper to leave with a human.

Step 4: Model the second-order effect, not just the cost swap

Cost savings alone often undersells the case. The better ROI usually comes from volume the agent unlocks that a human never had time for. In our own cold-email tool, the win wasn't just cheaper drafting, it was that we could personalize every single email instead of batching generic ones, which is a volume/quality tradeoff a human team can't make at scale. Separately, we found that changing the call-to-action from a calendar booking link to a soft "reply YES" consistently lifted reply rates, because scheduling friction was killing interest from people who were otherwise ready to engage. Neither of those shows up in a pure cost-per-unit calculation, but both moved the actual number that mattered: revenue per dollar spent on outreach.

When you're scoping ROI, ask what becomes possible at the new cost and speed, not just what gets cheaper.

Step 5: Set a kill threshold before you start

Decide upfront what "not working" looks like: a break-even date, an accuracy floor on evals, or a cost-per-unit ceiling. Write it down before the build starts, not after you're three months in and emotionally invested. This is the single biggest difference between teams that get real ROI from agents and teams that keep funding a project because stopping feels like admitting failure.

The short version

Cost the manual process honestly, price the agent's build and run cost from real test data, solve for break-even against your real volume, and decide in advance what would make you kill it. Volume and repeatability are what make agent ROI work, not the novelty of the technology.

If you want a second pair of eyes on whether a specific workflow clears that bar before you spend build budget on it, let's talk.

Originally published on the Pykero blog.

AI Data Residency Rules in Saudi Arabia and the UAE

Pykero — Sun, 23 Aug 2026 09:01:53 +0000

Saudi Arabia's PDPL and the UAE's federal data protection law both treat cross-border transfer of personal data as the exception, not the default. If your AI agent sends customer records, patient data, or citizen information to a model API hosted in the US or EU, you need a documented legal basis for that transfer before you ship, and for healthcare or government clients you should assume the answer is "host it in-region" until proven otherwise.

This catches founders off guard because most AI tooling defaults to whatever region the model provider happens to run in. OpenAI, Anthropic, and most managed LLM APIs process requests in US or EU data centers unless you specifically configure otherwise. That's a fine default for a SaaS dashboard. It's a real liability for a WhatsApp sales agent handling Saudi customer PII, or a voice agent transcribing patient calls for a UAE clinic.

What the two laws actually require

Saudi Arabia (PDPL): Personal data may not leave the Kingdom unless the destination country provides an "adequate" level of protection, the transfer is necessary for a specific purpose (like fulfilling a contract), or the data subject has given explicit consent. SDAIA, the regulator, publishes the current rules and enforcement guidance directly. Start there rather than a third-party summary: SDAIA.

UAE: The federal data protection framework similarly restricts transfers outside the country unless the receiving jurisdiction has comparable protections or you've put contractual safeguards in place. The UAE government's official portal keeps a plain-language summary current: u.ae data protection.

Neither law bans cross-border AI processing outright. Both make it a compliance step you have to actively clear, which means it needs to show up in your architecture decisions before launch, not as a patch after a client's legal team asks about it.

Where this actually bites founders

Healthcare voice agents. If you're transcribing patient calls with speech recognition, the audio and transcript are personal health data twice over. We've covered the cost side of this in healthcare voice AI cost and the architecture side in court-ready architecture for healthcare AI; data residency is the piece that connects both.
Government and public-sector pilots. Procurement teams will ask where the model runs before they ask what it does. See AI procurement checklist for government and healthcare for the fuller list of questions to expect.
Arabic and dialectal speech recognition. Gulf-Arabic ASR often means sending raw audio to a third-party transcription API. If that API is US-hosted and your users are UAE nationals, that's a transfer event worth documenting even if the content is mundane.

Self-hosting is the practical fix, not just the compliant one

When a client's data can't leave the country, or when leaving it requires a paper trail you'd rather not maintain, the real answer is usually to run the model in-region yourself. We wrote about the cost and control tradeoffs generally in self-hosting an LLM vs API cost and compliance, but the Gulf context adds a wrinkle: your options for a compliant managed regional endpoint are thinner than in the US or EU, so self-hosting an open-weight model on a Riyadh or Dubai availability zone is often the fastest path to "yes" in a procurement conversation, not a fallback.

We run into a version of this pattern in our own tooling. Our outreach engine scrapes each prospect's site with a self-hosted Firecrawl instance and a local LLM, doing fact extraction and email drafting in a single call rather than chaining it through a hosted API. We didn't build it that way for compliance, we built it because a single-call pattern beat multi-step chains on cost and quality. But the side effect is that nothing about a prospect's site data ever leaves our own infrastructure, which is exactly the property a Gulf healthcare or government client is asking for when they say "can this stay in-region." The lesson generalizes: if you're already going to self-host for latency or cost reasons, you get data residency for free.

A checklist before you deploy an AI agent for a Gulf client

Classify the data. Personal, health, or government-sensitive data gets the strict treatment; anonymized product analytics usually doesn't.
Ask every vendor where inference actually runs. Not where the company is headquartered, where the GPU is.
Check for a regional cloud option first. AWS, Azure, and Google Cloud all operate Middle East regions; confirm your model provider actually supports routing to them before assuming it does.
Default to self-hosting for health and government workloads. It removes the transfer question entirely instead of managing it contractually.
Put the answer in the contract, not just the architecture doc. Procurement teams want it written down, and "we host in-region" is a one-line clause that closes deals faster than a technical explanation ever will.

Data residency isn't a checkbox you clear once. Model providers change their regional footprint, laws get amended, and a pilot that started with anonymized test data can quietly start touching real PII once it's in production. Build the review into your deployment process, not just your kickoff call.

If you're scoping an AI agent for a Saudi or UAE client and need to figure out the residency question before you write a line of code, let's talk.

Originally published on the Pykero blog.

AI Agent Memory: Build vs Buy

Pykero — Sat, 15 Aug 2026 09:01:19 +0000

Build your own agent memory when your retrieval pattern is simple and stable: a Postgres table with a few typed columns and a recency filter will outperform a platform integration in both cost and debuggability. Buy a managed memory layer only once you have many users, each accumulating their own long-lived memory graph, and no spare engineering time to own decay, summarization, and ranking logic yourself.

"Agent memory" became a crowded category almost overnight: every AI agent vendor now ships a "memory" product, and every founder building an agent asks whether they need one. The honest answer is that most teams conflate three very different problems and end up buying a platform for a problem that a single SQL table would have solved.

What "memory" actually means for an agent

Strip away the marketing and agent memory is one of three things:

Session state: what happened in this conversation. This is just context window management, not memory. Don't buy anything for this.
Structured facts: a user's plan tier, their last five orders, a preference they stated once. This is a database problem with a lookup key.
Emergent recall: the agent surfaces something relevant that nobody explicitly indexed, weeks after it was mentioned, ranked by relevance and recency. This is the only piece that's genuinely hard.

Most products that ask "do we need agent memory" are actually asking about the second bucket, and the second bucket is a facts table with a user_id, a type, a value, and an updated_at column. No platform required.

The build case

If your memory lookups are predictable, roll your own. A typical schema is a Postgres table keyed by user or tenant, a type enum for the kind of fact, a JSON value column, and pgvector on a summary field for the cases where you genuinely need semantic search rather than exact lookup. See the PostgreSQL docs on JSONB and pgvector for the primitives.

The advantage isn't just cost, it's control. When memory misbehaves (an agent surfaces a stale fact, or forgets something it should have retained), you need to be able to SELECT * FROM facts WHERE user_id = ... and see exactly what's stored and why it was or wasn't retrieved. A managed platform turns that into a support ticket. We've found the same principle holds in our own single-call vs agent chains work: the fewer indirection layers between your code and the underlying store, the faster you can diagnose why an agent did something odd.

Build also wins when:

You have one primary retrieval pattern (by user, by conversation, by entity) rather than open-ended "find anything relevant."
Your data volume per user is small (tens to low hundreds of facts), so brute-force filtering beats a specialized ranking engine.
You already run Postgres and don't want another vendor in your AI agent vendor evaluation list.

The buy case

Managed memory platforms earn their keep when the retrieval problem becomes genuinely open-ended: many users, each with a large and growing memory graph, where the agent needs to decide which memories are relevant to the current turn without you writing that ranking logic by hand. That's a real engineering problem involving decay functions, contradiction resolution (the user's preference changed, which fact wins), and summarization of old memories into compressed ones so the graph doesn't grow unbounded.

Buy when:

You're running consumer-scale, multi-tenant agents where memory volume per user will keep growing indefinitely.
You need automatic summarization or forgetting, and building that well is a multi-week project you don't have room for.
Your team's differentiation is the product experience, not the memory infrastructure, and the platform's pricing is genuinely cheaper than the engineering time to replicate it.

The risk on the buy side mirrors what we've written about self-hosting LLMs vs API cost and compliance: you're trading a one-time build cost for an ongoing per-call or per-user fee, plus a dependency whose roadmap you don't control. Read the vendor's actual retrieval and ranking behavior before committing, the same way you'd read evals in an AI vendor contract before signing.

A pattern that generalizes: match the tool to the retrieval shape

We ran into a version of this same build-vs-buy question on our own outreach agent, which scrapes each prospect's site and drafts a tailored email. Early on we considered a multi-step pipeline with a separate memory step to track what it had already learned about a company across runs. It turned out a single-call pattern with the scraped facts passed directly in context beat the multi-step version on both cost and quality, because the "memory" we needed was really just "the last scrape result," not an evolving graph. The lesson carries over directly: don't reach for infrastructure shaped for open-ended recall when your actual retrieval pattern is a lookup.

A simple decision test

Ask three questions before picking a side:

Can you name the query in advance? If you can write the WHERE clause today, build it.
Will memory per user grow without bound? If yes and you have no plan to prune it, buying decay/summarization logic saves real time.
Do you have a person who owns this for the next year? Built systems need an owner. If nobody has bandwidth, buying shifts that ownership to the vendor, at a cost.

Most agent products, especially B2B tools with a few hundred structured facts per account, land on build. Consumer-scale personalization products land on buy. Know which one you're building before you shop for a platform.

If you're scoping an agent and want a second opinion on whether memory is even the right layer to invest in, let's talk.

Originally published on the Pykero blog.

Managed vs Self-Hosted Agent Runtime: The Real Tradeoffs

Pykero — Thu, 13 Aug 2026 09:01:25 +0000

A managed agent runtime is a vendor-run service that executes your AI agent's loop for you, so you write the prompts and tool definitions and they handle orchestration, retries, and scaling. Self-hosting means you run that loop yourself. The right choice depends less on cost and more on who needs to see your data and how much control you need over failure behavior.

Most founders don't think about the runtime at all until something breaks: a tool call hangs, a retry storm triples your LLM bill overnight, or a customer asks where their conversation transcripts are stored. That's usually the first moment "managed vs self-hosted" becomes a real decision instead of a default.

What the runtime actually does

Strip away the marketing and an agent runtime handles four things:

State management between steps, so the agent remembers what it already tried
Tool execution, calling your APIs, databases, or external services
Retry and timeout logic when a step fails or hangs
Observability, logging what the agent did and why

A managed platform (LangGraph Cloud, CrewAI's hosted offering, various vertical agent platforms) gives you all four out of the box, usually with a dashboard. Self-hosting means you build or configure each piece yourself, often on top of a queue (SQS, Redis) and a worker process you control.

What you give up either way

With a managed runtime, you give up:

Visibility into exactly how retries and timeouts are implemented, which matters when an agent silently retries a non-idempotent action (charging a card twice, sending a duplicate WhatsApp message)
Data residency guarantees, since prompts and tool outputs typically transit the vendor's servers, which is a real problem for healthcare or government clients who require in-region processing
Pricing predictability, because most charge per execution or per token pass-through on top of your LLM bill

With self-hosting, you give up:

Time. Building reliable retry logic, dead-letter handling, and step-level observability is a multi-week project, not a config flag
The vendor's SLA. If your worker process falls over at 2 a.m., that's your on-call rotation now
Ready-made debugging tools. Vendors bake in step-by-step replay UIs that take real effort to replicate

Neither side is free. The question is which cost you can absorb right now.

Where this bites founders in practice

We've seen the tradeoff most clearly in agent execution patterns, not just hosting location. When we built our own outreach engine, the first version chained multiple LLM calls together: one to extract facts about a prospect, another to draft the email, a third to refine tone. Running that chain through a managed orchestration layer worked, but every extra hop added latency and another point where a hung tool call could cascade into a stuck workflow the dashboard didn't clearly explain. Collapsing it into a single call that both extracted facts (via a self-hosted Firecrawl instance) and drafted the email in one pass cut both cost and failure surface, because there was simply less runtime state to manage or lose visibility into. The lesson generalizes: the fewer moving parts your agent's loop has, the less it matters whether you're managed or self-hosted, and the more it matters once your workflow grows past two or three steps. That's a variant of the same tradeoff covered in single-call vs agent chains.

Serverless self-hosting has its own trap: if your agent's tool calls can run long (waiting on a slow third-party API, a document parse, a human-in-the-loop approval), a platform like AWS Lambda enforces a hard 15-minute execution ceiling per invocation, per AWS's documented limits. Teams that don't check this upfront discover it when an agent workflow that worked fine in testing starts silently truncating in production under real-world API latency.

A rough decision framework

Go managed if:

You're validating whether the agent use case works at all before committing engineering time
The data passing through isn't regulated or customer-sensitive
You don't yet have someone who owns infrastructure reliability

Go self-hosted if:

The agent touches health records, financial data, or anything with a data residency requirement, similar reasoning to why we recommend self-hosting an LLM over API calls once compliance is in scope
You're running enough volume that per-execution fees are a real line item, not a rounding error
You need custom retry semantics because some of your tool calls aren't safe to retry blindly (payments, outbound messages, database writes)

Either way, decouple your agent's logic (prompts, tool schemas, state definitions) from the runtime's specific SDK. If every tool function calls a vendor-specific orchestration API directly, migrating later means rewriting the agent, not just redeploying it. This is the same due-diligence question we walk through in our agent vendor evaluation checklist: ask the vendor upfront what happens to your logic if you leave.

The maintenance cost nobody quotes upfront

Whichever you pick, budget for ongoing maintenance, not just initial build. Managed runtimes still need someone watching cost per execution and updating prompts as your product changes. Self-hosted runtimes need someone patching the worker infrastructure and handling incidents. We break down what that ongoing cost typically looks like in AI agent maintenance cost, and it's rarely zero on either path.

If you're weighing this decision for a specific product and want a second opinion on where the line should sit for your data and volume, let's talk.

Originally published on the Pykero blog.

Self-Hosting an LLM vs. API: When It Actually Pays Off

Pykero — Tue, 11 Aug 2026 09:01:24 +0000

Self-hosting an LLM pays off in two situations: you're running high enough volume that GPU cost per token beats API cost per token, or you have a data residency requirement that API calls can't satisfy no matter the price. Outside those two cases, calling OpenAI, Anthropic, or Google's API is cheaper, faster to ship, and easier to maintain than running your own inference stack.

We get this question a lot from healthcare and govtech founders specifically, because "can the data leave our network" is often not a cost question at all, it's a legal one. So the framework below splits the decision into cost and compliance, because founders usually only need one of the two answers, not both.

The cost math, worked through

A single NVIDIA A100 80GB costs roughly $2 to $3 per hour on-demand from providers like Lambda or CoreWeave. Run it 24/7 for a month and you're at $1,500 to $2,200, regardless of whether it processes one request or one million. That's the core problem with self-hosting: you're paying for capacity, not usage.

Compare that to API pricing, where you pay per token and nothing when idle. For a workload doing, say, 5 million tokens a day on a mid-tier model, API costs typically land well under $1,500/month, and you have zero ops burden. The crossover point where a dedicated GPU starts winning is usually somewhere north of 20 to 50 million tokens/day of sustained, predictable traffic, and even then you need someone maintaining the serving stack (vLLM, TGI, or similar) and handling GPU failures, driver updates, and scaling.

We went through a version of this math ourselves. Our cold-outreach tool scrapes each prospect's site with a self-hosted Firecrawl instance and a local model to extract facts and draft an email in a single call. At our volume, running that step locally was cheaper and gave us more control over rate limits than routing every scrape through a hosted API, but we didn't self-host the actual sales copywriting, that still goes to a hosted model because the volume doesn't justify dedicated GPU capacity and the quality bar is higher. Mixing the two, self-hosted for cheap deterministic tasks, API for high-stakes generation, is usually the right shape for early-stage products. It's the same reasoning we lay out in LLM cost optimization: match the model tier and hosting model to the task, not the other way around.

Where the "hidden" self-hosting costs come from

Serving infrastructure. vLLM or Text Generation Inference handle batching and KV-cache management, but someone has to operate them, patch them, and handle OOM crashes under load.
Model updates. Open-weight models improve fast. Committing to self-hosting means you own the fine-tuning and evaluation cycle every time a better base model ships, instead of a provider swapping it in behind an API.
Redundancy. One GPU node is a single point of failure. Real uptime needs at least two, which roughly doubles the baseline cost we quoted above.

The compliance case, which is a different question entirely

If your driver is data residency, PHI, or a client contract that prohibits sending data to a third party, the cost math above is mostly irrelevant. You're not comparing dollars, you're comparing "can we legally do this at all." That's the situation a lot of our healthcare and government-adjacent clients are in, and it's the same territory we cover in court-ready architecture for healthcare AI.

A few things worth being precise about here:

Self-hosting solves the "data leaves our network" problem, but it does not by itself make you HIPAA compliant. You still need encryption at rest and in transit, access logging, and a documented retention policy around the model and its logs.
Managed API providers do offer BAA-covered, HIPAA-eligible endpoints (both OpenAI and Anthropic offer these for enterprise customers), so "we need HIPAA compliance" doesn't automatically mean "we need to self-host." Check that option before assuming you need your own GPUs.
Government and defense contracts are a different story. Many require the model and data to sit inside an accredited environment (FedRAMP, IL4/IL5, or fully air-gapped), where a public API is a non-starter regardless of any BAA. That's where self-hosting stops being optional.

Open-weight model quality, honestly

For narrow tasks, classification, extraction, structured triage, intent routing, an open-weight model fine-tuned or well-prompted on your data can match a frontier closed model, and you keep full control over latency and privacy. Meta's Llama family and Alibaba's Qwen models are both reasonable starting points for self-hosted deployments.

For open-ended reasoning, long documents, or anything where the failure mode is "subtly wrong answer that sounds confident," the frontier API models still have an edge. Don't self-host your way into a quality regression to save money on a task where the model's judgment is the product.

A practical decision path

Estimate real token volume, not requests. Multiply average tokens per request by daily request count. If you're under roughly 10 to 20M tokens/day, start with an API.
Check if a HIPAA/BAA-eligible endpoint from a major provider satisfies your compliance requirement. If yes, you likely don't need to self-host.
If neither cost nor compliance forces your hand, don't self-host. The ops overhead is a distraction from building product in the first 12 to 18 months of a company's life.
If you do self-host, budget for redundancy and a standing on-call rotation, not just the GPU line item.

Same logic that applies to picking between managed and bring-your-own-key providers in BYOK vs. managed LLM keys: the cheapest-looking option on paper is rarely the cheapest option once you count the operational load it puts on your team.

If you're trying to work out where your product actually falls on this line, let's talk.

Originally published on the Pykero blog.

How to Design Escalation Paths for AI Agents

Pykero — Sun, 09 Aug 2026 09:02:06 +0000

An AI agent should escalate to a human whenever it hits a pre-defined confidence boundary, touches money or an irreversible action, or sees an input pattern outside what it was built to handle. If you're not designing for that moment before you ship, you're finding out about it from an angry customer instead.

Why "just make it more accurate" isn't the fix

Most teams treat escalation as a failure mode to eliminate rather than a feature to build. The instinct is: if the agent is wrong sometimes, add more training examples, tune the prompt, add another retry. That works up to a point, then hits diminishing returns, because some fraction of real-world inputs are genuinely ambiguous. A refund request that references a policy exception. A support message in a dialect the model handles poorly. A sales lead whose intent doesn't match any of your qualification categories.

No amount of prompt engineering removes ambiguity from the world. What you can control is what the agent does when it encounters it: guess and hope, or stop and ask.

Guessing is cheap until it's wrong. And it compounds if the agent operates in a chain, since one bad guess several steps in can send everything downstream (see AI agents vs. workflows for why chains fail differently than single-call systems). An agent that escalates instead of guessing costs you a small amount of latency on the hard cases and nothing on the easy ones.

What actually triggers a good escalation

Three signals are worth building around, and they're mechanically different from each other:

Confidence threshold. A confidence signal doesn't have to be a probability score the model reports about itself. It can be something you compute externally, like how many concrete facts the agent actually extracted before it tries to act. That's the exact mechanism behind the outreach-agent gate described below: not "is the model sure," but "does it have enough material to work with." Below whatever floor you set, stop and route to a human instead of returning the best guess.
Stakes classification. Tag actions by blast radius before the agent runs, not after. Sending a templated email is low stakes. Issuing a refund on a request that cites a policy exception, or messaging a customer on something with legal implications, is high stakes. High-stakes actions get a lower autonomy threshold regardless of confidence, because being 90% sure isn't good enough when the 10% failure is expensive or irreversible.
Out-of-distribution detection. Log what the agent has actually seen in production. A support message in a dialect the model wasn't tuned on, or a sales lead whose intent doesn't match any qualification category you built for, are both out-of-distribution in the same way: the input looks nothing like your training or eval set, and that's a signal independent of how confident the model claims to be about its own answer.

None of these require a research team. They require you to decide, up front, what "I don't know" looks like for your specific agent, and to build a path for it that isn't a generic error message.

Build the queue before you need it

The most common mistake we see is agencies and in-house teams building the happy path first and bolting escalation on after a bad outcome forces the issue. By then it's reactive: a human is triaging a mess instead of catching it at the decision point.

Build the escalation surface as part of the initial scope:

A place for flagged cases to land. This can be as simple as a Slack channel with the full context: the raw input (the refund request with its policy citation, the dialect message, the off-taxonomy lead), what the agent tried, and why it flagged rather than acted. Don't make a human dig through logs to reconstruct what happened.
A clear resolution path. Someone needs to own responding to escalations within a defined window, or the queue becomes a graveyard and the agent's flags become pointless. If nobody answers, the customer experience is worse than if the agent had just guessed.
A feedback loop back into the agent. Every human resolution is training data. If the same category of case keeps escalating, like the same dialect or the same off-taxonomy lead type, that's a signal to either expand the agent's scope for that category or accept it'll always need a human and design the UX around that permanently.

A pattern from our own agent work

We run an outreach agent internally that scrapes each prospect's site and drafts one tailored email per company. Early versions tried to always produce a draft, even when the scraped page had almost no usable content, a thin "coming soon" site, or a page that was mostly navigation with no actual business description. Forcing a draft out of thin content produced generic, obviously-templated emails that hurt more than they helped.

The fix wasn't a better prompt. It was adding a check: if the extracted facts fell below a minimum threshold of specificity, the agent skips the draft and flags the company for a human to either research manually or drop from the list. That one gate improved the average quality of what actually got sent, because the agent stopped forcing output in cases where it had nothing good to say. The lesson generalizes: an agent that can say "I don't have enough to work with" is more useful than one that always produces something.

Where this fits in scoping a project

If you're evaluating a vendor or planning your own build, escalation design should show up in the initial architecture discussion, not as a change request after launch. Ask any agency pitching you an agent: what happens when it's wrong, and how does it know? If the answer is "we'll monitor it and fix issues as they come up," that's a maintenance cost you're signing up for indefinitely (see our breakdown of AI agent maintenance costs). If the answer includes a specific confidence mechanism and a defined human handoff, that's a team that's thought about failure, not just the demo.

It's also worth checking this alongside your broader risk posture. Escalation design overlaps with the same questions covered in an AI agent security checklist: what can this system do without a human in the loop, and who's accountable when it does something wrong.

The bottom line

Escalation isn't a fallback you add when things break. It's the mechanism that keeps things from breaking in the first place, by giving the agent a legitimate third option beyond "succeed" or "fail silently." Design it at the same time you design the happy path, tie it to concrete triggers (confidence, stakes, distribution shift), and staff the queue like it matters, because the cases that land there are, by definition, the ones your agent couldn't handle alone.

If you're scoping an agentic system and want a second opinion on where the escalation boundaries should sit, let's talk.

Originally published on the Pykero blog.

Do You Need an llms.txt File?

Pykero — Fri, 07 Aug 2026 09:01:30 +0000

llms.txt is a plain markdown file you put at yourdomain.com/llms.txt that summarizes your site for AI models. It's worth adding, it takes an afternoon, and it costs you nothing. But don't confuse it with an actual AI visibility strategy: no major AI product has confirmed it reads the file, and the things that really determine whether your site gets crawled and cited by AI answers are the same things that determine whether Google can crawl it.

Where llms.txt came from

The spec was proposed in late 2024 by Jeremy Howard at Answer.AI, modeled loosely on robots.txt: a single, predictable location where a site tells automated readers what matters. The idea is that a language model has a limited context window and can't (or shouldn't have to) crawl your entire site to answer a question about it, so you hand it a curated index instead: your product pages, your docs, your pricing, in one short file with links and one-line descriptions. The full spec is at llmstxt.org.

It's a reasonable idea. The problem is adoption. As more than one developer has pointed out publicly this year, nobody has confirmed their crawler actually fetches it. OpenAI, Anthropic, and Google have not published documentation committing to parse llms.txt as part of how ChatGPT, Claude, or Gemini answer questions about your product. That doesn't mean it's useless, it means you should size the investment to match the uncertainty.

What it does and doesn't do

What it does: gives you a clean, low-effort way to hand-curate what an AI system sees if it does decide to look. It costs nothing to maintain if your site doesn't change often. It's a reasonable hedge, the same way you'd add a sitemap.xml even though most of your traffic doesn't come from crawlers reading it directly.

What it doesn't do:

It's not a ranking signal for Google or any traditional search engine.
It doesn't override or supplement your actual page content, if the linked pages are JavaScript-rendered client-side with no server-rendered HTML, an AI crawler that does try to follow the links will hit the same wall a search engine crawler does. We've written about this tradeoff before in the context of server-side rendering and SEO.
It doesn't fix a slow, bloated site. If your Core Web Vitals are bad, an AI agent trying to fetch and parse your pages within a reasonable timeout will bail the same way a human does.

The decision framework

If you're a founder or CTO deciding whether to spend engineering time on this, here's the honest breakdown:

Worth doing

You have a docs site, developer product, or API that AI coding assistants (Claude Code, Cursor, Copilot) might reference when a user asks "how do I integrate X."
Your marketing site already has clean, server-rendered pages, so llms.txt is additive, not a patch over a broken foundation.
You can generate it once and regenerate it on deploy with a small script rather than hand-maintaining it forever.

Not worth prioritizing

Your core pages are client-side rendered with no SSR, meaning the links in your llms.txt point to content a crawler can't actually read anyway. Fix that first.
You're treating it as an SEO strategy. It isn't one. Google has not indicated llms.txt affects ranking at all.
You'd need to build custom tooling to keep it in sync with a fast-changing site. At that point the maintenance cost exceeds the unconfirmed upside.

What actually gets you cited by AI answers

If the real goal is "when someone asks an AI assistant about a problem we solve, we want to show up," the leverage is almost entirely in things that predate llms.txt:

Server-rendered, crawlable content. Every major AI crawler (GPTBot, ClaudeBot, PerplexityBot) behaves like a search crawler: it fetches HTML, and if your content only appears after a client-side fetch, it often sees an empty shell.
Clean semantic structure. Real headings, real paragraph text, schema.org markup where relevant. This is the same discipline that makes a page rank well and the same discipline that makes it easy for an LLM to extract a clean answer.
Fast, stable pages. Crawlers time out. A bloated bundle that takes eight seconds to become interactive gets skipped the same way a slow page gets a lower crawl budget from Google.

This lines up with something we've seen directly in our own tooling, not on the SEO side but on the consumption side. We run an outreach engine that scrapes each prospect's site with a self-hosted Firecrawl instance and a local model to draft a tailored email per company. The pattern that actually worked reliably was a single call that extracts the facts we need and drafts the email in one pass, not a multi-step chain that tries to crawl a sitemap, summarize each page, then synthesize. The lesson translates directly to llms.txt: a clean, well-structured page that an LLM can read in one pass beats a curated index pointing at pages the model still has to fight to parse. If you're weighing similar tradeoffs in your own AI features, we've written more on single-call versus agent-chain design and on RAG for founders if the underlying question is really "how do we make our content retrievable by an LLM," which is the same problem from a different angle.

How to actually implement it

If you've decided it's worth the afternoon:

List your 10-20 most important pages: product, pricing, docs, key integration guides.
Write one honest sentence per page, no marketing copy, just what's there.
Serve it as a static file at /llms.txt using the format from the spec (H1 title, blockquote summary, H2 sections with linked bullet points).
Regenerate it as part of your build if your page set changes often, don't let it go stale.
Move on. Don't build a dashboard for it.

llms.txt is a cheap insurance policy, not a strategy. Spend real effort on server-rendered, fast, well-structured pages, that's what both search engines and AI crawlers actually need to cite you, and it's work you should be doing regardless of whether any model ever reads your llms.txt file.

If you're trying to figure out whether your site or product is actually AI-crawlable, and not just checking a box, let's talk.

Originally published on the Pykero blog.

Why Average Latency Is the Wrong Metric for AI Agents

Pykero — Wed, 05 Aug 2026 09:02:45 +0000

Average response time is the wrong number to optimize for AI agents because it hides exactly the requests that break trust: the slow tool call, the retried LLM step, the request that timed out and silently fell back. Track p95 and p99 latency per step instead, and ask any vendor for the same before you sign a contract.

The problem with averaging

Say your agent responds in 800ms on average. That sounds fine. But if 90% of requests finish in 400ms and the remaining 10% take 6 seconds because they hit a retry, a rate limit, or a slow downstream API, the average buries the part of the distribution your users actually feel. Nobody experiences the average. They experience their own request, and for one in ten users, that request is 15x slower than what your dashboard implies.

This is not a new idea in distributed systems generally, it's why the Google SRE book treats percentile latency (p50, p95, p99) as the standard for monitoring, not the mean. AI agents make this worse than typical web services because the tail is fatter: LLM inference time is itself variable, tool calls can hang, and agents often chain multiple calls where one slow link stalls the whole request.

Where the tail comes from in agent systems

A single well-scoped LLM call has one source of latency variance: the model's inference time, plus network. An agent that chains steps, retrieve context, call a tool, call the model again, validate output, call the model a third time, multiplies that variance at every hop. Each step also carries its own failure and retry probability, and retries don't just add latency, they add it unevenly. A request that needs one retry on step 3 might take twice as long as one that sails through, and that unevenness is exactly what averages erase.

We saw this directly building our own outreach tool, which scrapes a prospect's site and drafts a tailored email. Early versions used a three-step chain: extract facts from the scraped page, draft the email, then refine it. The extraction step was the one at the mercy of someone else's server, some prospect sites loaded fast, some were slow or bloated with tracking scripts, and however long that page took to fetch and parse became a floor under everything after it, since drafting couldn't start until extraction finished. The average looked fine because most prospect sites are fast. The tail was rough because the slow sites weren't rare, they were just unevenly distributed, and each one produced a full-length stall that a fast average never showed. Collapsing extraction and drafting into a single well-designed call didn't make individual sites load faster, but it removed the sequential dependency, there was one less handoff for a slow fetch to block. If you're deciding between an agent chain and a single call for your own product, that tradeoff is worth thinking through before you build, see our breakdown of single-call vs agent chains.

What to measure instead

For any AI agent, whether you're building it or evaluating a vendor's, ask for these numbers broken out by step, not just for the request as a whole:

p50 (median): what a typical request feels like.
p95: what one in twenty users experiences. This is usually where the real product complaints start.
p99: your worst-case tail. For a support bot handling thousands of conversations a day, p99 is not an edge case, it's dozens of real conversations every day.
Time to first token vs. total completion time: if the agent streams output, users tolerate a slower total time much better than a slow start. Measure both separately.
Retry rate and where retries happen: a 2% retry rate on one step sounds small until it's the step every request depends on.

Tools like OpenTelemetry exist specifically so you can trace latency through each step of a distributed call, including agent chains, rather than only seeing the total. If your current stack (or a vendor's) can't show you per-step traces, that's itself a useful data point.

How this shows up in vendor conversations

If you're evaluating an outside team to build or operate an AI agent for you, latency claims are one of the easiest places for a pitch to overstate reality. "Sub-second responses" is a claim about the average, almost always, and it's exactly the kind of number that hid the stall in our own outreach tool until we broke it down by step. The right follow-up questions are:

What's the p95 and p99 under realistic concurrent load, not a single warm request in a demo?
Which step in the pipeline is slowest, and how does that change under load?
What happens when a downstream call (a database, a search index, another API, or in our case a prospect's own website) is slow? Does the agent degrade gracefully or hang?

These questions belong in the same conversation as pricing and support terms. We cover the rest of that evaluation, including questions that have nothing to do with speed, in our AI agent vendor evaluation checklist.

Latency and cost are the same conversation

Worth noting: the same chain that produces a bad p99 usually also produces a bad bill. Extra steps mean extra tokens, extra retries mean paying twice for the same work, and extra model calls mean paying for coordination overhead that adds no value to the output. If you're already auditing latency, audit spend at the same time. Our guide on LLM cost optimization walks through the same trimming exercise from the cost side, and in practice the fixes overlap: fewer, better-scoped calls beat more, smaller ones on both dimensions.

The practical takeaway

Don't let "it feels fast in the demo" or "average response time: 800ms" stand in for real measurement. Before you ship an agent, or before you sign off on one someone else built for you, get the p95 and p99 numbers under load, broken down by step. If nobody can produce those numbers, that's your answer about how much testing has actually happened.

If you're building an AI agent and want a second set of eyes on the architecture before latency becomes a production problem, let's talk.

Originally published on the Pykero blog.