DEV Community: Bry

Building a RAG Pipeline from Scratch: Embeddings, Retrieval, and Claude

Bry — Tue, 15 Sep 2026 15:00:00 +0000

Key Points

RAG (Retrieval-Augmented Generation) connects an LLM to your private data at query time — no fine-tuning, no retraining, no data leakage into model weights.
The pipeline has five stages: Ingest → Chunk → Embed → Store → Query (retrieve, augment, generate). Getting chunking and retrieval right matters more than which LLM you pick.
ChromaDB runs in-process for local dev with zero infrastructure — collection.add() to insert, collection.query() to retrieve. Production options include Pinecone and pgvector.
Common failure modes — chunks too large, no deduplication, no query expansion, skipping evaluation — are all avoidable. This article shows how.

Introduction

Large language models hallucinate when they don't know the answer. The standard fix — fine-tuning on your private data — costs tens of thousands of dollars, takes weeks, and produces a static model that goes stale as soon as your data changes. RAG solves a different problem: instead of baking knowledge into weights, it retrieves the relevant facts at query time and gives them to the model as context. The model answers from evidence, not memory. I've built RAG pipelines for internal knowledge bases and customer-facing chat tools — the architecture is the same whether you're indexing 500 internal docs or 500,000 support tickets; the parameters are what change.

This article builds a complete RAG pipeline in Python using ChromaDB for vector storage, sentence-transformers for local embeddings, and Claude for generation. You will understand each stage, why the design choices matter, and what breaks in production if you skip them. The full working implementation is in src/pipeline.py and src/main.py.

What RAG Is — and What It Replaces

Before writing code, align on why RAG exists.

Fine-tuning trains the model on your data. It is expensive (typically $5,000–$50,000+ for a production run), takes days to weeks, and produces a checkpoint that bakes in your data as of the training date. If your documentation changes next week, the model doesn't know. Fine-tuning is correct when you need the model to learn a style, a domain vocabulary, or a task format — not when you need it to answer questions from a document set.

Prompt-only (stuffing) puts your documents directly into the context window. It works for small document sets (tens of pages) but breaks on large corpora: context windows have limits, filling them with irrelevant text degrades answer quality, and at scale the cost per query becomes prohibitive.

RAG indexes your documents, retrieves only the chunks relevant to each query, and gives the model a focused context. It handles large corpora, stays current as documents are updated, and costs a fraction of fine-tuning. The tradeoff is a retrieval layer you have to build and maintain.

Approach	Data scale	Latency	Cost	Freshness
Fine-tuning	Any	Low (no retrieval)	High (training + inference)	Static — retrain to update
Prompt-only	Small (< 50 pages)	Low	Medium	Fresh
RAG	Large (any)	Medium (retrieval + LLM)	Low–Medium	Fresh (re-embed on update)

RAG Pipeline Architecture

A RAG pipeline has two sides: the ingest side (run once per document update) and the query side (run on every user request). Together they form five stages.

Diagram: Ingest side (left) runs document processing once. Query side (right) runs on every user request.

The five stages:

Ingest — load raw documents. Source can be text files, PDFs, database records, or API responses.
Chunk — split documents into pieces small enough to embed meaningfully. Chunk size is the most consequential parameter in the pipeline.
Embed — convert each chunk to a vector. The embedding model maps semantic meaning to a point in high-dimensional space.
Store — persist vectors (with their source text and metadata) in a vector database.
Query — embed the user's question, find the nearest chunks, inject them into a prompt, and generate an answer.

Chunking Strategies

Chunking is where most RAG pipelines fail. A bad chunking strategy produces irrelevant retrievals; irrelevant retrievals produce hallucinated answers. The model can't conjure information that wasn't in the retrieved chunks. In practice, I spend more time tuning chunk size and overlap than on any other pipeline parameter — getting it wrong is invisible until a user catches the model confidently citing the wrong passage.

Fixed-Size Chunking

Split every N characters or tokens. Simple to implement, ignores document structure.

# Fixed-size chunking — simple but blunt
def chunk_fixed(text: str, size: int = 400, overlap: int = 50) -> list[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = start + size
        chunks.append(text[start:end])
        start += size - overlap  # overlap preserves context at boundaries
    return chunks

The overlap matters: without it, a sentence split across two chunks retrieves each half separately and both halves are incomplete.

Recursive Chunking

Try splitting on paragraph breaks (\n\n), then line breaks (\n), then spaces. Each level is a fallback when the previous splitter still produces chunks over the target size. This respects document structure — paragraphs before sentences before words.

def _chunk(self, text: str, max_tokens: int = 400) -> list[str]:
    """Recursively split text into chunks, preferring natural boundaries.

    Args:
        text: The input text to split.
        max_tokens: Approximate maximum chunk size in tokens (1 token ≈ 4 chars).

    Returns:
        List of text chunks, each within the max_tokens limit.
    """
    max_chars = max_tokens * 4  # rough token → char estimate

    if len(text) <= max_chars:
        return [text.strip()] if text.strip() else []

    # Try splitting on paragraph breaks first, then newlines, then spaces
    for separator in ["\n\n", "\n", " "]:
        parts = text.split(separator)
        if len(parts) > 1:
            chunks: list[str] = []
            current = ""
            for part in parts:
                candidate = (current + separator + part).strip() if current else part.strip()
                if len(candidate) <= max_chars:
                    current = candidate
                else:
                    if current:
                        chunks.append(current)
                    # Recurse on oversized parts
                    if len(part) > max_chars:
                        chunks.extend(self._chunk(part, max_tokens))
                        current = ""
                    else:
                        current = part.strip()
            if current:
                chunks.append(current)
            return [c for c in chunks if c]

    # No separator found — hard split
    return [text[:max_chars].strip(), text[max_chars:].strip()]

Semantic Chunking

Group sentences by semantic similarity — a sentence joins the current chunk if its embedding is similar enough; otherwise it starts a new chunk. Produces the most coherent chunks but is slower (requires embedding every sentence during ingestion).

Diagram: Chunking strategy selection — recursive approach, falling back from paragraphs to lines to words.

Embeddings

An embedding model converts text into a dense vector — a list of floats (e.g., 384 dimensions for all-MiniLM-L6-v2) where similar texts produce similar vectors. Semantic similarity becomes geometric proximity.

Local Embeddings: `sentence-transformers`

sentence-transformers runs entirely locally — no API key, no per-token cost, no latency from network calls.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

texts = [
    "Python is a high-level programming language.",
    "Guido van Rossum created Python in 1991.",
    "The weather is sunny today.",
]

embeddings = model.encode(texts)  # shape: (3, 384)
print(embeddings.shape)           # (3, 384)

all-MiniLM-L6-v2 is 80MB on disk, encodes ~14,000 sentences per second on CPU, and produces 384-dimensional vectors. It is the right default for local development and prototyping.

API Embeddings

For production, Anthropic's and OpenAI's embedding APIs trade local cost for higher-quality vectors at scale. Use them when your retrieval accuracy on domain-specific content drops below acceptable thresholds.

Option	Model	Dims	Cost	Notes
`sentence-transformers`	`all-MiniLM-L6-v2`	384	Free (local)	Best for dev/prototyping
`sentence-transformers`	`all-mpnet-base-v2`	768	Free (local)	Higher quality, 3× slower
OpenAI API	`text-embedding-3-small`	1536	$0.02 / 1M tokens	Good price/quality tradeoff
Anthropic API	Voyage-3 (via Voyage AI)	1024	$0.06 / 1M tokens	SOTA quality for RAG

Keep your embedding model consistent between ingestion and query time. If you embed documents with all-MiniLM-L6-v2 and query with text-embedding-3-small, your similarity scores will be meaningless.

Vector Storage with ChromaDB

ChromaDB is an in-process vector database for Python. It requires no server, no Docker container, no cloud account — import it and use it.

import chromadb

# In-memory client — resets between runs (good for unit tests)
client = chromadb.Client()

# Persistent client — stores to disk (good for development)
client = chromadb.PersistentClient(path="./chroma_db")

collection = client.get_or_create_collection(name="my_docs")

# Add documents with pre-computed embeddings
collection.add(
    ids=["chunk_0", "chunk_1", "chunk_2"],
    embeddings=[[0.1, 0.2, ...], [0.3, 0.1, ...], [0.9, 0.1, ...]],
    documents=["Python is...", "Guido van Rossum...", "Weather is..."],
    metadatas=[{"source": "python_intro.txt"}, {"source": "python_intro.txt"}, {"source": "weather.txt"}],
)

# Query — returns top-3 nearest chunks
results = collection.query(
    query_embeddings=[[0.15, 0.18, ...]],
    n_results=3,
    include=["documents", "metadatas", "distances"],
)

for doc, meta, dist in zip(
    results["documents"][0],
    results["metadatas"][0],
    results["distances"][0],
):
    print(f"[{dist:.3f}] {meta['source']}: {doc[:80]}")

ChromaDB uses cosine similarity by default. Lower distance = more similar.

Choosing a Vector Database

Feature	ChromaDB	Pinecone	pgvector
Setup	Zero (in-process)	Managed cloud	Add extension to Postgres
Scale	Millions of vectors	Billions	Millions (with tuning)
Cost	Free	$70/month+ (managed)	Postgres hosting cost
Filtering	Metadata filters	Metadata filters	Full SQL WHERE clauses
Persistence	Local disk	Cloud-managed	Postgres storage
Best for	Local dev, prototypes	Production at scale	Teams already on Postgres

Start with ChromaDB locally — it removes all infrastructure friction during development, which is where you need to iterate fastest. I reach for pgvector over Pinecone when the team is already on Postgres: the operational overhead is near-zero and full SQL filtering eliminates an entire class of retrieval bugs that metadata-only filters can't handle.

Retrieval Strategies

Retrieval is not just "find the nearest vectors." Three strategies matter in practice.

Cosine Similarity Search

The default. Compute the cosine similarity between the query vector and every stored vector; return the top-k most similar chunks. Fast, well-understood, and works well when the query language matches the document language.

Maximal Marginal Relevance (MMR)

Standard top-k retrieval can return five chunks that all say the same thing — high similarity, low diversity. MMR trades some similarity for diversity: each additional chunk is selected to be both similar to the query and different from already-selected chunks.

ChromaDB does not implement MMR natively. Implement it post-retrieval:

def _mmr_rerank(
    query_embedding: list[float],
    candidate_embeddings: list[list[float]],
    candidate_docs: list[str],
    k: int = 5,
    lambda_: float = 0.5,
) -> list[str]:
    """Select top-k chunks using Maximal Marginal Relevance.

    Args:
        query_embedding: Embedded user query.
        candidate_embeddings: Embeddings of candidate chunks (over-fetch, e.g., top-20).
        candidate_docs: Text of each candidate chunk.
        k: Number of chunks to return.
        lambda_: Trade-off between relevance (1.0) and diversity (0.0).

    Returns:
        k chunks selected for both relevance and diversity.
    """
    import numpy as np

    q = np.array(query_embedding)
    cands = np.array(candidate_embeddings)

    # Cosine similarity: query vs candidates
    sim_to_query = (cands @ q) / (np.linalg.norm(cands, axis=1) * np.linalg.norm(q) + 1e-9)

    selected_indices: list[int] = []
    remaining = list(range(len(candidate_docs)))

    for _ in range(min(k, len(remaining))):
        if not selected_indices:
            # First pick: highest similarity to query
            best = max(remaining, key=lambda i: sim_to_query[i])
        else:
            # Subsequent picks: balance relevance vs redundancy
            selected_vecs = cands[selected_indices]
            scores = []
            for i in remaining:
                sim_q = sim_to_query[i]
                sim_selected = max(
                    (cands[i] @ selected_vecs[j]) /
                    (np.linalg.norm(cands[i]) * np.linalg.norm(selected_vecs[j]) + 1e-9)
                    for j in range(len(selected_vecs))
                )
                scores.append(lambda_ * sim_q - (1 - lambda_) * sim_selected)
            best = remaining[scores.index(max(scores))]

        selected_indices.append(best)
        remaining.remove(best)

    return [candidate_docs[i] for i in selected_indices]

Hybrid Search (BM25 + Semantic)

Semantic search handles paraphrase and synonymy well. BM25 (keyword search) handles exact terms — product codes, names, technical identifiers — better. Hybrid search runs both and combines scores (typically via Reciprocal Rank Fusion). Use hybrid when your documents contain precise identifiers that semantic search might miss. I default to MMR over top-k similarity the moment a domain has redundant content — policy documentation, API reference pages, and anything generated from a template will poison straight similarity retrieval with near-duplicate chunks every time.

Context Augmentation

Retrieved chunks are useful only if the prompt tells Claude how to use them. A weak prompt ("here are some documents, answer the question") produces weak answers. A well-structured prompt produces grounded, citable answers.

def _build_prompt(self, question: str, chunks: list[str]) -> str:
    """Format retrieved chunks and question into a grounded generation prompt.

    Args:
        question: The user's original question.
        chunks: Retrieved document chunks, ordered by relevance (most relevant first).

    Returns:
        A formatted prompt that instructs Claude to cite sources and acknowledge gaps.
    """
    context_block = "\n\n---\n\n".join(
        f"[Source {i + 1}]\n{chunk}" for i, chunk in enumerate(chunks)
    )

    return f"""You are a helpful assistant. Answer the question below using only the provided sources.

Rules:
- Cite which source(s) support each claim (e.g., "According to Source 2...").
- If the sources do not contain enough information to answer the question, say so explicitly.
- Do not use knowledge outside the provided sources.

Sources:
{context_block}

Question: {question}

Answer:"""

The three rules matter. Citing sources forces the model to ground claims in retrieved text rather than training data. Acknowledging gaps prevents confident-sounding hallucinations. Forbidding outside knowledge keeps the model from blending retrieved context with parametric knowledge unpredictably.

Generation with Claude

The query method ties everything together: embed the question, retrieve chunks, build the prompt, call Claude, and return the answer.

Diagram: Full query path from user question to Claude-generated answer.

import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY from environment

response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[{"role": "user", "content": prompt}],
)

# Check why generation stopped
match response.stop_reason:
    case "end_turn":
        answer = response.content[0].text
    case "max_tokens":
        # Response was cut off — increase max_tokens or reduce prompt length
        answer = response.content[0].text + "\n\n[Response truncated — max_tokens reached]"
    case _:
        answer = f"[Generation stopped: {response.stop_reason}]"

Always check stop_reason. end_turn means the model finished naturally. max_tokens means the answer was cut off — a common silent failure when retrieved chunks are large and the combined prompt + answer exceeds your token budget.

For cheap classification tasks within a RAG system (e.g., query routing, intent detection, relevance filtering), use claude-haiku-4-5-20251001 instead of claude-sonnet-4-6. Haiku is significantly faster and cheaper for short-context classification where generation quality is less critical.

The `RAGPipeline` Class

The full implementation encapsulates all stages into a single class with three public methods: ingest, query, and the private helpers.

Diagram: RAGPipeline class structure — three public methods, three private helpers.

The constructor initializes all three dependencies. Chunk and embed are private because callers should not need to call them directly — they are implementation details of ingest and query.

Evaluation

A RAG pipeline with no evaluation is a guess. Add measurement before shipping.

Three metrics from the RAGAS framework cover the core failure modes:

Metric	What it measures	How it catches failures
Faithfulness	Is every claim in the answer supported by a retrieved chunk?	Catches hallucinations — the model added facts not in context
Answer Relevancy	Does the answer actually address the question?	Catches topic drift — the model answered a different question
Context Recall	Did retrieval surface the chunks needed to answer?	Catches retrieval failures — the right chunks weren't found

Run evaluation on a golden dataset — 20–50 question/answer pairs you've manually verified. Automate it in CI so a configuration change that breaks retrieval doesn't silently ship.

# RAGAS evaluation — requires: pip install ragas
from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevancy, context_recall
from datasets import Dataset

eval_data = {
    "question": ["What is Python?", "Who created Python?"],
    "answer": ["Python is a programming language.", "Guido van Rossum."],
    "contexts": [
        ["Python is a high-level language..."],
        ["Guido van Rossum created Python in 1991..."],
    ],
    "ground_truth": ["Python is a high-level programming language.", "Guido van Rossum."],
}

result = evaluate(
    Dataset.from_dict(eval_data),
    metrics=[faithfulness, answer_relevancy, context_recall],
)
print(result)

Common Mistakes

Mistake 1: Chunks larger than 600 tokens drown the retrieval signal.
A 600-token chunk covers multiple topics. When you embed it, the vector averages over those topics and becomes less specific to any one of them. Similarity search returns chunks that are vaguely related to the query, not precisely relevant. Keep chunks under 400 tokens. If a passage requires more context, overlap consecutive chunks by 50–100 tokens rather than enlarging the chunk size.

Mistake 2: Not deduplicating similar chunks before storing.
If your corpus contains repeated passages (headers, boilerplate, legal disclaimers), those chunks will dominate retrieval — every query returns five variations of the same paragraph. Deduplicate before ingestion: compute a hash of each chunk's text and discard exact duplicates, then use a similarity threshold (cosine similarity > 0.95) to collapse near-duplicates. I've seen this sink a support-bot demo: the corpus was a product manual where the safety warning appeared verbatim on 40 of 200 pages — top-5 retrieval returned the warning for nearly every query, and the model dutifully answered "please refer to a qualified technician" regardless of what was asked.

Mistake 3: Embedding the raw query without expansion.
A user asking "how does Python handle memory?" may not use the same words as your documentation ("garbage collection", "reference counting", "memory management"). Query expansion generates alternative phrasings before embedding: original_query + hypothetical_answer_keywords. Hypothetical Document Embeddings (HyDE) generates a fake answer to the question and embeds that instead — the fake answer often matches document language better than the raw question.

Mistake 4: Skipping evaluation entirely.
Most RAG pipelines are shipped based on developer intuition ("the answers look right in testing"). Without a golden dataset and automated metrics, you won't know when a configuration change — a new chunk size, a different top-k, an updated embedding model — makes retrieval worse. Define your evaluation dataset before you tune parameters, not after. I've watched a team spend two weeks tuning chunk size upward because answers "felt" more complete — then discover via RAGAS that faithfulness had dropped from 0.89 to 0.71 because larger chunks were diluting the retrieved context with off-topic sentences.

Mistake 5: Using the same model for embedding and generation.
Anthropic's Claude models are generation models, not embedding models. Using a generation model's hidden states as embeddings produces inferior vectors compared to models trained specifically for semantic similarity (sentence-transformers, OpenAI's text-embedding-3-*, Voyage AI). Keep embedding and generation as separate concerns: sentence-transformers (or a dedicated embedding API) for vectors, Claude for generation.

Full Example

The complete implementation lives in two files:

src/pipeline.py — RAGPipeline class with all methods documented and implemented.
src/main.py — demo that ingests five documents about Python and runs three queries.

Setup:

python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install -r requirements.txt

cp .env.example .env
# Add your ANTHROPIC_API_KEY to .env

python src/main.py

Expected output:

Ingested 5 documents → 12 chunks stored.

Q: What is Python's GIL?
A: According to Source 1, Python's Global Interpreter Lock (GIL) is a mutex that protects access to Python objects, preventing multiple threads from executing Python bytecodes simultaneously...

Q: Who created Python and when?
A: According to Source 3, Python was created by Guido van Rossum and first released in 1991...

Q: What is list comprehension?
A: According to Source 2, list comprehension is a concise syntax for creating lists based on existing iterables...

Full source: GitHub link → ai-integration/rag-pipeline/ — see README for setup steps.

Conclusion

RAG is the right architecture when your LLM needs access to private, large, or frequently updated data. The implementation is not what makes or breaks it — the parameters are: chunk size, overlap, retrieval strategy, and whether you bother to measure any of it. Most pipelines that fail in production fail because they were tuned by intuition and never measured. Add the RAGAS evaluation step before you ship anything to users, keep your chunks under 400 tokens, and you will avoid the failure modes that make RAG look unreliable. The architecture isn't fragile — the shortcuts are.

Who Absorbs the Margin — Client, SIer, or the AI Vendor: Renegotiating SI Economics

Bry — Wed, 09 Sep 2026 15:00:00 +0000

Key Points

Every dollar an AI tool saves an SI on delivery cost is a dollar three different parties have a plausible claim to: the client (who's aware operating costs went down), the AI platform vendor (whose licensing terms increasingly shift toward capturing usage-based value), and the SI (whose judgment turned a raw capability into a delivered outcome).
Procurement teams are learning to sort renewals into three buckets: renew and absorb, renegotiate to consumption, or replace and build. Expect more SIs to get pushed into the second and third buckets as clients understand AI has genuinely lowered the SI's own cost base.
93% of sellers reported struggling to quantify and defend the value they'd actually added, which puts most SIs into these renegotiations without a real number to counter a client's discount demand.
SIs that come through this pressure well are the ones that stop treating margin as something to defend by obscuring cost structure, and start treating it as something to defend by pricing a specific, demonstrable judgment contribution the client couldn't get from the AI platform directly.

Introduction

A client's CFO said something to me directly not long ago that a lot of SI account teams are starting to hear some version of: "I know what your AI licensing costs. I know what it used to cost you to deliver this. Explain to me why I'm still paying the old number." It's not an unreasonable question, and at this point in the AI-services transition, most SIs don't have a rehearsed answer for it. For a while, plenty of firms have been quietly pocketing the AI-driven cost reduction as pure margin, hoping nobody with enough visibility into the underlying economics would ask.

The honest framing is that AI-driven delivery savings don't belong to any one party by default. They're contested, three ways. Clients have a real claim: operational costs genuinely came down, and clients increasingly know it, because AI tool pricing itself became public and metered rather than opaque. AI platform vendors have a claim too, since a growing share of enterprise software licensing is shifting toward capturing usage-based value directly, rather than just charging a flat seat fee and letting downstream margin sit wherever it lands. And the SI has a claim, the one most vulnerable to getting argued away in a renegotiation: the actual judgment, integration work, and delivery risk absorbed in turning a raw AI capability into something a client can actually rely on in production.

Procurement organizations are getting sophisticated about this fast. A fairly standard renewal framework is already sorting vendor relationships into three buckets: renew and absorb (accept the current pricing, usually for low-volatility, well-understood work), renegotiate to consumption (move rigid seat- or hour-based pricing to a usage-based model that tracks real utilization), or replace and build (walk away from the vendor relationship entirely and bring the capability in-house, now that AI has made that a realistic option for more capabilities than it used to be). SIs that show up to these conversations without a defensible, quantified account of their own value contribution are getting sorted into renegotiate-to-consumption or replace-and-build far more often than SIs that can show precisely what they still add. A Holden Advisors study found 93% of sellers admitted they struggled to accurately quantify and defend their value, which means most SIs are walking into these renegotiations structurally unprepared for a client who's done their homework.

Where the Margin Actually Goes: Three Claims on the Same Savings

Claimant	Basis for the Claim	What Wins the Argument in Practice
Client	Aware that AI genuinely lowered the vendor's delivery cost; expects some pass-through	A vendor with a metered, transparent cost basis wins credibility; one hiding behind an unchanged invoice loses it fast
AI platform vendor	Owns the underlying capability; increasingly prices on usage rather than flat seat fees	Whatever margin the platform vendor's own licensing terms capture directly. This claim is largely settled by contract, not negotiation
Systems integrator	Provided judgment, integration, delivery risk absorption, and a working result the client couldn't have gotten by licensing the AI tool directly	A quantified account of that judgment, specific defect rates avoided, integration complexity resolved, risk absorbed, wins. A vague "we add value" claim loses

Recommendation: don't walk into a renewal conversation assuming your margin is safe because it always has been. Assume the client already has a rough estimate of your AI-driven cost reduction, and come with a specific, demonstrable account of what you add beyond the tool itself, or expect to get renegotiated into the consumption bucket.

Defending Margin With a Real Number Instead of a Vague Claim

Quantify your own AI-driven cost reduction honestly, before the client does it for you. If you know the number, you control how it's framed. If the client calculates it independently and you're caught unprepared, you're negotiating from a defensive position for the rest of the relationship.
Separate "what the AI platform did" from "what we did with it," in terms specific enough to price. Defect rates avoided, integration issues resolved that the AI output alone would have missed, delivery risk actually absorbed: name the specific contribution, not a general appeal to expertise.
Offer the renegotiation before the client demands it. SIs that proactively bring a fair, transparent pricing update to a renewal conversation are keeping far more of their margin than SIs that wait to be confronted, because proactive framing reads as partnership and reactive framing reads as having been caught.
Know which of your engagements are genuinely replace-and-build candidates, and don't fight that battle. Some capabilities really have become cheap enough for a client to bring in-house. Contesting that reality burns credibility you'll need for the engagements where your judgment genuinely isn't replaceable.

Questions to Ask Your Team

Do we know our own AI-driven delivery cost reduction, on our top client relationships, well enough to defend a number if the client brings one first?
Can we name, specifically, what we add beyond the AI platform's raw output on our current engagements, or would we be making a vague appeal to "expertise" in a real renegotiation?
Are we proactively bringing pricing conversations to clients as AI changes our cost base, or waiting for a CFO to ask the question first?
Which of our current engagements are honest replace-and-build candidates for the client, and are we prepared for that conversation instead of resisting it?

Conclusion

There's no default answer to who keeps the margin AI creates. It depends on who shows up to the renegotiation with a real number. Clients who understand the underlying cost shift will ask for a share of it, and they're not wrong to. AI platform vendors are already capturing their share through licensing terms most SIs don't control. What's actually contestable is the SI's own piece, and 93% of sellers walking into that conversation without a quantified value story is exactly why so much margin is likely to get renegotiated away from firms that could have kept more of it. The ones that hold their ground will be the ones treating their judgment as a line item with a number attached, not an assumption everyone keeps taking on faith.

Amazon SQS vs SNS: Queues, Fan-Out, and Picking the Right One from the CLI

Bry — Tue, 08 Sep 2026 15:00:00 +0000

Key Points

SQS is a queue — one message, delivered to one consumer, held until processed. SNS is pub/sub — one message, fanned out to every subscriber. The fan-out pattern combining both is the standard, not an either/or choice.
SNS-to-SQS delivery requires the queue's access policy to explicitly allow the topic to call SendMessage — skip this and messages vanish with no error on the publish side.
FIFO queues cap at 3,000 messages/second batched (300/s unbatched) unless you enable high-throughput mode, which raises that to 70,000/s. Both SQS and SNS also bill in 64 KB payload chunks, same mechanic as EventBridge.

Prerequisites

CLI/SDK version tested against: aws-cli/2.35.x
An IAM role with sqs:* and sns:* permissions
jq installed for parsing CLI JSON output in the examples below

Introduction

I still get asked "should I use SQS or SNS?" as though it's a fork in the road. Most of the time the honest answer is both, wired together — SNS decides who gets notified, SQS makes sure each of them actually gets to process the message at their own pace without losing it if they're briefly unavailable. Treating them as competing choices is how teams end up building a fan-out mechanism inside application code that SNS already does for free.

The part that trips people up isn't the concept. It's the access policy. Subscribe an SQS queue to an SNS topic without granting the topic permission to send to that queue, and the subscription succeeds, the topic publish succeeds, and the message simply never arrives — no error surfaces anywhere in that chain. I've debugged this exact silent failure in a client's fan-out pipeline, and it's the single most common gap in CLI tutorials for this pattern.

This article builds a standalone queue, a standalone topic, and the full fan-out pattern with the access policy step included, then covers where FIFO changes the throughput math.

Queue vs Topic, Conceptually

Diagram: SQS delivers each message to exactly one consumer; SNS delivers a copy to every subscriber.

A message sitting in an SQS queue waits until something polls for it. A message published to an SNS topic is pushed immediately to every current subscriber — there's no "waiting" concept on the topic itself, which is exactly why SNS alone is a poor fit for a subscriber that might be temporarily down. That's what the fan-out pattern fixes.

Standalone SQS

# Standard queue — no ordering guarantee, near-unlimited throughput.
aws sqs create-queue --queue-name orders-standard-queue

QUEUE_URL=$(aws sqs get-queue-url --queue-name orders-standard-queue --query QueueUrl --output text)

aws sqs send-message \
  --queue-url "$QUEUE_URL" \
  --message-body '{"orderId":"order-abc123","status":"pending"}'

aws sqs receive-message --queue-url "$QUEUE_URL" --max-number-of-messages 1

# FIFO queue — strict ordering within a message group, capped throughput.
aws sqs create-queue \
  --queue-name orders-fifo-queue.fifo \
  --attributes '{"FifoQueue":"true","ContentBasedDeduplication":"true"}'

Queue type is permanent. There's no update-queue-type command — you create a new queue and migrate, you don't convert an existing one. Decide standard vs FIFO before you have production traffic depending on the answer.

Standalone SNS

aws sns create-topic --name orders-topic

TOPIC_ARN=$(aws sns list-topics --query "Topics[?contains(TopicArn,'orders-topic')].TopicArn" --output text)

aws sns publish \
  --topic-arn "$TOPIC_ARN" \
  --message '{"orderId":"order-abc123","status":"pending"}' \
  --subject "Order Created"

That publish immediately pushes to every current subscriber — email, SMS, HTTPS endpoint, Lambda, or SQS. Nothing is retained after delivery attempts complete. If you need durability — a subscriber that's slow or briefly offline shouldn't lose the message — that's the case for combining SNS with SQS, not using SNS alone.

The Fan-Out Pattern — Including the Step Everyone Skips

Diagram: fan-out delivery depends entirely on each queue's access policy explicitly trusting the topic — there's no other permission gate in this chain.

aws sqs create-queue --queue-name orders-notifications-queue
QUEUE_URL=$(aws sqs get-queue-url --queue-name orders-notifications-queue --query QueueUrl --output text)
QUEUE_ARN=$(aws sqs get-queue-attributes --queue-url "$QUEUE_URL" --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)

# This is the step that gets skipped. Without it, the subscription
# below will succeed and messages will vanish with zero errors.
aws sqs set-queue-attributes \
  --queue-url "$QUEUE_URL" \
  --attributes "{\"Policy\":\"{\\\"Version\\\":\\\"2012-10-17\\\",\\\"Statement\\\":[{\\\"Effect\\\":\\\"Allow\\\",\\\"Principal\\\":{\\\"Service\\\":\\\"sns.amazonaws.com\\\"},\\\"Action\\\":\\\"sqs:SendMessage\\\",\\\"Resource\\\":\\\"${QUEUE_ARN}\\\",\\\"Condition\\\":{\\\"ArnEquals\\\":{\\\"aws:SourceArn\\\":\\\"${TOPIC_ARN}\\\"}}}]}\"}"

aws sns subscribe \
  --topic-arn "$TOPIC_ARN" \
  --protocol sqs \
  --notification-endpoint "$QUEUE_ARN" \
  --attributes RawMessageDelivery=true

RawMessageDelivery=true matters as much as the policy step. Without it, the queue receives SNS's full JSON envelope — Type, MessageId, TopicArn, and the actual message nested inside a Message string field — instead of your original payload directly. Consumers that expect the raw body will parse the wrong structure and either error out or silently read undefined fields.

FIFO Throughput Limits

Configuration	Throughput ceiling
FIFO queue, batched	3,000 messages/second
FIFO queue, unbatched	300 messages/second
FIFO queue, high-throughput mode	70,000 messages/second
SNS FIFO topic	3,000 messages/second or 20 MB/second, whichever is hit first
Standard queue/topic	No published ceiling

High-throughput mode isn't automatic — it's an explicit setting (DeduplicationScope and FifoThroughputLimit attributes set to messageGroup and perMessageGroupId respectively) that trades a small amount of strict cross-group ordering guarantee for the throughput increase. If you need strict global ordering across all message groups, don't enable it. If your ordering requirement is per-customer or per-order (a common case), high-throughput mode is almost always the right call.

Common Mistakes

Mistake 1: Subscribing SQS to SNS without a queue access policy
The subscription API call succeeds regardless. The failure is invisible until someone notices messages aren't arriving — which could be minutes or weeks later depending on traffic patterns.

Mistake 2: Forgetting RawMessageDelivery
Consumers built against the raw payload shape break silently or throw confusing parsing errors when they receive SNS's wrapped envelope instead.

Mistake 3: Choosing FIFO by default "to be safe"
FIFO's throughput ceiling and stricter deduplication requirements are a real cost. Most systems don't actually need strict ordering — verify the requirement before paying for it in complexity and throughput headroom.

Mistake 4: Trying to convert a standard queue to FIFO
There's no such command. You create a new FIFO queue and migrate producers and consumers to it — plan for that as a deploy, not a config change.

Production Considerations

Performance: Long polling (--wait-time-seconds 20 on receive-message) eliminates the empty-receive charges that pile up from aggressive short polling — this alone is one of the two biggest SQS cost levers, the other being batching sends and receives.

Security: Scope queue access policies to the specific topic ARN, not a wildcard principal. The AWS Tip source in this article's research log describes a real incident where a queue policy pointed at a stale topic ARN after a topic recreation — silent failure, same root cause as skipping the policy entirely.

Cost: Both services bill in 64 KB payload chunks. A consistently large message body (nested JSON, embedded metadata) multiplies your bill the same way it does on EventBridge — trim payloads, pass references where you can.

Monitoring: Alarm on ApproximateAgeOfOldestMessage for SQS queues — a rising value means consumers aren't keeping up, well before the queue depth itself looks alarming.

Full Example: Fan-Out Setup Script

#!/usr/bin/env bash
set -euo pipefail

TOPIC_NAME="${TOPIC_NAME:-orders-topic}"
QUEUE_NAME="${QUEUE_NAME:-orders-notifications-queue}"

aws sns create-topic --name "$TOPIC_NAME" >/dev/null
TOPIC_ARN=$(aws sns list-topics --query "Topics[?contains(TopicArn,'${TOPIC_NAME}')].TopicArn" --output text)

aws sqs create-queue --queue-name "$QUEUE_NAME" >/dev/null
QUEUE_URL=$(aws sqs get-queue-url --queue-name "$QUEUE_NAME" --query QueueUrl --output text)
QUEUE_ARN=$(aws sqs get-queue-attributes --queue-url "$QUEUE_URL" --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)

POLICY=$(cat << EOF
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"sns.amazonaws.com"},"Action":"sqs:SendMessage","Resource":"${QUEUE_ARN}","Condition":{"ArnEquals":{"aws:SourceArn":"${TOPIC_ARN}"}}}]}
EOF
)

aws sqs set-queue-attributes --queue-url "$QUEUE_URL" --attributes "{\"Policy\":$(echo "$POLICY" | jq -Rs .)}"
aws sns subscribe --topic-arn "$TOPIC_ARN" --protocol sqs --notification-endpoint "$QUEUE_ARN" --attributes RawMessageDelivery=true

echo "Fan-out ready: $TOPIC_NAME -> $QUEUE_NAME"

Full source including a batched consumer with long polling: GitHub → cloud-apis/amazon-sqs-vs-sns-cli/

Conclusion

Stop framing SQS and SNS as competing choices — SNS decides who hears about something, SQS makes sure each listener actually gets to act on it without losing the message if they're briefly unavailable. The fan-out pattern combining both is the default for a reason. The one step that will actually cost you debugging time if skipped is the queue access policy: get it wrong and everything upstream reports success while the message goes nowhere.

The Multi-Tier Subcontracting Pyramid Under Pressure: What Happens to Body-Shop SIers When AI Writes the Code

Bry — Fri, 04 Sep 2026 13:30:00 +0000

Key Points

Japan's tajuu-shitauke (multi-layer subcontracting) structure routes enterprise IT work from a prime contractor (motoke) down through one or more subcontractor tiers to the engineers who actually write and test the code, a pattern that's persisted, largely unchanged, since the mainframe era.
Each tier historically takes its margin off the top before passing reduced-price work downward. A client-paid ¥1,000,000 monthly rate can leave the engineer actually doing the work with roughly ¥350,000 to ¥400,000 after cascading through two or three intermediary layers.
AI-assisted coding tools compress exactly the layer this structure depends on most: routine implementation, testing, and documentation work performed by second- and third-tier subcontractors and individual SES engineers.
The pyramid's top (client relationships, architecture, governance) looks likely to hold up. Its middle and bottom, firms and engineers whose entire value proposition is executing defined tasks at a lower price than the tier above, are facing the sharpest margin compression in the industry.
Japan's 2026 Subcontract Act reform, which expanded transparency requirements to creative and consulting services, gives lower-tier firms a real legal lever to push back on unfair pass-through pricing right as AI is already reshaping what "fair" pricing even means.

Introduction

A subcontractor relationship I've watched come under real strain recently is a fairly typical version of what's starting to happen across the industry. This is a second-tier firm that built a stable, unglamorous business supplying detailed-design and coding-and-testing engineers to a prime contractor's enterprise projects, at a rate that undercuts the tier above it. That's the entire business model: execute defined tasks, reliably, for less than the next tier up would charge. As the prime contractor's own AI-assisted tooling starts producing that same detailed-design and coding output directly, at a fraction of the cost and turnaround time, there's no lower price the second-tier firm can offer to stay competitive. It isn't underpriced. It's becoming structurally obsolete.

The pyramid this firm sits inside is one of the most distinctive features of Japan's IT industry. The prime contractor, or motoke, wins the client relationship and the overall project, then delegates substantial portions of detailed design, implementation, integration testing, and commissioning to one or more subcontractor tiers below it. Prime contractors capture the largest margins, commonly cited around 30-40%. First-tier subcontractors run 20-30%. Second-tier and below drop to 10-15%. Individual engineers, often working under System Engineering Service (SES) staffing arrangements, sit at the bottom, with compensation shrinking at each layer above them. A documented rate cascade shows the mechanism concretely: a client paying roughly ¥1,000,000 a month for an engineer's time can leave that engineer with something in the range of ¥350,000 to ¥400,000 after the intermediary tiers each take their cut.

This structure has survived for decades because it solved a real coordination problem: large enterprise projects needed a way to scale headcount up and down without every firm in the chain carrying the client relationship or the delivery risk directly. AI-assisted coding isn't attacking that coordination function. It's attacking the thing the lower tiers are actually selling, person-hours of implementation and testing labor, priced progressively lower the further down the pyramid you look. As a prime contractor, or increasingly the client itself, can generate a larger share of that same output directly, the lower tiers' entire value proposition, execute cheaply, stops being a viable business on its own.

Pyramid Layer Economics Under AI Compression

graph TD
    Client[Client] -->|pays full rate| Prime[Prime Contractor Motoke]
    Prime -->|passes down reduced rate, keeps 30 to 40 percent margin| Tier1[First-Tier Subcontractor]
    Tier1 -->|passes down reduced rate, keeps 20 to 30 percent margin| Tier2[Second-Tier Subcontractor]
    Tier2 -->|passes down reduced rate, keeps 10 to 15 percent margin| SES[Individual Engineer SES]
    AI[AI-Assisted Coding Tooling] -.->|replaces routine output of| Tier2
    AI -.->|replaces routine output of| SES

Layer	Historical Role	Historical Margin	Effect of AI-Assisted Coding
Client	Pays full contracted rate	N/A	Increasingly aware of the gap between what AI can produce and what's still being billed for
Prime contractor (motoke)	Client relationship, architecture, governance, overall delivery risk	~30-40%	Largely intact so far. Judgment and relationship work AI doesn't replace
First-tier subcontractor	Mid-scale delivery management, some architecture	~20-30%	Under pressure, but retains value where it manages integration complexity AI can't own alone
Second-tier subcontractor and below	Detailed design, coding, testing execution	~10-15%	Hardest hit. This is exactly the work AI-assisted tooling compresses
Individual engineer (SES)	Task-level implementation	Remainder after cascading cuts	Displaced where the task is routine; in higher demand where verification and AI-output review are the actual job

Recommendation: if your firm's position in this chain has always been "we execute the same task the tier above us does, for less," that position won't survive AI-assisted coding regardless of how thin you cut your margin further. The layers most likely to survive are the ones pricing judgment, integration complexity, or client trust, not marginal labor cost.

What's Likely to Survive, and What a Lower-Tier Firm Should Change Now

Firms moving up the value chain, not just down on price. The subcontractors most likely to survive won't compete on being cheaper than the tier above. They're taking on integration and governance responsibility that used to sit with the prime contractor, becoming harder to replace with AI output alone.
Firms specializing in verifying AI-generated output, not just producing more of it. Reviewing and validating AI-generated code against a client's actual production constraints is turning out to require exactly the kind of experienced-engineer judgment that routine implementation work didn't.
Firms using Japan's 2026 Subcontract Act reform actively. The reform's expanded transparency requirements give lower-tier firms real standing to contest unfair pass-through pricing. Firms that understand and use this leverage should renegotiate from a stronger position than firms that don't know the reform applies to them.
Firms consolidating instead of competing on the same shrinking margin. Some second- and third-tier firms are merging capabilities, combining a compressed coding-and-testing practice with a smaller firm's domain expertise, to offer something closer to the first-tier's integration value than to commodity execution.
Firms that change nothing. These are the most exposed to winding down, getting acquired for their client relationships and remaining engineers, or shrinking into a much smaller commodity-execution niche with correspondingly thinner margins.

Questions to Ask Your Team

Where does our firm actually sit in our clients' delivery chains, and is our value proposition still "we execute this cheaper than the tier above," or has it genuinely moved to something AI-assisted tooling can't produce directly?
If a prime contractor above us started generating our layer's output directly with AI tooling, what would we still have left to sell them?
Are we aware of what Japan's 2026 Subcontract Act reform actually changed for transparency and fair pricing in multi-tier arrangements, and have we used it in a renegotiation?
Have we tested whether our engineers are more valuable reviewing and validating AI-generated output than producing new output themselves, and are we pricing that shift yet?

Conclusion

The multi-tier subcontracting pyramid isn't collapsing, and it's not going to. What it built its lower tiers on, the assumption that there's always a cheaper price for the same routine task one layer further down, is what's breaking. Firms most likely to weather the pressure are moving toward judgment, integration, and verification work. Firms trying to survive by cutting their price further into an already-thin margin are the most exposed. The prime-contractor layer looks likely to hold up fine, because it was never really selling person-hours in the first place. Everyone underneath it needs to figure out, quickly, whether they've been selling person-hours the whole time, and if so, what else they actually have.

AI Cost Optimization: A Business Guide to LLM API Budgeting

Bry — Tue, 01 Sep 2026 15:00:00 +0000

Key Points

LLM API costs double every six months for most growing organizations — model selection alone can reduce your bill by 50–80% without changing what you build.
Prompt caching and response caching together eliminate the majority of repeated compute costs, saving 60–90% on input tokens for the right workloads.
Batch processing cuts per-token prices by 50% for any task that doesn't require a real-time answer.
Visibility comes first: you cannot optimize spend you cannot see — budget alerts, cost attribution by feature, and a 4-phase adoption roadmap make savings stick.

The AI Bill Nobody Expected

The CFO opens the cloud invoice. Line 37: LLM API usage — $48,000. Last month it was $22,000. Three months ago it was $4,000.

This is not a hypothetical. Enterprise LLM API spending doubled in under six months as teams moved from experiments to production workloads, and most organizations had no framework in place to manage the acceleration. Unlike compute or storage — where costs scale predictably with users — AI API costs are shaped by choices made at the code level: which model you pick, how you structure requests, whether you cache responses, and whether you batch non-urgent jobs. Those choices are invisible to finance and often undiscussed with engineering leadership.

I've watched this play out across organizations at different scales — from startups where one engineer's prototype quietly became a $30K/month production workload, to larger teams where cost attribution was a year-long engineering project after the fact. This guide gives business leaders — CTOs, product managers, and anyone signing cloud invoices — a clear, non-technical framework for getting AI spend under control without sacrificing the capabilities your teams depend on.

Why AI Costs Spiral

The pattern is consistent across organizations of every size: costs spiral because of three compounding defaults.

Teams default to the most powerful model. When engineers prototype an AI feature, they reach for the frontier model — it produces the best output, reduces debugging time, and avoids internal debates about quality. That default rarely gets revisited after launch. The same flagship model handling nuanced legal analysis ends up powering an FAQ bot that answers "What are your business hours?"

Nobody tracks token usage by feature. Unlike server costs that map neatly to infrastructure, LLM costs are buried in a single API line item. Without attribution — which feature consumed how many tokens — there is no feedback loop. Expensive patterns persist indefinitely because nobody can see them.

Users trigger expensive chains without limits. In production, one user action can silently trigger a cascade of API calls: an initial query, a summarization step, a classification call, a response synthesis step. Each hop multiplies the token count. Without circuit breakers or cost caps, a single power user can consume more than the entire intended monthly budget in a week. I've seen a five-step document analysis pipeline — perfectly reasonable in isolation — go uncapped in production and rack up more in a weekend than the team had budgeted for the entire month, because one user stress-tested it with 400-page PDFs.

How LLM Pricing Works

AI APIs charge by the token — a chunk of text roughly equivalent to 0.75 words in English. A 100-word paragraph is approximately 130 tokens. Two separate charges apply to every request: input tokens (what you send to the model) and output tokens (what the model generates in response).

Output tokens are significantly more expensive. Across all major providers, output pricing runs 4–5 times higher per token than input pricing. This matters because it means that asking a model to write a long report costs far more than asking it to classify a sentence — even if both requests contain the same amount of input text.

Here is the current pricing landscape for the major models your engineering team is most likely using:

Model	Input $/1M tokens	Output $/1M tokens	Context	Best For
Claude Opus 4.8	$5.00	$25.00	200K	Complex reasoning, multi-step analysis, legal/financial review
Claude Sonnet 4.6	$3.00	$15.00	200K	Balanced quality and cost; most production workloads
Claude Haiku 4.5	$1.00	$5.00	200K	High-volume, simple tasks: classification, extraction, FAQ
GPT-4o	$2.50	$10.00	128K	General-purpose; strong tool-use and structured output
Gemini 2.5 Pro	$1.25	$10.00	1M	Very long documents; high-context summarization

Pricing as of June 2026. Verify current rates at official provider pricing pages before committing to budget projections.

Context window refers to the maximum amount of text a model can consider in a single request. Larger context windows are critical for processing lengthy contracts, codebases, or conversation histories — but they also increase token costs proportionally.

The Biggest Lever: Model Selection

If you take one action from this guide, make it this: match task complexity to model tier.

The cost gap between tiers is not incremental — it is multiplicative. Using Claude Opus for a simple classification task costs roughly 20 times more per request than using Claude Haiku for the same task. At scale, that ratio becomes a six-figure annual line item for a mid-sized product.

The business principle is straightforward: the model's capability should match the task's demand. Sending a routine FAQ to your most powerful model is the equivalent of deploying a senior partner to answer reception phone calls.

Use this decision framework when evaluating which model tier a task belongs in:

My recommendation: start the audit with your highest-volume features, not your most complex ones. FAQ responses, form classification, data extraction, and templated summaries almost always belong in the Haiku tier — and those tend to be your volume leaders. Complex document analysis, strategic synthesis, and nuanced content generation belong in the Sonnet or Opus tier, but they're rarely the source of the runaway bill.

Prompt Caching: Pay Once, Reuse Many Times

Every AI application sends instructions to the model with every request. A customer service bot might include a 2,000-word system prompt describing the company's products, policies, and tone guidelines — and that prompt gets sent, and charged, with every single user message.

Prompt caching solves this. When you send the same instructions repeatedly, the provider stores that content in a temporary cache. Subsequent requests that use the same cached prefix are charged at 10% of the normal input price — a 90% discount on those tokens.

For applications with long, stable system prompts — support bots, document processors, product assistants — prompt caching typically reduces input token costs by 60–90%. The cached content stays valid for a configurable duration (5 minutes or 1 hour on Anthropic's API) and is automatically refreshed when accessed.

From a business perspective: if your engineering team is not using prompt caching on any application that includes a system prompt longer than a few hundred words, you are paying full price for tokens you have already paid for. In practice, this is the first thing I check when a team tells me their AI costs feel out of control — it is almost always uncached, and enabling it is usually a one-day engineering task that pays for itself within the first billing cycle.

Response Caching: Skip the API Call Entirely

Prompt caching operates at the provider level and reduces the cost of repeated inputs. Response caching operates at your application level and eliminates the API call entirely for repeated questions.

The concept is simple: when a user asks a question, your application checks a local database before calling the AI API. If the same question has been asked before and the answer is still valid, return the stored answer. No tokens consumed, no API cost, no latency.

This approach works well for:

FAQ bots: The same 200 questions account for 80% of support volume in most businesses
Product descriptions: Thousands of users view the same product content
Templated reports: Weekly summaries generated from the same data schema

It does not work well for:

Real-time data requests: Questions about live inventory, current prices, or today's metrics require fresh API calls
User-specific personalization: Responses that incorporate individual user history or preferences cannot be safely reused across users

The infrastructure investment is modest — a simple key-value store or database table — and the return on high-repetition workloads is significant. I use response caching as the first conversation to have with any team running a support bot or product FAQ: the ROI calculation is fast, the engineering effort is low, and approval is easy when you can show finance that 60% of calls will cost nothing after day three. On FAQ-heavy applications, organizations typically see 40–70% of requests served from cache after the first few days of production traffic.

Batch Processing: Trade Speed for 50% Off

Real-time API calls — where your application waits for an immediate response — carry a premium price. For tasks where a response is not needed within seconds, every major provider offers a batch processing API at 50% off standard pricing.

Anthropic's Batch API and OpenAI's Batch API both accept large volumes of requests submitted at once and return results within hours, typically overnight. The same models, the same quality — at half the cost.

Batch processing is well-suited for:

Overnight report generation
Bulk document classification or extraction
Weekly content summarization jobs
Training data generation or quality review
Large-scale sentiment analysis

The key trade-off is latency. Batch jobs are asynchronous — you submit the work and retrieve results later. For any workflow where a user is waiting for a response, batch processing is not appropriate. For workflows that run on a schedule or process accumulated data, it is one of the simplest cost reductions available.

A team spending $10,000 per month on overnight AI processing jobs that currently use the real-time API can reduce that line item to $5,000 with a single architectural change. If your organization runs any scheduled AI jobs — weekly digests, monthly classification sweeps, overnight data enrichment — batch processing should be a standing agenda item in your next budget review: the savings are predictable, the risk is low, and the approval case is straightforward.

Budget Controls and Monitoring

Cost optimization techniques only work if you can see whether they are working. Visibility is the prerequisite for everything else.

Set spend alerts in provider dashboards. Both the Anthropic Console and the OpenAI usage dashboard allow you to configure email alerts when monthly spend crosses a threshold. Set alerts at 50%, 75%, and 100% of your planned monthly budget — not just at the limit. Early warning gives engineering time to investigate before costs become a crisis.

Implement soft limits in application code. Provider-level alerts fire after costs have already accumulated. Application-level limits stop the accumulation. Work with your engineering team to implement per-user, per-feature, and per-workflow token budgets. When a workflow hits its limit, it either degrades gracefully (using a cheaper model) or queues the request for batch processing rather than failing loudly.

Track cost per feature, per user, per workflow. A single API line item tells you nothing actionable. Attribution — which feature consumed which tokens — is what makes optimization possible. Organizations that implement cost attribution consistently report identifying two or three "cost sink" features that account for the majority of spend, often features that had never been considered high-cost during development.

Adoption Roadmap: 4 Phases to Controlled AI Spend

Cost optimization is not a one-time project. It is an ongoing practice. Organizations that achieve and sustain 50–80% cost reductions follow a consistent phased approach.

Phase 1 — Measure your baseline. Before changing anything, establish what you are spending, by feature and model. This takes two to four weeks and requires engineering effort to add cost attribution to your existing AI calls. Without this data, you are optimizing blind.

Phase 2 — Identify expensive patterns. With attribution in place, surface the top cost drivers. Typical findings: a high-volume feature using the wrong model tier, a long system prompt without caching, a batch-eligible workflow running in real-time.

Phase 3 — Apply targeted optimizations. Address findings in order of impact. Model right-sizing is usually first — it requires minimal engineering effort and delivers immediate, compounding returns. Prompt caching is second. Response caching and batch processing follow.

Phase 4 — Monitor and iterate. New features introduce new cost patterns. Set a recurring monthly review cadence where engineering and finance align on spend vs. budget and flag new anomalies. The loop between Phase 4 and Phase 2 is what prevents costs from spiraling again after the initial optimization effort.

Questions to Ask Your Engineering Team

Before your next budget review or AI project kickoff, bring these questions to your engineering leadership:

Which AI features are using which models? Can you show me a list of every production AI feature and the model tier it currently runs on?
Do we have cost attribution? Can we see our monthly AI spend broken down by feature, workflow, or user segment — not just as a single total?
Are we using prompt caching on any feature with a long system prompt? If a feature sends the same instructions with every request and is not using prompt caching, what is the estimated monthly savings from enabling it?
Which AI workflows run in real-time that could run overnight? Is there a list of report generation, bulk processing, or classification jobs that currently use the real-time API but don't need to?
Do we have spend alerts configured? At what thresholds do we receive notifications, and who receives them?
Are there application-level rate limits or cost caps per user? What prevents a single user or workflow from consuming an outsized share of our monthly token budget?
When did we last review whether each feature is on the right model tier? Has any feature been moved from a flagship model to a lower-cost tier after initial development — or do we still run the same models we used during prototyping?

Conclusion

AI API costs are not a fixed overhead — they are a function of architectural decisions your engineering team makes every day. The organizations I've seen get this right share one thing: they treated visibility as non-negotiable from the start, not as a cleanup project after the bill became alarming. Once you can see where the money goes, the optimizations follow naturally — and they tend to be faster and cheaper to implement than anyone expected. The goal is not to spend less on AI. It is to stop funding waste so the budget can go toward the AI capabilities that actually move your business forward.

From Billing Hours to Billing Outcomes: New Contract Structures for AI-Assisted SI Projects

Bry — Wed, 26 Aug 2026 15:00:00 +0000

Key Points

Time-and-materials billing gives the vendor no financial incentive to finish faster. Every extra hour is another invoice line. AI-accelerated delivery is making that misalignment obvious to clients who can see the work speeding up while the invoice logic stays the same.
73% of consulting clients already tell researchers they prefer value-based or outcome-driven pricing over hourly rates. That preference is likely to harden into the default contract structure across most of the systems-integration market within the next several years.
Outcome-based contracts don't eliminate risk. They move it. Vendors absorb schedule risk on defined deliverables, which is forcing real changes to scoping discipline, delivery governance, and which projects an SI will even accept.
Time-and-materials isn't disappearing. It survives, correctly, for genuinely exploratory work where the scope can't be defined upfront. The mistake to watch for is applying it to well-defined work out of habit, not because T&M is inherently wrong.
IDC's forecast, 30% of IT services contracts outcome-based by 2029, is starting to look conservative. If AI-driven delivery speed keeps compounding, outcome-based and hybrid subscription-plus-usage structures could cover a majority of new systems-integration engagements within the decade, in markets where clients have any real pricing leverage.

Introduction

Why the Man-Month Is Dying: How AI Broke IT Services' Oldest Pricing Unit in this thread covered why the man-month is dying as a pricing unit. This one covers what's actually starting to get written into contracts once firms stop defaulting to it, because "bill outcomes, not hours" is a slogan, and slogans don't survive contact with a real statement of work.

The economic case against pure time-and-materials was already documented before AI made it urgent. PMI's Pulse of the Profession data shows T&M engagements running 23% over budget on average, which on a $50,000 project is $11,500 of unplanned client spend, with no penalty to the vendor for the overrun. AI-assisted delivery hasn't fixed that structural misalignment. If anything, it's making it worse in the near term, because a vendor billing by the hour has an active disincentive to let AI tools cut delivery time, and clients increasingly can tell. Futurum Research already found 73% of consulting clients favor value-based or outcome-driven pricing over hourly rates, largely because AI's delivery-speed gains have made time-based billing look indefensible rather than merely inefficient.

What's replacing it won't be a single template. Outcome-based contracts, payment tied to specific, measurable deliverables with acceptance criteria, are taking over the well-defined end of the market: data migrations, defined integrations, modernization projects with a clear "done" state. Hybrid structures, pairing a base subscription for fixed costs with a usage or outcome-based component above a baseline, are showing up in ongoing managed-service relationships. Time-and-materials should survive where it always made sense: genuinely exploratory work, early-stage product discovery, engagements where the client themselves doesn't yet know the final scope. The mistake to avoid over the next few years isn't choosing outcome-based pricing. It's applying whichever model a firm already knows how to bill, regardless of which one actually fits the work.

Contract Model Fit by Project Type

Project Type	Best-Fit Model	Why
Defined data migration or system integration	Outcome-based	Scope and success criteria are specifiable in advance; AI-accelerated delivery becomes vendor margin, not client discount
Early-stage product discovery, undefined scope	Time-and-materials	Scope genuinely can't be fixed upfront; forcing outcome pricing here just relabels the guesswork
Ongoing managed services / maintenance	Hybrid subscription + usage	Base cost stays predictable for the client; usage component tracks real variability instead of headcount
Fixed-scope modernization with a hard deadline	Fixed-price with milestone acceptance	Client needs cost certainty; vendor absorbs schedule risk in exchange for a defined, unchanging scope
Regulatory-driven compliance rebuild	Outcome-based, tied to audit-passable state	Client cares about a certifiable end state, not hours logged getting there

Recommendation: match the contract model to how well-defined the scope actually is, not to which model your firm is most comfortable billing. An SI that only offers T&M today is telling well-defined-scope clients, correctly, that it hasn't done the scoping work outcome-based pricing requires.

Making the Switch: A Phased Approach

Build the measurement infrastructure before changing the invoice template. Outcome-based billing requires a checkable definition of "done": acceptance criteria, a metering system for usage, or an audit standard. Firms that flip their contract language before building this lose money on undefined "outcomes."
Start with your most well-understood, most frequently repeated project type. A data migration you've delivered fifty times is far easier to price on outcome than a novel integration you've never scoped before. Don't lead the transition with your hardest, most novel work.
Renegotiate existing T&M relationships in the open, not by stealth. Clients who discover a vendor quietly pocketing AI-driven speed gains under an unchanged T&M invoice react far worse than clients told directly that pricing is moving to outcomes and shown why.
Keep a genuine T&M option for genuinely exploratory engagements. Retiring T&M entirely just pushes clients with undefined scope toward vendors willing to be honest that some work can't be priced by outcome yet.
Revisit pricing on every engagement renewal, not just new business. The real lesson behind IDC's forecast isn't the 30% number. It's that firms which wait for contract renewal cycles to force the pricing conversation, rather than initiating it, cede the framing to whichever competitor gets there first.

Questions to Ask Your Team

For our current book of business, how many engagements are still billed by the hour purely out of habit, on work that's actually well-defined enough to price on outcome?
If a client asked us directly why our pricing model doesn't reward them for AI-driven delivery speed, do we have an honest answer, or an evasive one?
Do we have real acceptance criteria and measurement infrastructure in place before we've committed to an outcome-based number, or are we guessing at "outcomes" the same way we used to guess at hours?
Are we keeping time-and-materials available for the engagements that genuinely need it, or treating it as a legacy model to be phased out everywhere regardless of fit?

Conclusion

The contract structures most likely to survive the man-month's decline won't be chosen because they sound better in a sales deck. Outcome-based pricing is winning the well-defined end of the market because it aligns vendor incentive with AI-driven speed instead of punishing it. Time-and-materials should survive at the genuinely exploratory end because forcing a false "outcome" definition onto undefined scope just relabels the same uncertainty. The firms most likely to struggle through this transition aren't the ones that pick the wrong model. They're the ones that pick one model and apply it everywhere, regardless of whether the work in front of them actually fits it.

Azure Event Grid: Push-Based Events Without the Kafka Overhead

Bry — Tue, 25 Aug 2026 15:00:00 +0000

Key Points

Event Grid is a push-based event router, not a streaming log. Azure's actual Kafka-equivalent is Event Hubs — conflating the two leads teams to pick the wrong service for high-throughput streaming.
Pricing is $0.60 per million operations after the first 100,000 free per month, and delivery is at-least-once, never exactly-once — every subscriber you build must be idempotent, full stop.
A custom topic, a webhook subscription, and dead-letter configuration take four CLI commands — dead-lettering specifically is the one step almost every quickstart skips.
Choose Event Grid when you need to react to discrete events (a blob uploaded, an order placed). Choose Event Hubs when you need to process a continuous stream and replay it.

Prerequisites

CLI/SDK version tested against: az-cli 2.6x.x
An Azure resource group and a storage account for dead-letter destination (az storage account create)
A publicly reachable HTTPS webhook endpoint to receive events — the examples use a placeholder Azure Function URL

Introduction

I've had the same conversation with two different teams: "we need event-driven architecture on Azure, so we need Kafka." Neither actually needed Kafka. Both needed to react when something happened — a file landed in storage, an order got placed — not to process an unbounded stream of records with replay and consumer-group semantics. That's Event Grid's job, and it does it in far less setup than standing up Event Hubs or a managed Kafka cluster would require.

The confusion is understandable. Azure has three services that all get called "event-driven": Event Grid, Event Hubs, and Service Bus. Only Event Hubs is the Kafka-equivalent. Event Grid is closer to a fan-out push notification system — publish an event, Event Grid delivers it via HTTPS webhook (or a handful of other supported destinations) to every matching subscriber, with retries and dead-lettering built in.

This will explain how to build a real topic and subscription from the CLI, including the dead-letter setup nearly every quickstart skips, and draw the line clearly between what Event Grid is for and what it isn't.

Where Event Grid Sits

Diagram: one topic, multiple independent subscriptions, each with its own delivery and dead-letter configuration.

Diagram: Event Grid pushes discrete events to subscribers; Event Hubs is the pull-based, replayable stream for continuous data — they solve different problems.

Building It From the CLI

# 1. A custom topic — the entry point events get published to.
az eventgrid topic create \
  --name orders-topic \
  --resource-group rg-eventgrid-demo \
  --location eastus

TOPIC_ID=$(az eventgrid topic show \
  --name orders-topic \
  --resource-group rg-eventgrid-demo \
  --query id --output tsv)

TOPIC_ENDPOINT=$(az eventgrid topic show \
  --name orders-topic \
  --resource-group rg-eventgrid-demo \
  --query endpoint --output tsv)

TOPIC_KEY=$(az eventgrid topic key list \
  --name orders-topic \
  --resource-group rg-eventgrid-demo \
  --query key1 --output tsv)

# 2. A storage container for dead-lettered events — set this up before
#    the subscription, not after something starts failing silently.
az storage container create \
  --name eventgrid-deadletter \
  --account-name eventgriddemostorage

# 3. A webhook subscription with retry policy and dead-letter destination.
az eventgrid event-subscription create \
  --name order-notifications \
  --source-resource-id "$TOPIC_ID" \
  --endpoint "https://order-processor.azurewebsites.net/api/handle-order-event" \
  --max-delivery-attempts 10 \
  --event-ttl 1440 \
  --deadletter-endpoint "/subscriptions/${SUBSCRIPTION_ID}/resourceGroups/rg-eventgrid-demo/providers/Microsoft.Storage/storageAccounts/eventgriddemostorage/blobServices/default/containers/eventgrid-deadletter"

# 4. Publish a test event to confirm the wiring.
curl -X POST "$TOPIC_ENDPOINT" \
  -H "aeg-sas-key: $TOPIC_KEY" \
  -H "Content-Type: application/json" \
  -d '[{
    "id": "evt-001",
    "eventType": "OrderCreated",
    "subject": "orders/order-abc123",
    "eventTime": "2026-08-01T12:00:00Z",
    "data": { "orderId": "order-abc123", "status": "pending" },
    "dataVersion": "1.0"
  }]'

Four commands, and unlike the EventBridge build in AWS EventBridge, publishing here goes over plain HTTPS with a SAS key header rather than through a dedicated CLI verb — Event Grid's publish path is the topic's own HTTPS endpoint, not an az eventgrid publish command.

At-Least-Once Delivery Is Not Optional Reading

Event Grid retries failed deliveries — up to --max-delivery-attempts times, with exponential backoff, before giving up and routing to the dead-letter destination. That retry behavior means your webhook handler will, eventually, receive the same event more than once. Not as an edge case — as the documented contract.

Diagram: Event Grid's retry-then-dead-letter sequence. A handler that isn't idempotent will process the same logical event twice on any successful retry after a prior partial failure.

Design the handler to key off id (the event ID) and treat re-delivery of a seen ID as a no-op. This is the single most common production bug I've seen with Event Grid consumers — not a delivery failure, but a handler that wasn't written to expect its own retries.

Event Grid vs Event Hubs vs Service Bus

	Event Grid	Event Hubs	Service Bus
Model	Push (webhook/fan-out)	Pull (streaming log, consumer groups)	Pull (queue/topic, message broker)
Kafka-equivalent	No	Yes	No
Replay	No	Yes (retention window)	No (messages consumed once, unless using sessions/dead-letter)
Delivery guarantee	At-least-once	At-least-once (consumer-managed offsets)	At-least-once, FIFO available with sessions
Best for	React to discrete events (blob created, order placed)	High-throughput continuous streams, analytics pipelines	Ordered work queues, transactional messaging
Pricing model	Per-operation ($0.60/M after free tier)	Per-throughput-unit + per-million-events	Per-tier flat + per-operation (Premium)

Recommendation: If your use case is "when X happens, notify Y," that's Event Grid. If it's "process an unbounded stream and be able to replay from three days ago," that's Event Hubs. If it's "guarantee this work item gets processed exactly once, in order, by exactly one worker," that's Service Bus.

Common Mistakes

Mistake 1: Reaching for Event Grid to replace Kafka
Event Grid has no replay and no consumer-group semantics. If a stakeholder says "we need Kafka on Azure," the answer is Event Hubs, not Event Grid — don't let the word "events" in both names cause a wrong pick.

Mistake 2: Building webhook handlers that assume single delivery
At-least-once means exactly what it says. A handler that isn't idempotent against the event id will eventually double-process something, usually during a transient network blip that has nothing to do with your code.

Mistake 3: Skipping dead-letter configuration
Without --deadletter-endpoint, an event that exhausts all delivery attempts is simply gone. No default fallback storage exists. Configure it at subscription creation, not after the first unexplained missing event.

Mistake 4: Setting --event-ttl too short for slow downstream recovery
The default and commonly-used TTL is 1440 minutes (24 hours). If your webhook endpoint can plausibly be down longer than that during an incident, either extend the TTL or accept that events older than it are dead-lettered, and build your recovery runbook around checking that container.

Production Considerations

Performance: Event Grid's push model means latency is dominated by your subscriber's response time, not Event Grid itself — a slow webhook handler triggers Event Grid's own retry logic, which compounds load on an already-struggling endpoint.

Security: Validate the aeg-event-type header for the initial SubscriptionValidation handshake event, and verify the event signature or use Azure AD-based authentication on the webhook endpoint rather than relying solely on the SAS key being unguessable.

Cost: At $0.60/million operations after 100K free per month, Event Grid is inexpensive even at meaningful scale — the cost driver worth watching is retry volume from a flaky subscriber, not the base event rate.

Monitoring: Track PublishSuccessCount, DeliveryAttemptFailCount, and DeadLetteredCount metrics per subscription. A rising DeadLetteredCount with no corresponding alert is how "the event system is just quietly dropping stuff" becomes a support ticket weeks later.

Full Example: Idempotent Webhook Handler

import express, { Request, Response } from 'express';

const app = express();
app.use(express.json());

/** Tracks processed event ids to make delivery idempotent — swap for Redis/DB in production. */
const processedEventIds = new Set<string>();

/**
 * POST /api/handle-order-event — Event Grid webhook target.
 *
 * @remarks
 * Handles the SubscriptionValidation handshake on first subscribe, then
 * processes OrderCreated events. Re-delivered events (same `id`) are
 * acknowledged with 200 but not reprocessed — required given Event Grid's
 * at-least-once delivery guarantee.
 *
 * @returns 200 on successful handling or validation; 400 on malformed payload.
 */
app.post('/api/handle-order-event', (req: Request, res: Response) => {
  const events = req.body as Array<{ id: string; eventType: string; data: unknown }>;

  for (const event of events) {
    if (event.eventType === 'Microsoft.EventGrid.SubscriptionValidationEvent') {
      const validationCode = (event.data as { validationCode: string }).validationCode;
      res.status(200).json({ validationResponse: validationCode });
      return;
    }

    if (processedEventIds.has(event.id)) {
      continue; // Already handled — Event Grid retried a prior delivery.
    }

    processedEventIds.add(event.id);
    console.log(`Processing order event ${event.id}`);
  }

  res.status(200).send();
});

app.listen(process.env.PORT ? parseInt(process.env.PORT, 10) : 3000);

Full source with the teardown script: GitHub → cloud-apis/azure-event-grid-cli/

Conclusion

Event Grid is the right tool when you need to react to something that happened, delivered via push with retries and dead-lettering handled for you — not when you need a replayable, high-throughput stream, which is Event Hubs' job. Build every subscriber assuming at-least-once delivery from day one, configure dead-lettering before you need it rather than after an event goes missing, and the four-command CLI setup above will get you further, faster, than most teams expect from an "enterprise event routing" service.

Why Japanese Enterprises Buy Differently: The RFP-to-Contract Cycle for Tech Vendors

Bry — Thu, 20 Aug 2026 15:00:00 +0000

Key Points

Japanese enterprise sales cycles for foreign vendors without local presence run 6–18 months, against 3–6 months in the US — a structural difference, not a sign the deal is stalling.
Two named mechanisms drive the length: nemawashi (informal, one-on-one consensus building before anything is formal) and ringi (the formal document that circulates for sign-off once consensus already exists).
By the time a ringi-sho actually starts circulating, the real decision is usually already made — the formal approval stage is closer to ratification than to open deliberation.
Pushing for a faster close during nemawashi doesn't speed up the deal. It signals you don't understand the process you're actually in, which is a worse outcome than a slow yes.

Introduction

A first-time-in-Japan sales lead I worked with was two months into a promising deal, had presented three times, and felt like the deal had gone quiet. His read: the champion had lost interest, or a competitor had swooped in. Neither was true. He was watching nemawashi happen and reading it as silence.

Japan's enterprise buying process runs through two distinct, named phases most foreign vendors have never heard of by name even when they've lived through them: nemawashi, the informal groundwork that happens before anything is written down, and ringi, the formal document that circulates for sign-off once that groundwork is done. Japan Market Entry for Cloud & SaaS Vendors in this series introduced why localization has to go deeper than translation to clear a Japanese buying process at all. This article is the buying process itself — what actually happens between your first meeting and a signed contract, and why the timeline that looks like inefficiency from the outside is closer to due diligence than delay.

Nemawashi and Ringi: The Two Mechanisms

Think of nemawashi like a gardener preparing a tree's root system for months before a transplant, which is literally where the word comes from ("going around the roots"). You don't just dig up the tree and move it; you prepare every root individually so the move, when it happens, doesn't shock the plant. Nemawashi is the same idea applied to a business decision: your champion has a private conversation with every stakeholder who'll eventually need to sign off, works through their specific objection, adjusts the proposal in response, and keeps going until everyone who matters has quietly signaled they won't block it.

Ringi is what happens after that groundwork is done. A formal document, the ringi-sho, circulates through the stakeholders nemawashi already aligned, and each one adds their hanko as a recorded sign-off. The critical detail most foreign vendors miss: by the time the ringi-sho starts moving, the real decision was already made during nemawashi. The formal circulation is ratification of a consensus that already exists, not an open debate where your deal could still be won or lost.

The problem it solves: without knowing these two phases exist, a foreign sales team reads the quiet nemawashi period as a stalled deal, pushes for a decision meeting to "move things forward," and in doing so signals to the champion that the vendor doesn't understand or respect the process — which is a specific kind of damage nemawashi, done well, is supposed to prevent.

The Four-Phase Cycle in Practice

Phase 1 — Introduction and relationship building (1–3 months). Initial meetings, meishi (business card) exchange, company overview presentations. Western sales instinct is to get to a needs-assessment conversation fast; in Japan, skipping straight past relationship-building reads as presumptuous, not efficient.

Phase 2 — Needs assessment and technical evaluation (2–4 months). Detailed requirements gathering across multiple departments, security and compliance review, reference checks against your existing Japanese customers if you have any (and a real handicap if you don't).

Phase 3 — Nemawashi (2–4 months). Your champion works the stakeholder list one conversation at a time. Questions come back to you filtered through the champion, not directly — a structural reason "just get us in a room with the decision-maker" doesn't work the way it might elsewhere. This is the quiet period that reads as radio silence to a vendor who doesn't know what it is.

Phase 4 — Formal proposal, ringi, and contract (1–3 months). Japanese-language pricing proposal, contract terms, hanko-sealed execution. By this point the deal is close to done; this phase is largely mechanical.

Add it up and a deal that would close in six months against a US buyer routinely runs 6–18 months against a Japanese enterprise without existing local presence — a JETRO-cited comparison puts US cycles at 3–6 months and Western Europe at 4–9 months against that same 6–18 month Japan range.

Comparison: Pushing the Timeline vs Working the Process vs Skipping the Groundwork

Criteria	Pushing for a Faster Close	Working the Nemawashi Process	Skipping Straight to Formal Proposal
Champion relationship	Damaged — reads as impatience or disrespect for process	Strengthened — champion sees you as easy to work with	Champion put in an awkward position, unprepared stakeholders
Actual deal velocity	Often slower — objections surface late and derail the ringi stage	Fastest realistic path — objections resolved before they can block	Frequently rejected outright at the ringi stage
Stakeholder buy-in	Shallow — sign-offs given reluctantly, deal at risk post-signature	Deep — sign-offs are ratifying a decision people already support	Absent — stakeholders haven't had their concerns heard
Best for	Never — this consistently backfires in Japan specifically	Any enterprise deal without an existing fast-track relationship	Only viable for the rare vendor with deep, established local trust already

Bottom line: work the nemawashi process deliberately rather than trying to compress it. The vendors who push for speed lose more deals to a mishandled Phase 3 than they gain from the weeks they saved trying to skip it.

Pros and Cons

Advantages of Understanding This Process

Higher close rates once you're in Phase 3: a deal that reaches active nemawashi with a real champion has already cleared most of the risk — the objections get surfaced and handled before the formal stage, not after.
Durable post-signature relationships: because ringi ratifies genuine, not coerced, consensus, contracts signed this way tend to expand rather than churn.
A real moat against less patient competitors: many foreign vendors give up or mishandle Phase 3, which is exactly why the vendors who work it properly face less competitive pressure at the point that matters most.

Disadvantages and Risks

Real opportunity cost: 12–18 months is a long cash-flow assumption to build a go-to-market plan around, and a board expecting Western-market velocity needs to be told this explicitly, early.
Limited visibility during Phase 3: you genuinely don't get to see the internal conversation — you have to trust your champion is representing you well, which makes champion selection itself a critical, and easy to underweight, decision.
No shortcut exists for a vendor without a champion: if you can't identify and cultivate someone internally who'll do the nemawashi work on your behalf, the deal has a structural ceiling regardless of product quality.

Is This Right for You?

Plan for the full cycle if:

You're entering without an existing Japan presence or reference customers.
Your target account is a large enterprise with a multi-department approval structure.

You may see a faster cycle if:

You already have a strong local reference customer in the same industry.
Your champion has run this process for a similar tool before and can move faster because the stakeholder map is already known to them.

Reconsider the target account if:

You have no path to an internal champion at all — no product, however good, closes a deal that has nobody running nemawashi on your behalf.

A Realistic First Engagement

Identify your champion early and invest in them specifically — not just the person who took your first meeting, the person who'll actually walk the ringi-sho through stakeholders.
Prepare materials for nemawashi, not just for your pitch — one-pagers your champion can hand to a stakeholder in a hallway conversation matter more here than your deck does.
Expect and plan for the quiet period — brief your own leadership in advance so a two-month silence in Phase 3 doesn't get misread internally as a dying deal.
Have Japanese-language, yen-denominated materials ready before Phase 4 starts, not scrambled together once the ringi-sho is already circulating.

Cost Considerations

Cost Type	What to Budget For	Typical Range
Extended sales cycle carrying cost	Sales team time and CAC amortized over 6–18 months instead of 3–6	2–3x the carrying cost of an equivalent US deal
Local champion cultivation	Travel, relationship-building trips, in-person meetings (4–5 typical before pricing is even discussed)	3–5 in-person visits over the cycle if not locally based
Materials localization for nemawashi	One-pagers, technical briefs your champion can circulate informally	$5,000–$15,000 one-time
Formal proposal and contract stage	Japanese-language pricing proposal, dual-language contract	Covered under Japan Market Entry for Cloud & SaaS Vendors commercial localization estimate

ROI signal: treat the extended cycle as a cost of entry, not a red flag mid-deal — a Japanese enterprise contract that clears ringi tends to expand and renew more predictably than an equivalent Western deal, because the consensus behind it is real, not a single champion's individual sign-off.

Questions to Ask Your Team

"Do we have an identified internal champion, and have we invested time specifically in them, not just the stakeholder group broadly?"
"Is our sales forecast modeling a 6–18 month cycle, or did we copy our US timeline into this account's projection?"
"When the deal goes quiet, do we have a plan to check in appropriately, or will we default to pushing for a decision meeting?"
"Are our materials ready for a champion to hand to a stakeholder informally, not just for a formal pitch meeting?"
"Have we told our own leadership what nemawashi is, so a quiet Phase 3 doesn't get misread as a dying deal internally?"

Conclusion

The length of a Japanese enterprise sales cycle isn't inefficiency waiting to be optimized away — it's nemawashi and ringi doing exactly what they're designed to do: surface every objection before it can kill the deal, and turn a formal sign-off into ratification instead of a fresh vote. Plan your cash flow and your patience around 6–18 months, invest specifically in your champion, and resist the instinct to push for speed during the quiet period. The vendors who understand this close more deals, not fewer, than the ones treating Japan like a slower version of their home market.

Estimating a Project When AI Can Prototype the Whole Thing in a Day

Bry — Wed, 19 Aug 2026 15:00:00 +0000

Key Points

Traditional estimation assumed the biggest unknown was implementation time. AI-assisted prototyping is collapsing that unknown for a large class of projects. A working, demoable version of "the whole thing" is increasingly a same-day artifact, not a milestone.
This isn't making estimation easier. It's moving the hard part upstream, from "how long will this take to build" to "what does this client actually need, and which parts of a fast prototype are load-bearing versus disposable."
Clients who watch a prototype appear in a day reasonably ask why the full engagement still takes months. An SI that can't answer that clearly, precisely, and early loses the deal before pricing even comes up.

Introduction

I watched a prospective client's face change during a discovery call recently, the moment our team screen-shared a working prototype of their requested feature, built that morning, in response to the brief they'd sent the night before. The reaction wasn't delight. It was suspicion. If that took a morning, the client asked, what exactly is the six-week, six-figure proposal for?

It's a fair question, and estimation as a discipline has to answer it honestly or lose credibility entirely. For most of the industry's history, the dominant uncertainty in a project estimate was implementation time: how long would it actually take to write, wire up, and test the thing being asked for. AI-assisted prototyping tools are attacking that uncertainty directly. What took three to five days of estimation and scaffolding work now often takes twenty minutes to produce a workable first pass, and a demoable version of an entire application can be a same-day artifact rather than a weeks-away milestone. That's not marginal. It's inverting which part of a project is actually hard to estimate.

What isn't getting faster is figuring out what the client actually needs versus what they asked for, which requirements are genuinely load-bearing versus assumed, and which parts of a fast prototype are production-viable versus a convincing façade over unhandled edge cases, missing data governance, and untested failure modes. A prototype answers "can this exist." It says almost nothing about "should this exist in production, serving real traffic, under this client's compliance obligations." Conflating the two is the single most common estimation failure I'm seeing since prototyping got fast, and it burns trust exactly when trust is what the estimate needs most.

Old Estimation Bottleneck vs. Post-AI Estimation

Criteria	Pre-AI Estimation	Post-AI Estimation
Dominant source of uncertainty	Implementation time: how long to actually build it	Requirements clarity: what the client actually needs versus what they described
Role of a prototype in the sales cycle	Proof delivered weeks into the engagement, after signing	Increasingly delivered same-day, before or during the pitch itself
What the client is really paying for	Labor to produce a working system	Judgment separating prototype-viable from production-viable, and scoping the gap between them
Biggest estimation risk today	Underestimating build time	Underestimating the gap between "it demos" and "it survives production"
What a fast prototype proves	Very little on its own	Real value, once paired with an honest gap analysis against production requirements

Recommendation: stop treating a same-day prototype as evidence the whole engagement should be same-day priced. Treat it as the starting artifact for the estimate, and spend the actual estimation effort on the gap between what's on screen and what a production system serving that client's real constraints requires.

A Fast-Prototype Estimation Process That Actually Holds Up

Build the prototype first, in front of the client if possible. It's the fastest way to surface misunderstood requirements. Clients correct a prototype far more precisely than they answer an abstract discovery question.
Separate "demoed" from "load-bearing" explicitly, in writing. For every feature the prototype shows working, state whether it's handling real data volumes, real failure modes, real auth, and real compliance requirements, or whether it's a scripted happy path.
Price the gap, not the prototype. The estimate should account for hardening, integration with existing systems, data migration, and the client's actual compliance and scale requirements: the parts a prototype is specifically designed to skip.
Show the client the gap analysis, not just the number. A client who's seen a prototype built in a day will not accept a six-figure quote without an explanation. Show them exactly what's between "it demos" and "it ships," and the number stops looking arbitrary.
Revisit the estimate at the first production-scale test, not at the end. AI-assisted prototypes routinely hide scale and integration problems that only surface once real data and real load hit the system. Build a checkpoint for that into the estimate itself, not as a change-order surprise.

Questions to Ask Your Team

When we show a client a fast prototype, do we also show them, explicitly, what's not yet handled, or do we let the demo imply more readiness than exists?
Is our estimation effort actually going toward requirements clarity now, or are we still budgeting most of our estimation time for implementation guesswork that AI has already answered?
Have we ever lost a client's trust because a fast prototype made our full-engagement timeline look inflated, and did we have a clear answer ready when they asked why?
Do our estimates get revisited at a real production-scale checkpoint, or only at the end when a scale problem becomes an expensive surprise?

Conclusion

A same-day prototype is a genuinely useful artifact and a genuinely dangerous one, depending on what an SI does with it. Used as evidence that judgment is unnecessary, it wrecks client trust the moment the full quote lands. Used as the starting point for an honest, explicit gap analysis between what demos and what ships, it's the fastest way to have the requirements conversation that used to take weeks. The estimation skill that mattered a few years ago, guessing implementation time, is fading fast. The one replacing it, telling a client precisely what stands between a convincing demo and a production system they can bet their business on, is harder. It's also the one clients are increasingly paying for.

API Gateway Patterns: AWS vs Azure vs GCP

Bry — Tue, 18 Aug 2026 15:00:00 +0000

Key Points

AWS API Gateway comes in two flavors: HTTP API (cheaper, faster, 90% of use cases) and REST API (full feature set, higher cost). Use HTTP API unless you need REST API's request/response transformation or private integrations.
Azure API Management (APIM) is the only managed gateway with a built-in developer portal and a policy engine powerful enough to handle enterprise API governance without custom code.
GCP offers three overlapping products: API Gateway (serverless, OpenAPI-driven), Cloud Endpoints (gRPC and self-managed), and Apigee (enterprise-grade). Don't pick randomly — they are not interchangeable.
Common patterns — BFF (Backend for Frontend), API aggregation, and protocol translation — are cloud-agnostic in design but differ significantly in implementation cost across providers.
Cold starts on Lambda-backed gateways, APIM's XML policy syntax, and GCP's per-project quota model are the three gotchas that will cost you time in production.

Introduction

Picking an API gateway is a three-to-five year decision. Migrating between providers requires rewriting auth flows, rate-limiting config, and deployment pipelines — not just updating a URL. I've evaluated all three in production environments — migrating a client from AWS REST API to HTTP API, standing up APIM for an enterprise API program, and routing Cloud Run services through GCP API Gateway — and the differences that matter are rarely the ones in the marketing comparison tables. Most teams pick the gateway that matches their existing cloud provider without comparing what they're getting.

That's often the right call. But it helps to know what you're trading. AWS, Azure, and GCP have different design philosophies: AWS optimizes for serverless-native integration, Azure optimizes for enterprise API governance, and GCP gives you a choice between a lightweight managed service and a full API platform.

This article explains what each gateway does architecturally, compares them on the features that matter in production, covers the three patterns you'll implement on any of them, and flags the gotchas that each provider's documentation glosses over.

What an API Gateway Does

Before comparing providers, align on the job a gateway performs. An API gateway sits between clients and backend services and handles cross-cutting concerns so your services don't have to.

Diagram: Client traffic enters the gateway, which handles auth and routing before forwarding to backend services.

The five jobs a gateway performs:

Routing — match request paths to backend services, including path rewriting and load balancing.
Authentication and authorization — validate JWT tokens, API keys, or OAuth flows before traffic reaches your service.
Rate limiting — enforce per-client or per-route request quotas; protect backends from traffic spikes.
Request/response transformation — rewrite headers, translate between protocols (REST ↔ gRPC), strip or inject fields.
Observability — emit access logs, request metrics, and distributed traces without instrumenting each service.

Every provider covers these five jobs. How they expose them — and at what cost — is what differentiates them.

Request Flow Through a Gateway

Before choosing a provider, understand the sequence every request traverses. The auth, rate-limit, and routing steps happen in this order regardless of which gateway you use.

Diagram: Request flow through a gateway — auth and rate limiting happen before the backend ever receives the request.

AWS API Gateway

AWS offers three API products under the API Gateway brand: HTTP API, REST API, and WebSocket API. HTTP API and REST API are the ones backend engineers choose between daily.

HTTP API vs REST API

Feature	HTTP API	REST API
Price	$1.00 / million requests	$3.50 / million requests
Latency overhead	~6 ms	~11 ms
JWT authorizer (native)	Yes	No — Lambda authorizer required
Request/response transforms	No	Yes (mapping templates)
Usage plans / API keys	No	Yes
Private integrations (VPC Link)	Yes	Yes
WebSocket	No — separate product	No
AWS WAF integration	Yes	Yes

Use HTTP API for Lambda-backed services, JWT-authenticated endpoints, and any new API where you don't need request/response mapping templates. It costs 71% less than REST API and imposes half the latency overhead. I default to HTTP API for every new AWS project — the only time I've reached for REST API in the last two years was when a client needed usage plans for billing third-party API consumers.

Use REST API when you need usage plans and API keys for third-party developer access, request/response transformation via mapping templates, or Cognito user pool authorizers without a Lambda function.

Lambda Integration

The typical AWS pattern connects API Gateway directly to Lambda. The gateway acts as the event source; Lambda handles the business logic.

// AWS Lambda handler — receives API Gateway proxy event
import { APIGatewayProxyEventV2, APIGatewayProxyResultV2 } from 'aws-lambda';

export async function handler(
  event: APIGatewayProxyEventV2,
): Promise<APIGatewayProxyResultV2> {
  const userId = event.pathParameters?.userId;

  if (!userId) {
    return {
      statusCode: 400,
      headers: { 'Content-Type': 'application/problem+json' },
      body: JSON.stringify({
        type: 'https://api.example.com/errors/400',
        title: 'Bad Request',
        status: 400,
        detail: 'userId path parameter is required',
      }),
    };
  }

  // Business logic here
  return {
    statusCode: 200,
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ userId, status: 'active' }),
  };
}

# HTTP API route configuration (AWS SAM / CloudFormation)
Resources:
  GetUserFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: dist/handler.handler
      Runtime: nodejs22.x
      Events:
        GetUser:
          Type: HttpApi
          Properties:
            Path: /users/{userId}
            Method: GET
            Auth:
              Authorizer: JWTAuthorizer

  MyHttpApi:
    Type: AWS::Serverless::HttpApi
    Properties:
      Auth:
        Authorizers:
          JWTAuthorizer:
            JwtConfiguration:
              Audience: ["https://api.example.com"]
              Issuer: "https://auth.example.com/"
            IdentitySource: "$request.header.Authorization"

The Cold Start Gotcha

Lambda-backed HTTP APIs incur cold start latency when a function instance is not warm. For a Node.js 22.x Lambda with a small bundle, cold starts add 200–800 ms. For a Java or .NET Lambda with a heavy runtime, expect 1–5 seconds.

Mitigations:

Enable Provisioned Concurrency on the Lambda — eliminates cold starts for pre-warmed instances, billed hourly even when idle.
Keep Lambda bundle size under 5 MB (smaller = faster cold start).
Use SnapStart for Java Lambda functions (zero cold start after initial activation).

Cold starts do not affect AWS ECS or EKS backends connected via VPC Link — they only affect Lambda integrations. I've seen this burn teams who built their entire p99 latency budget around gateway benchmarks, then went live and got paged at 3 AM because a burst of new connections spiked p99 to 2.8 seconds — the Lambda was a .NET 6 function with a 40 MB bundle and no Provisioned Concurrency.

Azure API Management (APIM)

Azure APIM is architecturally different from AWS API Gateway. It is not a simple proxy — it is a full API management platform with three components:

Diagram: APIM's three-component architecture — the gateway, the management plane, and the developer portal are separate concerns.

The Policy Engine

APIM's differentiating feature is its XML-based policy engine. Policies apply at four scopes: global, product, API, and operation. They execute in order: inbound → backend → outbound → on-error.

<!-- APIM policy: validate JWT, rate-limit, inject backend header -->
<policies>
  <inbound>
    <base />

    <!-- Validate JWT from Authorization header -->
    <validate-jwt header-name="Authorization" failed-validation-httpcode="401"
                  failed-validation-error-message="Unauthorized">
      <openid-config url="https://auth.example.com/.well-known/openid-configuration" />
      <audiences>
        <audience>https://api.example.com</audience>
      </audiences>
    </validate-jwt>

    <!-- Rate limit: 100 calls per 60 seconds per subscription key -->
    <rate-limit-by-key calls="100" renewal-period="60"
                       counter-key="@(context.Subscription.Id)" />

    <!-- Inject caller identity into backend header -->
    <set-header name="X-User-Id" exists-action="override">
      <value>@(context.Request.Headers.GetValueOrDefault("Authorization","")
               .Replace("Bearer ",""))</value>
    </set-header>
  </inbound>
  <backend>
    <base />
  </backend>
  <outbound>
    <base />
    <!-- Strip internal headers before returning to client -->
    <set-header name="X-Internal-Trace-Id" exists-action="delete" />
  </outbound>
  <on-error>
    <base />
  </on-error>
</policies>

The policy engine handles authentication, rate limiting, caching, transformation, and error handling — without custom code. For an enterprise that manages 50+ APIs across multiple teams, that centralization is the primary value proposition.

Developer Portal

APIM includes a fully customizable developer portal out of the box. Developers browse APIs, subscribe to products, generate API keys, and test endpoints — all without involving the API team. AWS and GCP's native gateways require third-party tooling (e.g., Backstage, Stoplight) to achieve the same.

Pricing Tiers

Tier	Price (approx.)	Notes
Consumption	$3.50 / million calls	Pay-per-use. No SLA for developer portal.
Developer	~$50/month	Dev/test only. No production SLA.
Basic	~$150/month	Up to 1M calls/month included.
Standard	~$750/month	Full features, 99.9% SLA.
Premium	~$3,800+/month	Multi-region, VNET integration, 99.95% SLA.

The Consumption tier looks attractive but lacks the developer portal SLA and several advanced policy features. Standard is the minimum for production workloads with enterprise requirements. In practice, I recommend clients skip Consumption entirely for anything customer-facing — the cost delta between Consumption and Standard is irrelevant next to the engineering hours you'll spend working around its limitations.

GCP: API Gateway, Cloud Endpoints, and Apigee

GCP's three API products have overlapping names and confusingly similar descriptions. Choose the wrong one and you'll hit feature walls or pay for enterprise features you don't need.

Product	Best for	OpenAPI	gRPC	Developer portal	Pricing model
API Gateway	Serverless backends (Cloud Run, Functions)	Yes (OAS v2)	No	No	Per-call
Cloud Endpoints	gRPC services, self-managed gateway	Yes	Yes	No	Per-call
Apigee	Enterprise, partner APIs, monetization	Yes	Yes	Yes	Subscription

Use API Gateway when your backend runs on Cloud Run, Cloud Functions, or App Engine and you want a fully managed gateway with no infrastructure to operate. Import your OpenAPI spec and Google provisions the gateway.

Use Cloud Endpoints when your services speak gRPC, or when you need to host the gateway proxy (ESPv2) on your own runtime for private networking. Cloud Endpoints is also the right choice for local development with gRPC tooling.

Use Apigee when you need a developer portal, API monetization, advanced bot detection, or multi-cloud/hybrid deployment. Apigee is the only GCP option that competes directly with Azure APIM.

GCP API Gateway — OpenAPI-Driven Configuration

# openapi.yaml — GCP API Gateway configuration
swagger: "2.0"
info:
  title: User Service API
  version: "1.0"
host: api.example.com
schemes:
  - https
produces:
  - application/json

x-google-backend:
  address: https://user-service-xyz-uc.a.run.app
  deadline: 30.0

securityDefinitions:
  firebase:
    authorizationUrl: ""
    flow: implicit
    type: oauth2
    x-google-issuer: "https://securetoken.google.com/my-project"
    x-google-jwks_uri: "https://www.googleapis.com/service_accounts/v1/jwk/securetoken@system.gserviceaccount.com"
    x-google-audiences: "my-project"

paths:
  /users/{userId}:
    get:
      summary: Get user by ID
      operationId: getUser
      parameters:
        - name: userId
          in: path
          required: true
          type: string
      security:
        - firebase: []
      responses:
        "200":
          description: User found
        "401":
          description: Unauthorized
        "404":
          description: Not found

# Deploy to GCP API Gateway
gcloud api-gateway api-configs create user-service-v1 \
  --api=user-service \
  --openapi-spec=openapi.yaml \
  --project=my-project \
  --backend-auth-service-account=api-gateway-sa@my-project.iam.gserviceaccount.com

gcloud api-gateway gateways create user-service-gateway \
  --api=user-service \
  --api-config=user-service-v1 \
  --location=us-central1 \
  --project=my-project

The GCP Quota Gotcha

GCP API Gateway enforces quotas at the project level. If you run multiple APIs or environments in the same GCP project, they share quota limits. A traffic spike on one API eats into the quota headroom for every other API in that project. I've seen this catch teams when a batch job saturated API quotas during an off-hours data migration, which cascaded into 429s on the customer-facing API living in the same project — a production incident that had nothing to do with the customer-facing service itself.

Structure your GCP projects to isolate production APIs. One project per environment (dev, staging, prod) is the minimum. One project per service in production is safer for high-traffic APIs.

Feature Comparison

Feature	AWS HTTP API	AWS REST API	Azure APIM (Standard)	GCP API Gateway	GCP Apigee
Auth: JWT	Native	Lambda authorizer	Policy (validate-jwt)	OpenAPI extension	Policy
Auth: API keys	No	Yes	Yes	Yes	Yes
Auth: OAuth 2.0	Partial (JWT)	Lambda authorizer	Policy	No	Yes
Rate limiting	Yes	Yes (usage plans)	Yes (per subscription)	Yes (per consumer)	Yes (advanced)
Request transform	No	Yes (mapping templates)	Yes (policy engine)	No	Yes
Response transform	No	Yes	Yes	No	Yes
Protocol translation	No	No	Yes (REST ↔ SOAP)	No	Yes
Developer portal	No	No	Yes (built-in)	No	Yes
gRPC support	No	No	No	Via Endpoints	Yes
WebSocket	Separate product	No	No	No	No
Multi-region active-active	No (regional)	No	Premium tier	No	Yes
Pricing model	Per-call	Per-call	Tier (monthly)	Per-call	Subscription
Cold starts	Lambda-dependent	Lambda-dependent	None	None	None

Common Patterns

Backend for Frontend (BFF)

The BFF pattern uses a dedicated gateway layer per client type — one for the web app, one for the mobile app — to avoid forcing different clients to negotiate a single general-purpose API.

Diagram: BFF pattern — each client type gets a dedicated aggregation layer tailored to its data needs.

A BFF does data aggregation and transformation — it is not where business logic lives. Business rules belong in the downstream services.

On AWS, implement each BFF as a separate Lambda function or ECS service behind its own HTTP API route. On Azure, use APIM products to route mobile vs web clients to different backends. On GCP, deploy separate Cloud Run services and separate API Gateway configurations.

API Aggregation

Aggregation collapses multiple downstream service calls into a single response for the client. The gateway (or a BFF) fans out the requests in parallel and merges the results.

// API aggregation pattern — call multiple services in parallel
import express, { Request, Response } from 'express';

const app = express();

interface OrderSummary {
  order: unknown;
  inventory: unknown;
  shipping: unknown;
}

/**
 * GET /orders/:id/summary — returns a merged view of order, inventory, and shipping data.
 *
 * @remarks
 * Fans out to three downstream services in parallel. If any service fails,
 * the endpoint returns 502 with the name of the failing service.
 *
 * @returns Merged order summary or 502 with failing service name.
 */
app.get('/orders/:id/summary', async (req: Request, res: Response) => {
  const { id } = req.params;

  const [orderRes, inventoryRes, shippingRes] = await Promise.allSettled([
    fetch(`http://order-service/orders/${id}`),
    fetch(`http://inventory-service/stock/${id}`),
    fetch(`http://shipping-service/shipments/${id}`),
  ]);

  for (const [name, result] of [
    ['order', orderRes],
    ['inventory', inventoryRes],
    ['shipping', shippingRes],
  ] as const) {
    if (result.status === 'rejected') {
      res.status(502).json({ error: `upstream_failure`, service: name });
      return;
    }
  }

  const summary: OrderSummary = {
    order: await (orderRes as PromiseFulfilledResult<Response>).value.json(),
    inventory: await (inventoryRes as PromiseFulfilledResult<Response>).value.json(),
    shipping: await (shippingRes as PromiseFulfilledResult<Response>).value.json(),
  };

  res.json(summary);
});

app.listen(3000);

Protocol Translation

Protocol translation converts between API protocols at the gateway layer. The most common case in enterprise environments is REST-to-SOAP, where a new REST API sits in front of a legacy SOAP backend.

Azure APIM handles this natively with the soap-to-rest policy. On AWS or GCP, you write a Lambda or Cloud Run adapter that performs the translation.

<!-- Azure APIM: expose a REST endpoint backed by a SOAP service -->
<policies>
  <inbound>
    <base />
    <!-- Convert REST JSON body to SOAP XML envelope -->
    <set-body>
      @{
        var body = context.Request.Body.As<JObject>();
        return string.Format(
          @"<soapenv:Envelope xmlns:soapenv='http://schemas.xmlsoap.org/soap/envelope/'
                              xmlns:usr='http://legacy.example.com/user'>
              <soapenv:Body>
                <usr:GetUser>
                  <usr:UserId>{0}</usr:UserId>
                </usr:GetUser>
              </soapenv:Body>
            </soapenv:Envelope>",
          body["userId"]
        );
      }
    </set-body>
    <set-header name="Content-Type" exists-action="override">
      <value>text/xml; charset=utf-8</value>
    </set-header>
    <set-header name="SOAPAction" exists-action="override">
      <value>http://legacy.example.com/user/GetUser</value>
    </set-header>
  </inbound>
  <backend>
    <base />
  </backend>
  <outbound>
    <base />
    <!-- Parse SOAP response and return JSON -->
    <xml-to-json kind="direct" apply="always" consider-accept-header="false" />
  </outbound>
</policies>

Protocol translation is one of the strongest arguments for Azure APIM. Implementing the same pattern on AWS requires a Lambda adapter and custom XML parsing logic. If your organization has legacy SOAP services that need a REST facade, I would pick Azure APIM over any other provider specifically for this — the AWS alternative is a Lambda that becomes a maintenance liability the moment the original SOAP developer leaves.

API Lifecycle Management

Every gateway manages API versions moving through environments. The state diagram below represents the lifecycle that applies regardless of provider.

Diagram: API lifecycle from development through retirement. Breaking changes are only safe before the API reaches Production.

On AWS, implement environment promotion by deploying to separate API Gateway stages (dev, staging, prod). On Azure, use APIM's revision system for non-breaking changes and a new API version for breaking changes. On GCP API Gateway, deploy to separate gateway instances per environment.

Choosing the Right Gateway

Diagram: Decision flowchart for selecting a gateway.

Common Mistakes

Mistake 1: Using AWS REST API when HTTP API covers your needs
REST API costs 3.5× more per million requests and adds ~5 ms of latency over HTTP API. The only reasons to choose REST API over HTTP API are mapping templates for request/response transformation, usage plans with API keys, or Cognito user pool authorizers. If none of those apply, HTTP API is the correct choice.

Mistake 2: Ignoring Lambda cold starts in latency budgets
Teams measure gateway latency and see 6 ms. They assume their API responds in under 100 ms. Then they deploy to production and discover p99 latency is 700 ms because the Lambda behind the gateway cold-starts on every new burst. Cold starts are not a gateway problem — they are a Lambda problem — but they manifest as gateway latency and are often debugged at the wrong layer.

Mistake 3: Choosing Azure APIM Consumption for production without reading the tier limitations
The Consumption tier has no SLA for the developer portal, limits on policy features, and no built-in cache. Teams select it for cost reasons and hit limitations mid-project. Standard is the correct production tier for most workloads.

Mistake 4: Mixing multiple APIs in a single GCP project
GCP API Gateway enforces quotas at the project level. Multiple APIs in one project share quota. A traffic spike on one API reduces the available quota for all others in the same project. Separate projects per environment (and per service in production for high-traffic APIs) is the safe default.

Mistake 5: Building complex API aggregation in gateway mapping templates
AWS REST API mapping templates (Velocity Template Language) and Azure APIM policies can both perform data transformation. Use them for simple field renaming or header injection. Do not use them to aggregate multiple backend calls, implement business logic, or transform complex nested structures — that complexity belongs in a BFF service where it is testable and maintainable.

Production Considerations

Performance

AWS HTTP API adds ~6 ms of median gateway overhead. REST API adds ~11 ms. Set your latency budget accordingly.
Azure APIM Standard tier median overhead is 5–15 ms depending on policy complexity. A policy chain with JWT validation, rate limiting, and transformation adds up.
GCP API Gateway overhead is 5–20 ms. Apigee overhead is higher due to the full policy engine.

Security

On AWS, attach a WAF WebACL to your HTTP API or REST API. The gateway itself validates JWT signatures but does not inspect payloads for SQL injection or XSS patterns.
On Azure APIM, use the ip-filter policy to block known-bad IP ranges and the validate-content policy for schema validation. APIM integrates with Azure DDoS Protection at the Premium tier.
On GCP, pair API Gateway with Cloud Armor for DDoS protection and request filtering. Apigee has built-in bot detection and threat protection.

Cost

AWS HTTP API at 1 billion requests/month: ~$1,000. Lambda execution costs are separate.
Azure APIM Standard: ~$750/month flat plus overage. Standard includes 1M calls/month; additional calls are $3.50/million.
GCP API Gateway at 1 billion requests/month: ~$2,000 (calls over the free tier at $3.00/million above 2M).

Monitoring

On AWS, enable CloudWatch detailed metrics for your API Gateway and set alarms on 4XXError, 5XXError, Latency, and IntegrationLatency. IntegrationLatency isolates backend latency from gateway latency — critical for cold start diagnosis.
On Azure, use APIM's built-in Application Insights integration. Track Backend Duration separately from Gateway Duration to identify whether slowness is in the gateway policies or the backend.
On GCP, Cloud Logging and Cloud Trace integrate automatically. Set a budget alert on API call costs in the GCP Console — GCP quotas can run up quickly on traffic spikes.

Full Example: API Aggregation Service (TypeScript)

This aggregation service sits behind an API Gateway (on any provider) and fans out to three downstream services in parallel. It returns a merged response or reports which upstream failed.

import express, { Request, Response, NextFunction } from 'express';
import { randomUUID } from 'crypto';

const app = express();
app.use(express.json());

// ─── Types ───────────────────────────────────────────────────────────────────

/** Represents the shape of a downstream service response for an order. */
interface OrderData {
  id: string;
  status: string;
  total: number;
  lineItems: Array<{ productId: string; qty: number }>;
}

/** Inventory availability per product. */
interface InventoryData {
  productId: string;
  available: number;
  reserved: number;
}

/** Shipment tracking details. */
interface ShipmentData {
  trackingNumber: string;
  carrier: string;
  estimatedDelivery: string;
  status: string;
}

/** Merged summary returned to the client. */
interface OrderSummaryResponse {
  requestId: string;
  order: OrderData;
  inventory: InventoryData[];
  shipment: ShipmentData | null;
}

/** RFC 7807 problem detail shape for error responses. */
interface ProblemDetail {
  type: string;
  title: string;
  status: number;
  detail: string;
  instance?: string;
}

// ─── Helpers ─────────────────────────────────────────────────────────────────

function problem(
  res: Response,
  status: number,
  title: string,
  detail: string,
  instance?: string,
): void {
  const body: ProblemDetail = {
    type: `https://api.example.com/errors/${status}`,
    title,
    status,
    detail,
    ...(instance ? { instance } : {}),
  };
  res.status(status).contentType('application/problem+json').json(body);
}

/**
 * Fetch a downstream service and parse the JSON response.
 *
 * @remarks Throws with a structured error if the upstream returns non-2xx.
 * Caller is responsible for catching via Promise.allSettled.
 *
 * @param url - Full URL of the downstream service endpoint.
 * @returns Parsed JSON response body.
 * @throws {Error} when upstream returns a non-2xx status.
 */
async function fetchUpstream<T>(url: string): Promise<T> {
  const res = await fetch(url, {
    headers: { 'X-Request-Id': randomUUID() },
    signal: AbortSignal.timeout(5000), // 5 s timeout per upstream call
  });

  if (!res.ok) {
    throw new Error(`Upstream ${url} returned ${res.status}`);
  }

  return res.json() as Promise<T>;
}

// ─── Routes ──────────────────────────────────────────────────────────────────

/**
 * GET /orders/:id/summary — fan-out aggregation across order, inventory, and shipping services.
 *
 * @remarks
 * **Path parameters:**
 * - `id` — Order UUID. Must exist in the order service.
 *
 * **Responses:**
 * - `200 OK` — All three upstreams responded successfully.
 * - `404 Not Found` — Order service returned 404.
 * - `502 Bad Gateway` — One or more upstreams failed or timed out; body identifies which.
 */
app.get('/orders/:id/summary', async (req: Request, res: Response) => {
  const { id } = req.params;
  const requestId = randomUUID();

  const ORDER_SVC = process.env.ORDER_SERVICE_URL ?? 'http://order-service';
  const INV_SVC = process.env.INVENTORY_SERVICE_URL ?? 'http://inventory-service';
  const SHIP_SVC = process.env.SHIPPING_SERVICE_URL ?? 'http://shipping-service';

  const [orderResult, shipmentResult] = await Promise.allSettled([
    fetchUpstream<OrderData>(`${ORDER_SVC}/orders/${id}`),
    fetchUpstream<ShipmentData>(`${SHIP_SVC}/shipments/${id}`),
  ]);

  if (orderResult.status === 'rejected') {
    problem(res, 502, 'Upstream Failure', `Order service failed: ${orderResult.reason}`, req.path);
    return;
  }

  const order = orderResult.value;

  // Fetch inventory for each line item in parallel
  const inventoryResults = await Promise.allSettled(
    order.lineItems.map(item =>
      fetchUpstream<InventoryData>(`${INV_SVC}/stock/${item.productId}`),
    ),
  );

  const inventoryFailure = inventoryResults.find(r => r.status === 'rejected');
  if (inventoryFailure) {
    problem(res, 502, 'Upstream Failure', 'Inventory service failed for one or more products', req.path);
    return;
  }

  const summary: OrderSummaryResponse = {
    requestId,
    order,
    inventory: inventoryResults.map(r => (r as PromiseFulfilledResult<InventoryData>).value),
    shipment: shipmentResult.status === 'fulfilled' ? shipmentResult.value : null,
  };

  res.json(summary);
});

// ─── Error handler ───────────────────────────────────────────────────────────

app.use((err: Error, req: Request, res: Response, _next: NextFunction) => {
  console.error('Unhandled error', { path: req.path, error: err.message });
  problem(res, 500, 'Internal Server Error', 'An unexpected error occurred.', req.path);
});

// ─── Start ───────────────────────────────────────────────────────────────────

const PORT = process.env.PORT ? parseInt(process.env.PORT, 10) : 3000;
app.listen(PORT, () => {
  console.log(`Aggregation service running on port ${PORT}`);
});

Run the service:

npm install
ORDER_SERVICE_URL=http://localhost:3001 \
INVENTORY_SERVICE_URL=http://localhost:3002 \
SHIPPING_SERVICE_URL=http://localhost:3003 \
npm run dev

Test the aggregation endpoint:

# Aggregated response from three services in one call
curl -s http://localhost:3000/orders/order-abc123/summary | jq

# Expected shape:
# {
#   "requestId": "uuid",
#   "order": { "id": "order-abc123", "status": "confirmed", ... },
#   "inventory": [{ "productId": "...", "available": 10 }, ...],
#   "shipment": { "trackingNumber": "...", "status": "in_transit" }
# }

Full source with tests: GitHub link → cloud-apis/api-gateway-patterns/

Conclusion

AWS HTTP API is the right default for Lambda-backed services — it is faster and costs less than REST API, and covers 90% of use cases. Azure APIM wins when your organization needs a developer portal, enterprise API governance, or REST-to-SOAP translation without custom code. GCP's three options serve different tiers: API Gateway for serverless, Cloud Endpoints for gRPC, and Apigee for enterprise. The BFF pattern, API aggregation, and protocol translation are cloud-agnostic in concept but differ significantly in implementation cost — Azure APIM handles protocol translation natively, while AWS and GCP require adapter services. The decision you should walk away from this article having made: stop treating gateway selection as a checkbox in your cloud provider's onboarding wizard. The wrong choice costs you months of workarounds; the right one disappears into your infrastructure and you stop thinking about it entirely — which is exactly what a good gateway should do.

Story Points After AI: When Velocity Metrics Stop Meaning What They Used To

Bry — Wed, 12 Aug 2026 15:00:00 +0000

Key Points

Story points and velocity were built on an assumption: that the effort to draft a solution roughly tracked the effort to deliver it. AI is breaking that link by collapsing drafting effort toward zero while leaving verification effort largely where it was.
A team whose velocity jumps from 50 to 5,000 in a sprint hasn't gotten 100x better. It's produced a number that no longer measures the thing anyone cares about, and more engineering leaders are quietly admitting this every quarter.
What's replacing velocity isn't a single metric. It's a shift toward outcome-oriented measures: defect density in AI-generated code, rework percentage, lead time to value, and issue resolution time, none of which can be inflated just by generating more draft code faster.
Why the Man-Month Is Dying: How AI Broke IT Services' Oldest Pricing Unit covered why the man-month is dying as a pricing unit. This one covers the parallel death of story points as a delivery-measurement unit. The two are breaking for the same underlying reason, on different sides of the same contract.

Introduction

A client-side engineering director told me recently that her team's velocity chart looked like a hockey stick after they rolled out agentic coding tools, and that she'd stopped showing it to her own leadership because it made her look like either a liar or an idiot. Neither was true. Her team had actually improved. The chart just couldn't say by how much, or at what, because the number it produced no longer meant what it used to mean.

Story points measure estimated effort, and velocity is story points delivered per sprint, a proxy that worked reasonably well as long as the bottleneck in software delivery was drafting: writing the code, wiring up the boilerplate, working through the mechanical parts of a ticket. AI-assisted development attacks exactly that bottleneck and leaves the other one, verification, largely untouched. When drafting effort collapses toward zero and verification effort doesn't move much, a velocity number built on the old ratio between the two stops correlating with anything real. Scrum.org's own framing of the problem is blunt: if a team's velocity jumps from 50 to 5,000 in a single sprint, they haven't gotten 100x better. They've broken the metric, because it was always a proxy for effort, and AI reduces the drafting effort to near-zero while leaving verification complexity high.

The industry data backs up how fast this is becoming unignorable. Teams running agentic AI tools report 10-20x velocity gains alongside 40-78% fewer bugs, numbers that sound like unambiguous good news until you try to use them for the same planning and comparison purposes velocity was always used for. You cannot forecast a roadmap, compare two teams, or set a hiring plan against a metric that swings by two orders of magnitude depending on which AI tool got rolled out last quarter. Teams that haven't found a replacement are flying blind with better instruments than they've ever had, which is its own kind of dangerous.

Old Velocity Metrics vs. Post-AI Delivery Metrics

Criteria	Story Points / Velocity	Post-AI Delivery Metrics
What it actually measures	Estimated drafting effort per sprint	Defect density, rework rate, lead time to value, issue resolution time
Inflatable by AI tooling alone	Yes, trivially, without any real gain	No, each metric requires an actual outcome to move
Useful for cross-team comparison	Only if both teams use identical tooling and estimation habits	Yes, outcomes are tooling-agnostic
Useful for client-facing reporting	Increasingly not, since AI adoption accelerated in 2026	Yes, clients can verify a defect rate or a resolution time against their own experience
What a spike in the number tells you	Nothing reliable	That verification-adjusted output actually improved

Recommendation: if your organization is still running sprint retros around a velocity chart, check whether anyone on the team can explain what a 10x jump in that number would actually mean. If the honest answer is "the tooling changed," the metric isn't tracking delivery anymore. It's tracking tool adoption, and you should be measuring that separately from outcome quality.

Where This Is Likely Headed

Right now: denial and dashboards for most teams. Most teams are keeping the velocity chart because the tooling to replace it (defect-density tracking, rework attribution, lead-time-to-value instrumentation) isn't built yet, and nobody wants to be first to admit their headline metric is cosmetic.
Expect a credibility break within the next year or two. As more teams get publicly embarrassed by an unexplainable 50-to-5,000 velocity swing, engineering leadership will stop defending the metric in front of executives and clients. "Agent efficiency" language is already starting to replace "velocity" in serious planning conversations.
Outcome metrics should go from experimental to standard within a few years. Rework percentage and AI-attributed defect rate are becoming normal line items on engineering dashboards, not because they're perfect, but because they can't be gamed by pointing an agent at a backlog overnight.
The end state: metrics become a client-facing artifact, not just an internal one. Firms moving toward outcome-based contracts are starting to share rework and defect data directly with clients, because the metric is finally honest enough to survive that exposure.

Questions to Ask Your Team

If our velocity doubled next sprint, could anyone on this team explain exactly why, in terms a client would find credible?
Are we tracking rework and defect rate on AI-generated code specifically, or lumping it in with everything else and losing the signal?
Would we be comfortable sharing our current delivery metrics directly with a client, or are they really an internal-only number dressed up as evidence of progress?
Is any part of a team's evaluation or bonus still tied to raw velocity, in a way that quietly rewards inflating a number everyone privately agrees is broken?

Conclusion

Story points and velocity aren't failing because someone calculated them wrong. They're failing because the thing they measured, effort, is no longer correlating with the thing everyone actually wanted to know, which was progress. The teams adapting fastest aren't the ones with the flashiest AI tooling. They're the ones willing to admit their dashboard is lying to them and replace it with metrics, defect density, rework rate, lead time to value, that can't be inflated by the same tools that broke the old ones. The man-month is dying on the pricing side of the contract. Velocity is dying on the delivery side. Neither death is really about AI being too good. It's about a proxy finally getting exposed as a proxy.

AWS EventBridge: Event Buses and Rules You Can Script from Day One

Bry — Tue, 11 Aug 2026 15:00:00 +0000

Key Points

EventBridge charges $1.00 per million custom events published; rule evaluation itself is free, up to 300 rules per bus. The real billing surprise isn't rule count — it's payload size, since events bill in 64 KB chunks and the January 2026 limit increase to 1 MB means one large event can now bill as up to 16.
A custom event bus, a rule with a JSON event pattern, and a Lambda target take four CLI commands total. Use InputTransformer on targets to trim events down to only the fields a target needs, rather than forwarding the full body downstream.
Archive and replay are cheap to enable and expensive to forget about — archived data bills monthly storage whether or not you ever replay it.

Prerequisites

CLI/SDK version tested against: aws-cli/2.35.x
An IAM role with events:* and lambda:InvokeFunction permissions
A deployed Lambda function to act as a target — the examples use order-processor

Introduction

EventBridge gets pitched as "SNS but smarter" often enough that people underrate how little code it takes to get real value from it. I've stood up event-driven order processing for two different clients using nothing but a custom bus, a handful of pattern-matched rules, and existing Lambda functions — no message broker to run, no queue infrastructure to patch.

The part that doesn't show up in the marketing is the cost mechanic that changed in January 2026: AWS raised the maximum event payload from 256 KB to 1 MB, which sounds like a pure win until you notice events still bill in 64 KB chunks. A single 1 MB event can now bill as 16 separate chunked events at the custom-event rate. That's not a reason to avoid EventBridge — it's a reason to design your event payloads deliberately instead of dumping an entire domain object into detail.

This article builds a working bus, rule, and target from the terminal, and treats the payload mechanic as a design input, not an afterthought.

Bus, Rules, Targets: The Three Pieces

Diagram: one bus, multiple rules each matching a different event pattern, each routing to its own target.

A rule without a target does nothing but sit there for free. A rule with a matching event pattern and a wired-up target is what actually moves data. Rules and targets are separate API calls on purpose — one rule can fan out to up to five targets.

Building It From the CLI

# 1. A custom bus — separate from the default bus so your rules
#    only ever see events your own services publish.
aws events create-event-bus --name orders-bus

# 2. A rule matching a specific event pattern on that bus.
aws events put-rule \
  --name order-created-rule \
  --event-bus-name orders-bus \
  --event-pattern '{
    "source": ["orders-service"],
    "detail-type": ["OrderCreated"],
    "detail": {
      "status": ["pending"]
    }
  }'

# 3. Grant the rule permission to invoke the Lambda target.
aws lambda add-permission \
  --function-name order-processor \
  --statement-id eventbridge-invoke \
  --action lambda:InvokeFunction \
  --principal events.amazonaws.com \
  --source-arn "arn:aws:events:us-east-1:123456789012:rule/orders-bus/order-created-rule"

# 4. Wire the rule to the Lambda target.
aws events put-targets \
  --rule order-created-rule \
  --event-bus-name orders-bus \
  --targets '[{
    "Id": "order-processor-target",
    "Arn": "arn:aws:lambda:us-east-1:123456789012:function:order-processor"
  }]'

Four commands, and the bus is live. Publish a test event to confirm the wiring:

aws events put-events --entries '[{
  "Source": "orders-service",
  "DetailType": "OrderCreated",
  "Detail": "{\"orderId\":\"order-abc123\",\"status\":\"pending\",\"total\":49.99}",
  "EventBusName": "orders-bus"
}]'

A successful response returns "FailedEntryCount": 0. If it's 1, check Errors in the same response — the two most common causes are a malformed Detail JSON string or a detail-type/source combination that doesn't match any rule, which fails silently rather than erroring (the event is simply dropped with no matching rule, and EventBridge does not treat that as a failure).

Designing Around the Payload-Chunking Rule

Diagram: EventBridge's per-64KB chunking mechanic, effective with the January 2026 payload limit increase to 1 MB.

The fix isn't "never send large events." It's "don't send more than the target needs." Two concrete patterns:

Send a reference, not the full object. If order-created-rule's target only needs the order ID to go fetch full details itself, publish {"orderId": "order-abc123"} in detail, not the entire order document with line items, customer data, and shipping addresses.

Use InputTransformer to trim before delivery. When the target genuinely needs specific fields from a larger event, extract them at the rule level instead of forwarding everything:

aws events put-targets \
  --rule order-created-rule \
  --event-bus-name orders-bus \
  --targets '[{
    "Id": "order-processor-target",
    "Arn": "arn:aws:lambda:us-east-1:123456789012:function:order-processor",
    "InputTransformer": {
      "InputPathsMap": {
        "orderId": "$.detail.orderId",
        "status": "$.detail.status"
      },
      "InputTemplate": "{\"orderId\": <orderId>, \"status\": <status>}"
    }
  }]'

InputTransformer runs on EventBridge's side before delivery — it doesn't reduce what you're billed for publishing the original event, but it does mean your Lambda isn't parsing (and your logs aren't storing) a payload larger than it needs.

Archive and Replay

Archiving lets you replay events later — useful for reprocessing after a bug fix, or reconstructing state after a downstream outage. It's cheap per operation and easy to forget about entirely.

aws events create-archive \
  --archive-name orders-archive \
  --event-source-arn arn:aws:events:us-east-1:123456789012:event-bus/orders-bus \
  --retention-days 30

Component	Price
Archive processing	$0.10/GB archived
Archive storage	$0.023/GB-month
Replayed events	$1.00/million (same as custom event rate)

Storage bills every month an archive exists, whether or not anything ever gets replayed from it. Set --retention-days deliberately — 30 days is a reasonable default for operational replay; longer retention should be a conscious compliance or audit decision, not the default you forgot to change.

Common Mistakes

Mistake 1: Publishing to the default bus for everything
The default bus receives AWS service events too. Mixing your application events into it makes rule patterns harder to write precisely and makes the bus noisier to reason about. Use a custom bus per domain area.

Mistake 2: Assuming a failed match throws an error
It doesn't. An event with no matching rule is dropped silently — put-events still returns success. Test your event patterns against real sample events before relying on them in production.

Mistake 3: Forwarding entire event payloads to every target
This is both a cost issue (larger payloads, more chunked billing on any downstream re-publish) and a coupling issue — targets that receive more than they need tend to accumulate implicit dependencies on fields nobody documented.

Mistake 4: Enabling archive without a retention policy decision
Storage costs accrue every month regardless of replay activity. An archive with no retention limit and no owner is exactly the kind of AWS bill line item nobody can explain six months later.

Production Considerations

Performance: Rule matching is fast and free regardless of rule count up to the 300-per-bus limit. If you're approaching that limit, it's usually a sign you need multiple buses segmented by domain, not one bus straining to model everything.

Security: Use resource-based policies on the event bus (put-permission) to control which accounts or services can publish, rather than relying solely on IAM policies at the producer side.

Cost: Re-run the payload math whenever a producer's event shape grows. A field added "just in case" on a high-volume event source compounds fast at the chunked billing rate.

Monitoring: CloudWatch metrics on TriggeredRules, FailedInvocations, and ThrottledRules are free and on by default — set an alarm on FailedInvocations per target, since a misconfigured target fails silently from the publisher's perspective.

Full Example: Teardown Script

#!/usr/bin/env bash
set -euo pipefail

BUS_NAME="${BUS_NAME:-orders-bus}"
RULE_NAME="${RULE_NAME:-order-created-rule}"

aws events remove-targets --rule "$RULE_NAME" --event-bus-name "$BUS_NAME" --ids order-processor-target
aws events delete-rule --name "$RULE_NAME" --event-bus-name "$BUS_NAME"
aws events delete-archive --archive-name orders-archive || true
aws events delete-event-bus --name "$BUS_NAME"

echo "Torn down $BUS_NAME and its rules."

Full source including the sample Lambda target and pattern-matching tests: GitHub → cloud-apis/aws-eventbridge-cli/

Conclusion

EventBridge is one of the fastest AWS services to script from zero — a working bus, rule, and target is four commands. The part worth designing around deliberately is payload size: with the 2026 increase to a 1 MB event limit, it's easy to publish something large enough to bill as multiple events without noticing. Send references and trim payloads with InputTransformer rather than forwarding full objects downstream, and treat archive retention as a decision you make on purpose, not a default you forgot existed.

DEV Community: Bry

Building a RAG Pipeline from Scratch: Embeddings, Retrieval, and Claude

Key Points

Introduction

What RAG Is — and What It Replaces

RAG Pipeline Architecture

Chunking Strategies

Fixed-Size Chunking

Recursive Chunking

Semantic Chunking

Embeddings

Local Embeddings: sentence-transformers

API Embeddings

Vector Storage with ChromaDB

Choosing a Vector Database

Retrieval Strategies

Cosine Similarity Search

Maximal Marginal Relevance (MMR)

Hybrid Search (BM25 + Semantic)

Context Augmentation

Generation with Claude

The RAGPipeline Class

Evaluation

Common Mistakes

Full Example

Conclusion

Further Reading

Who Absorbs the Margin — Client, SIer, or the AI Vendor: Renegotiating SI Economics

Key Points

Introduction

Where the Margin Actually Goes: Three Claims on the Same Savings

Defending Margin With a Real Number Instead of a Vague Claim

Questions to Ask Your Team

Conclusion

Further Reading

Amazon SQS vs SNS: Queues, Fan-Out, and Picking the Right One from the CLI

Key Points

Prerequisites

Introduction

Queue vs Topic, Conceptually

Standalone SQS

Standalone SNS

The Fan-Out Pattern — Including the Step Everyone Skips

FIFO Throughput Limits

Common Mistakes

Production Considerations

Full Example: Fan-Out Setup Script

Conclusion

Further Reading

The Multi-Tier Subcontracting Pyramid Under Pressure: What Happens to Body-Shop SIers When AI Writes the Code

Key Points

Introduction

Pyramid Layer Economics Under AI Compression

What's Likely to Survive, and What a Lower-Tier Firm Should Change Now

Questions to Ask Your Team

Conclusion

Further Reading

AI Cost Optimization: A Business Guide to LLM API Budgeting

The AI Bill Nobody Expected

Why AI Costs Spiral

How LLM Pricing Works

The Biggest Lever: Model Selection

Prompt Caching: Pay Once, Reuse Many Times

Response Caching: Skip the API Call Entirely

Batch Processing: Trade Speed for 50% Off

Budget Controls and Monitoring

Adoption Roadmap: 4 Phases to Controlled AI Spend

Questions to Ask Your Engineering Team

Conclusion

Further Reading

From Billing Hours to Billing Outcomes: New Contract Structures for AI-Assisted SI Projects

Key Points

Introduction

Contract Model Fit by Project Type

Making the Switch: A Phased Approach

Questions to Ask Your Team

Conclusion

Further Reading

Azure Event Grid: Push-Based Events Without the Kafka Overhead

Key Points

Local Embeddings: `sentence-transformers`

The `RAGPipeline` Class