DEV Community: Lars

turbovec: Local RAG Without the 60 GB Tax

Lars — Mon, 25 May 2026 16:43:12 +0000

A 1536-dimensional float32 embedding is 6 KB. A corpus of 10 million documents is roughly 60 GB of raw vectors before any index overhead. That doesn't fit in laptop RAM, and even on a machine with 64 GB you've left yourself no headroom for anything else.

I kept reaching for FAISS. It works, but I kept hitting two friction points: training requires a representative sample of your corpus upfront, and compression quality depends on how well that sample matches the real distribution. If your data distribution shifts, you're rebuilding.

turbovec solves both, and the TurboQuant paper (arXiv April 2025, Google Research + NYU) explains the math behind why it can skip the training step entirely.

What TurboQuant actually does

The core idea is a mathematical trick: apply a random rotation to your vectors before compressing them.

After rotation, each coordinate follows a scaled Beta distribution that converges to Gaussian N(0, 1/d) in high dimensions. The coordinates also become nearly independent. That combination is what makes training-free quantization possible: you can precompute optimal bucket boundaries from pure math, with no data required upfront.

The algorithm in four steps:

Normalize each vector to unit length; store the norm as a float
Apply a fixed random rotation matrix (same matrix for the whole index, computed once at setup)
Quantize each rotated coordinate against precomputed bucket boundaries; at 4-bit that's 16 buckets per coordinate
Pack the integers: a 1536-dim vector goes from 6,144 bytes (float32) to 384 bytes

A 10M-doc corpus: ~60 GB float32 becomes ~7.5 GB at 4-bit, an 8x reduction. The paper proves the MSE distortion lands within a factor of √3·π/2 ≈ 2.7 of the information-theoretic lower bound at any bit-width, which is tight for a training-free method. At 4-bit specifically, MSE is approximately 0.009.

Search doesn't decompress vectors. It rotates the query once into the same domain and scores against codebook centroids using SIMD kernels (NEON on ARM, AVX-512 on x86). Per turbovec's own benchmarks, on ARM it beats FAISS IndexPQFastScan by 12-20%.

The part I initially glossed over: MSE and inner product are different problems

For RAG, what matters is preserving similarity scores, and MSE-optimized quantizers don't do that.

When you search a vector index, you're finding stored vectors with the highest dot product against your query. The TurboQuant paper proves that quantizers optimized purely for reconstruction accuracy introduce bias into inner product estimates. The compressed vectors rebuild accurately, but their similarity scores with a query vector are systematically off. You get wrong nearest neighbors.

TurboQuant fixes this with a two-stage approach. Stage one applies MSE quantization at one fewer bit than your target budget (so 3 bits if you want 4-bit total), which minimizes reconstruction error and shrinks the residual as much as possible. Stage two takes that residual and applies a 1-bit random projection transform called QJL (Quantized Johnson-Lindenstrauss). QJL is an optimal 1-bit inner product quantizer: it reduces the residual to a single bit per dimension using sign(random_matrix · vector), and the paper proves this makes the combined estimator unbiased.

The whole thing is data-oblivious. It works on the first vector you add to the index. The result is near-optimal reconstruction accuracy and unbiased similarity scores at your target bit-width.

For KV cache compression in long-context LLMs (storing attention keys and values), the paper tests Llama-3.1-8B on LongBench-E: 3.5 bits per channel matches unquantized quality, 2.5 bits shows only marginal degradation, while compressing the cache by more than 5x. The inner product unbiasedness property is what makes it work for attention computation.

The practical part: one import swap

turbovec ships drop-in replacements for the in-memory vector stores in LangChain, LlamaIndex, Haystack, and Agno. For LangChain:

pip install turbovec[langchain]

# Before
from langchain_core.vectorstores import InMemoryVectorStore

# After — same API, smaller footprint, faster search
from turbovec.integrations.langchain import TurboVecVectorStore as InMemoryVectorStore

Everything else in the pipeline stays the same. I swapped this into an existing LangChain project in a few minutes. Memory dropped by roughly 8x and retrieval got a bit faster.

For IdMapIndex (when you need stable IDs that survive deletes):

from turbovec import IdMapIndex
import numpy as np

index = IdMapIndex(dim=1536, bit_width=4)
index.add_with_ids(vectors, np.array([1001, 1002, 1003], dtype=np.uint64))

scores, ids = index.search(query, k=10)
index.remove(1002)  # O(1) by id

What the pgvector benchmarks actually show

I have been exploring turboquant for use with pgvector. To evaluate its performance, I ran the RAG benchmarks created by Johann-Peter Hartmann.

The storage and index scan wins are real. At 4-bit, your vector column shrinks by around 8x, and index scans run faster because you're moving far less data through memory. On a large corpus, that gap is meaningful.

The retrieval quality story is less clean. Quantizing inside pgvector degrades recall measurably compared to full float32 search. You can lose real top candidates from your top-k window. The TurboQuant unbiasedness proof is mathematically correct, but unbiased inner product estimates still carry variance at 4 bits, and in dense retrieval that variance pushes results around. The second-best document in float32 might not appear in your top-10 at 4-bit.

Two cases where the trade-off still makes sense: storage-constrained deployments where approximate retrieval is acceptable, or pipelines that rerank with a cross-encoder anyway (the reranker recovers from retrieval noise). If you're running semantic search where missing the true top result matters, measure your recall on a held-out set before committing.

If you want to run this comparison yourself against your own corpus, here's the benchmark setup I used:

import numpy as np
import time
from turbovec import IdMapIndex

dim = 1536
num_vectors = 1_000_000

embeddings = np.random.randn(num_vectors, dim).astype("float32")
embeddings /= np.linalg.norm(embeddings, axis=1, keepdims=True)
ids = np.arange(1, num_vectors + 1, dtype=np.uint64)

queries = np.random.randn(100, dim).astype("float32")
queries /= np.linalg.norm(queries, axis=1, keepdims=True)

t0 = time.perf_counter()
index = IdMapIndex(dim=dim, bit_width=4)
index.add_with_ids(embeddings, ids)
print(f"build: {time.perf_counter() - t0:.2f}s")

latencies = []
for q in queries:
    t0 = time.perf_counter()
    index.search(q, k=10)
    latencies.append(time.perf_counter() - t0)

avg_ms = np.mean(latencies) * 1000
p99_ms = np.percentile(latencies, 99) * 1000
print(f"avg: {avg_ms:.2f}ms  p99: {p99_ms:.2f}ms")

No training pass, no codebook warmup. The index is ready to search after the first add_with_ids call. Swap in your real embeddings and IDs, then run the same timing loop against FAISS IndexPQFastScan at the same bit-width to get a direct comparison.

When FAISS is still the right tool

turbovec is an in-memory flat index: it searches all vectors on every query. For a few million vectors on a single machine, that's fine. At hundreds of millions you need IVF partitioning to reduce the search scope, and FAISS handles that.

The ARM picture is clean: turbovec beats FAISS IndexPQFastScan by 12-20% across typical configurations. x86 is more conditional. At 4-bit, turbovec wins by 1-6% due to tighter cache lines and faster bit-unpacking. At 2-bit single-threaded, they run within 1% of each other. At 2-bit multi-threaded on AVX-512 hardware, FAISS pulls ahead by 2-4%; it exploits AVX-512 VBMI for bit manipulation during concurrent sweeps, an instruction path turbovec doesn't yet use. On enterprise x86 with high thread counts at 2-bit, that edge is real.

At high dimensions (d=1536, d=3072), turbovec matches or beats FAISS at R@1; both converge to 1.0 recall by k=4-8. At d=200 (GloVe territory), turbovec trails at R@1 because the near-Gaussian approximation from the random rotation weakens at low dimensions.

The rule: turbovec for local RAG with modern embedding dimensions, FAISS for very large corpora, GPU-accelerated search, or multi-threaded 2-bit lookups on AVX-512 servers.

What I'm using it for

I'm running turbovec in ThoughtForge for per-space semantic search. The nomic-embed-text-v1.5 model produces 768-dimensional embeddings; at 4-bit compression the full index is small enough that loading at app startup takes under a second. Local embeddings, local index, no data leaves the machine.

If you're building local RAG and hitting the float32 memory wall, this is the first thing I'd try.

Modern React Performance Without the Overhead

Lars — Mon, 25 May 2026 16:40:20 +0000

A product manager says "we'll use React, performance won't be a problem." Two sprints later, the Lighthouse INP score is 800ms and the main thread is blocked on a 400 KB vendor bundle nobody audited.

React itself isn't the problem. The flexibility is.

INP replaced FID, and most teams haven't adjusted

Google's Core Web Vitals now measure three things:

LCP (Largest Contentful Paint): loading speed for elements inside the viewport. Anything below the fold doesn't count.
CLS (Cumulative Layout Shift): visual stability. React 19's native document metadata handling solves a category of CLS issues caused by stylesheets loading after first paint.
INP (Interaction to Next Paint): the one most teams are failing now. It replaced First Input Delay in March 2024 and measures every interaction across the page lifetime, not just the first. A single slow click handler tanks your score.

FID was easy to pass because it only measured the first interaction. INP measures all of them, which means large synchronous bundles and heavy render trees anywhere on the page will show up.

Three browser profiles, not one

Your daily browser lies to you. Extensions inject scripts, manipulate the DOM, and run background tasks. Lighthouse scores measured under those conditions aren't representative of what users see.

Set up three profiles in Chrome or Firefox:

Dev profile: all your extensions (React DevTools, Apollo, Redux, etc.). Use this for development.
Profiling profile: zero extensions, CPU throttling enabled. Use this for Lighthouse and manual profiling.
Normal profile: whatever you use day-to-day.

Always profile a production build. Development mode adds diagnostic overhead that inflates render times and bundle sizes in ways that have nothing to do with what ships. Run npm run build, serve it locally, then open Lighthouse in the clean profile. The Lighthouse Tree Map shows you which dependencies are inflating which chunks; look there before opening the bundle analyzer.

The React Compiler doesn't fix bad state placement

Before React 19, avoiding cascading re-renders meant wrapping things in useMemo and useCallback. It worked when done carefully, failed silently when done wrong, and cluttered codebases either way.

The React Compiler statically analyzes your component tree and applies automatic memoization. You write straightforward JavaScript; the compiler handles the caching. This eliminates the category of "forgot to memoize this callback" bugs.

What it doesn't fix: state placed too high in the component tree.

If you have a text input and the user sees lag under CPU throttling, check where the state lives. The runtime evaluates every component between the state and the consumer on each keystroke. Move state as close to the consuming component as possible. The compiler can't restructure your component hierarchy; that decision belongs to you.

Bundle size and layout shift

Heavy components loaded before they're needed block the main thread. React.lazy and Suspense fix this by fetching a component only when it's required:

const HeavyChart = React.lazy(() => import("./HeavyChart"));

function Dashboard() {
  return (
    <Suspense fallback={<ChartSkeleton />}>
      <HeavyChart />
    </Suspense>
  );
}

Check packages on Bundlephobia before importing. A date-picker that pulls in 80 KB gzipped when you need one function is a problem you choose.

React 19 adds native support for resource hoisting. You can prefetch DNS and preload assets from components without managing <head> manually, and <title>, <meta>, and <link> tags rendered anywhere in the component tree are automatically hoisted to <head>. That removes the need for react-helmet or equivalent libraries for the common case:

import { prefetchDNS, preload } from "react-dom";

prefetchDNS("https://fonts.googleapis.com");
preload("/hero.webp", { as: "image" });

For CLS: give images explicit width and height attributes. The browser reserves the correct space before the asset arrives; without them, the layout shifts when the image loads and you lose CLS points for every user on a slow connection. Use loading skeletons rather than spinners; a skeleton anchors the layout geometry while data fetches.

Actions and optimistic UI

Form handling in React pre-19 involved isLoading state, disabled buttons, and a UI that froze until the server responded. That pattern hurts INP because the interaction latency is the full round-trip time.

React 19 introduces Actions and useOptimistic. The interaction updates the UI immediately; the server call runs in the background:

"use client";
import { useOptimistic, useActionState } from "react";
import { addToCart } from "./actions";

export default function BuyButton({ product }) {
  const [optimisticQty, addOptimistic] = useOptimistic(0, (q, delta) => q + delta);
  const [state, action, pending] = useActionState(addToCart, { ok: false });

  return (
    <form action={(formData) => {
      addOptimistic(1);
      action(formData);
    }}>
      <input type="hidden" name="productId" value={product.id} />
      <button disabled={pending}>
        {pending ? "Adding…" : "Add to cart"}
      </button>
      <p>In cart: {optimisticQty}</p>
    </form>
  );
}

The click registers immediately. INP measures the time between the interaction and the next paint; with useOptimistic, that's milliseconds rather than the full server latency.

React 19 also introduces use(), a new primitive that reads Promises or Context directly inside the render phase. Unlike hooks, it can be called conditionally, so you can suspend a component mid-render while a Promise resolves rather than managing that state manually:

import { use, Suspense } from "react";

function UserName({ userPromise }) {
  const user = use(userPromise);
  return <span>{user.name}</span>;
}

export default function Profile({ userPromise }) {
  return (
    <Suspense fallback={<span>Loading…</span>}>
      <UserName userPromise={userPromise} />
    </Suspense>
  );
}

The Suspense boundary catches the suspension; the parent doesn't need to know a Promise is involved at all.

What the RSC boundary actually costs

Working with React Server Components on high-traffic sites, the payload crossing the network boundary is where performance gets lost, not the client rendering.

The mistake: passing full database objects as props to Client Components. Every field on that object gets serialized into the RSC payload even if the component uses three of them. On a product page with a 40-field database record, that's a lot of JSON the browser decodes and discards.

Pass exactly the fields the client needs, nothing more. Push "use client" as far down the tree as possible so the boundary is small. Stream static layout immediately and wrap slow data fetches in <Suspense>:

async function ProductPage({ id }) {
  const product = await db.products.findById(id);

  return (
    <div>
      <StaticLayout />
      <Suspense fallback={<ProductSkeleton />}>
        <ProductClient
          name={product.name}
          price={product.price}
          inStock={product.inStock}
        />
      </Suspense>
    </div>
  );
}

I came up from backend work before moving heavily into frontend and edge delivery, so I've seen both sides of this equation. The bottleneck is almost never the framework itself; it's the payload crossing the network boundary and what you decided to put in it.

The tooling in React 19 is genuinely better. The structural rule is the same as it's always been: don't ship what the user doesn't immediately need.

AWS Summit Hamburg 2026: The Year Agentic AI Went from Hype to Production

Lars — Mon, 25 May 2026 16:34:38 +0000

Hamburg 2026 was my first AWS Summit in person; I'd only followed it through recordings before. The shift in the AI sessions was immediate. Last year the talks were mostly roadmaps: architecture diagrams for systems still being built, phrases like "we're exploring" and "we're excited about the potential." This year, several major companies, from automotive manufacturing to energy, e-commerce, and food delivery, each opened with results.

The infrastructure for agentic AI is real now. The questions still open are about safety, not capability.

Strands Agents: letting the model decide

I started the day with the Strands Agents talk. We're already using Strands at work, so I expected mostly familiar ground. What was actually new was how far they've pushed the model taking over orchestration decisions that developers used to make manually.

The shift sounds small but changes everything: you don't define which agent to call when. The orchestrator figures it out. The model reads the task, decides what capability it needs, and routes itself. That means less brittle wiring in your agent graph and more flexibility when edge cases appear at runtime.

Three capabilities stood out:

Self-extending: Agents write their own Python tools on the fly. Because the framework can reload tools at runtime, an agent can recognize a missing capability, write the code for it, save it, and execute that new tool immediately, without a restart.

Self-directing: Agents update their own system prompts based on interactions and store those updates in persistent memory. When deployed with Amazon Bedrock AgentCore, they use Amazon Bedrock Knowledge Bases to retrieve past sessions, so context accumulates across conversations.

Meta-agents: A primary orchestrator spawns sub-agents based on the task. Three patterns: Swarm (sub-agents work on parts of the problem in parallel, sharing a context space), Graph (directed hand-offs between specialized agents in sequence), and Think (the agent recursively spawns instances of itself to deepen reasoning before answering).

Similar talk from AWS re:Invent 2025: Strands Agents deep-dive.

When an enterprise HR platform runs itself

A global energy company's HR team had a legacy data distribution system. Expensive, rigid, built for a world where data formats and sources didn't change often. They replaced it with a serverless platform on AWS, powered by Amazon Bedrock Agents and Amazon Bedrock AgentCore.

What made it interesting wasn't the migration. It was what they built on top: the platform is self-documenting, self-modifying, and self-healing. When a new data source appears, the system builds the ingestion Lambda and Step Functions itself. When a transformation fails, it diagnoses the failure, writes a fix, and deploys it. Engineers can ask the platform questions in natural language and get answers about its own internals.

The result: $800K in annual license savings, and a data delivery platform that largely operates without engineers in the loop for routine changes.

That's Level 3 maturity, which I'll explain below. Most teams aren't there yet. But seeing it running in production at a company of this size makes it harder to argue it's years away.

Supply chain planning with agents

A global automotive manufacturer has been running ML-based forecasting and planning on AWS for years. The agentic layer they've added isn't a replacement for that infrastructure; it's a reasoning layer on top of it, one that can work through problems humans currently have to resolve manually.

The concrete example that stuck with me: "where is my tire and when will it arrive?" Sounds simple. Answering it requires querying across production schedules, capacity constraints, distribution paths, and carrier networks simultaneously, then reconciling those answers into something actionable. Before agentic AI, that took hours. Now it takes five minutes.

The architecture uses specialized agents that investigate production blocks and capacity constraints in parallel, then synthesize their findings into business-friendly explanations. That last part matters: the team specifically called out the move from black-box optimization to transparent decision-making. Supply chain planners needed to understand why the system made a recommendation, not just what it recommended. Agents that can explain their reasoning are the only version that gets adopted.

Privacy-first, then ship

One talk stood out for its honesty about what the path to production actually looks like. The team behind a European e-commerce platform didn't start with a multi-agent architecture. They started with a single Bedrock API call for intent classification and predefined response templates, built data privacy compliance in from day one, then iterated from there: prompt engineering, Bedrock Agents, multi-agent orchestration, and finally Amazon Bedrock AgentCore. Each phase had a ceiling where the simpler approach stopped being good enough.

The result of that iteration: over 50% of first-level support tickets are now handled by AI, and resolution time is down 60%.

Privacy was the first design constraint throughout, not an afterthought. Before any ticket content reaches an LLM, it goes through an anonymization layer that strips PII using pattern matching and rules. Only anonymized content goes forward. The gate pipeline:

flowchart LR
    A[Ticket arrives] --> B[Anonymize PII]
    B --> C[Route to team]
    C --> D[Confidence check]
    D --> E[Add context]
    E --> F[Lifecycle check]
    F --> G[AI response]

One detail I hadn't considered before: they tried sending the AI response five seconds after ticket submission. Customer satisfaction dropped. People felt the speed was inhuman and didn't trust the answer. They now add an artificial delay. The content didn't change, the timing did, and satisfaction went back up.

The system splits B2C and B2B into separate agent chains, because B2B tickets carry more company-specific context and require more specialized handling. Observability is built in from the start: every response is traced, retrieval success rates and model usage are tracked, and quality scores feed back into the system. That observability also changed how the team debugs. The question shifted from "what did the model answer?" to "why did the workflow behave this way?", which means tracing which tools were called, what was retrieved and how relevant it was, where latency came from, and what CSAT scores and ticket reopen rates are telling you about specific workflow paths.

flowchart LR
    A[Normalized Support Request] --> B[Router Agent]

    B --> C[Consumer Support Agent]
    B --> D[B2B Support Agent]
    B --> E[Human Handoff Agent]

    C --> F[Ticket Update Composer]
    D --> F

    F --> G[Ticket API]

Multi-agent does not mean everyone talks to everyone.

Bounded specialization reduces accidental cross-talk and makes handoff behavior easier to reason about. A few architectural decisions that make this work in practice:

The Router Agent decides which specialized agent handles the request; nothing else does routing.
Specialized agents are isolated by responsibility and don't call each other directly.
Only the Ticket Update Composer writes back to external systems (Ticket Systems).
Human escalation is an explicit path, not a fallback you discover at 2am.
Tool permissions and handoff contracts are defined up front, not negotiated at runtime.

The architecture is only part of what makes this production-grade. Agents introduced operational concerns that became part of the product itself:

IaC and reproducibility: agents, collaborators, roles, Lambda functions, API Gateway, and knowledge base configuration all need reproducible deployment. Drift between environments is a real failure mode.
Aliases and versioning: promote tested versions explicitly; don't rely on draft agent behavior in production.
Latency budgets: multi-tool workflows can exceed webhook timeouts. Latency is a design constraint, not a monitoring afterthought.
Structured traces: log intent, retrieval, tool inputs, API errors, and response payloads. Debugging an agentic workflow without traces is guesswork.
Human QA sampling: review low CSAT scores, ticket reopen rates, and escalation reasons on a regular cadence.
Cost guardrails: cap agent steps, retries, token budgets, and retrieval depth. Unbounded agents are a billing incident waiting to happen.

The customer sees the answer. Engineering owns the routing, policy, traces, and feedback loop.

Choosing the right level of agent complexity is its own decision. Not everything needs a full multi-agent setup:

Use case	Direct Bedrock call	Bedrock Agent	AgentCore
One decision, known outputs	Best fit	Overkill	Overkill
Needs RAG + tools	Possible but manual	Best fit	Good fit
Multiple topics / generated answer	Limited	Best fit	Good fit
Cross-channel middleware	Possible	Best fit	Good fit
Longer-term memory / operations	Limited	Partial	Best fit

Don't optimize for "most agentic." Optimize for the minimum autonomy that solves the customer problem with acceptable operational risk.

Their lesson for anyone starting this: get your data security and privacy team involved before you write the first line of agent code. The design decisions they'll ask for are the ones you can't retrofit later.

AgentCore: building multi-tenant AI as a service

I also caught the expert-level session on building multi-tenant SaaS agents with Amazon Bedrock AgentCore. If you're building a platform where multiple customers each get their own AI agents, the isolation problem is harder than it looks.

Tenant isolation in agentic systems runs across five dimensions: identity (each tenant's agents act with scoped credentials), memory (one tenant's conversation history can't leak into another's context), gateway (routing and rate-limiting per tenant), observability (tenant-scoped traces so you can debug without seeing another customer's data), and runtime (compute isolation so a runaway agent in one tenant doesn't affect others). The session walked through working examples for each.

The framing they used, "intelligence as a service," is worth keeping. If you're building AI capabilities that other teams or customers consume, the SaaS constructs of onboarding, isolation, and identity propagation apply just as much as they do to any other service you'd build. The AgentCore primitives give you the building blocks; you still have to wire them together intentionally.

Similar session recording: Building Multi-Tenant SaaS Agents with Amazon Bedrock AgentCore.

One published example that shows AgentCore working on a real regulated domain: an open-source medical content review application that cross-checks pharmaceutical marketing claims against clinical references, PubMed, OpenFDA, and ClinicalTrials.gov. A few architectural decisions in it are worth studying regardless of your domain. First, the reviewer sub-agents persist their findings to S3 and return only an S3 URI to the orchestrator; hundreds of findings from a 30-page document never flow through the orchestrator's context window, which would cause it to summarize and drop findings. Second, user identity is extracted server-side from the JWT sub claim, never from the request payload, which closes the impersonation-via-prompt-injection vector directly. Both patterns are reusable. Full writeup and open-source repo: Accelerate Medical Content Review with Amazon Bedrock AgentCore.

The maturity ladder

Across several talks, a rough framework emerged for how companies are thinking about agentic AI maturity.

Level 1 is rules-based AI. The system follows defined policies, humans define every decision path, and the AI fills in specific gaps.

Level 2 is autonomous task AI. The system handles entire workflows: self-documentation, quality monitoring, task routing. Humans oversee outcomes, not individual steps.

Level 3 is self-monitoring systems. The system diagnoses its own failures and builds capabilities it didn't have before. Human oversight is exception-based, not routine.

Most of the companies presenting were somewhere in the Level 1 to Level 2 transition. The energy company's HR platform was the one example of Level 3 running in production. That gap is worth knowing about before you start planning, because Level 2 requires different architecture decisions than Level 1, and Level 3 requires different trust decisions than Level 2.

What's still open

The results are real: $800K saved, hours to five minutes, resolution time down 60%. These aren't demos.

But three problems came up in nearly every talk, and nobody presented a clean solution.

Hallucination in production has operational consequences now. When your agent is writing transformation plugins and deploying them, a confidently wrong answer triggers real failures. Teams are managing this with human-in-the-loop gates at specific checkpoints.

Prompt injection and prompt protection got less stage time than they deserved. As agents act on data from external systems, the attack surface grows. One concrete mitigation that came up in the AgentCore examples: extract user identity from the JWT server-side, never from the request payload, so attackers can't impersonate users through crafted input. That closes one specific vector; the broader problem of agents acting on adversarial content from external sources is still largely unsolved in practice.

Data privacy at scale is hard even with PII anonymization. Cross-border data flows, multi-system agents, and context accumulating in memory all create compliance complexity that rules-based anonymization doesn't fully address.

These are reasons to build carefully, with privacy by default and observability from day one. They're not reasons to wait.

Context engineering is the next skill

Prompt engineering was the first visible handle: write better prompts, get better outputs. The feedback loop was tight. The harder problem in agentic systems is what surrounds the prompt: what context an agent has access to, when it gets loaded, how much fits in the window, and what happens when it doesn't.

The AgentCore medical content review example makes this concrete. Reviewer sub-agents persist findings to S3 and hand the orchestrator a URI instead of the content; the full set is loaded once, at the final editing pass. That's a context engineering decision: controlling which information exists in which agent's window at which moment, specifically to prevent the model from silently dropping findings it can't fit.

The same pattern shows up in the Strands memory model (Bedrock Knowledge Bases retrieve relevant past sessions, not all of them) and in the e-commerce platform's lifecycle gates (explicit stages controlling what context is available at each step). Every multi-agent architecture at the summit made the same choice. The common thread is explicit decisions about what each agent knows and when.

Context windows are a real production constraint, not a theoretical one. As agents chain together and produce intermediate output, what you carry forward and what you discard is an architectural choice. Getting it wrong makes the system quietly incorrect in ways that are hard to debug, because the model won't tell you it dropped something.

Treat context design with the same seriousness as a data model.

What I took away

The energy was different from the recordings I'd watched in previous years. Companies were comparing notes, sharing what broke and what they'd do differently.

The part worth repeating to anyone building with AI right now: share the positive stories. The results from these companies are real and worth talking about with your team. And while you're doing that, keep hallucination and data protection as first-class design constraints, not late-stage reviews. Include the people who care about those things early; they make the system better, not slower.

These are my personal impressions from the conference. The views here are my own and don't represent my employer or any of the companies mentioned. If I got something wrong or misunderstood a detail, ping me on LinkedIn and I'll correct it.