DEV Community: Israel Vásquez

Make AI Generated Code Traceable with Requirement IDs

Israel Vásquez — Mon, 12 Oct 2026 17:03:33 +0000

A numbered requirement can connect a product decision to the task, test, and code that implement it. In our spec driven workflow, IDs such as 1.1 and 1.3 give engineers and AI agents a durable path from a feature’s intent to its implementation.

That link matters because an AI coding session can leave behind code without the reasoning that produced it. When requirements and decisions live only in a prompt or an agent’s temporary plan, future changes have to reconstruct the original intent from the implementation.

Put feature intent in the repository

For non trivial features, we write a specification before implementation. It lives in a dedicated specs/ folder, usually as Markdown, and describes expected behavior, relevant interactions, edge cases, technical decisions, and how to verify the result.

The spec is a shared source of truth for the team and the agent. It is not documentation added after the work, or a place to collect every engineering convention. The goal is to make feature intent available while the work is being designed, tested, and changed.

This approach is especially useful when a brief leaves product behavior unresolved. For example, free tier limits require decisions about what happens when a user reaches a storage limit, attempts an upload, or upgrades. Turning those scenarios into testable requirements gives the agent something more precise to follow than a broad request to “add limits.”

Use IDs to connect intent to code

Each requirement receives an identifier, such as 1.1. That same ID can appear in the task list, tests, implementation comments, and relevant code paths. A developer who encounters a behavior later can follow the reference back to the spec instead of inferring its purpose from code alone.

IDs also help keep tests tied to expected behavior. In the workflow, the agent reads the approved spec, writes tests first, and implements until they pass. The tests are grounded in identified requirements, so they can express which behavior they verify rather than merely showing that a particular implementation runs.

Keep the references lightweight. The ID points to the explanation; it does not need to carry the whole requirement into every comment or test name. That keeps code readable while preserving a route back to the decision.

Approve decisions before implementation

The process has four phases: requirements, design, tasks, and implementation. A person approves each phase before the next begins.

Requirements turn the feature brief into acceptance criteria. Design makes technical choices explicit, including API shape, database design, data model changes, product behavior, and architectural constraints. Tasks break the approved plan into implementation steps. Only then does implementation start.

This sequence gives an agent reviewed context before it changes code. It also creates checkpoints where a person can correct a misunderstanding before it spreads into design, tasks, tests, and implementation. After implementation, automated reviewers such as Gemini or Cursor Bug Bot can review the code before local testing.

The workflow is still evolving. In particular, tasks can drift or become harder to keep aligned as features grow. Mapping each task to a commit can make the connection between the spec, work performed, and code history cleaner, but teams should watch whether their task structure stays useful over time.

Keep feature specs separate from repo rules

A feature spec should describe that feature’s behavior and decisions. General engineering conventions belong elsewhere, such as repository instruction files, agent rules, or skills. Templates can keep requirements, design, and task files consistent.

This boundary prevents every new spec from repeating coding preferences or carrying unrelated architectural policy. It also helps an agent find the right context: feature behavior in the spec, durable project conventions in repository rules, and repeatable procedures in skills.

Broader architecture changes need the same care. A notifications feature, for instance, raises questions about delivery timing, triggering events, persistence, and channels. Those are feature decisions to resolve. Introducing a message queue or a new data store is a broader architectural shift, and may require repo level rules or skills rather than being smuggled into one feature spec.

A practical starting point is to create one spec for a feature with several meaningful edge cases. Number the acceptance criteria, reference them in tasks and tests, and keep the implementation tied to the approved design. Then review whether someone joining later can trace a behavior back to the requirement that explains it.

Specs should ship with the code because future engineers and agents need the intent alongside the implementation.

What shipping Sanity projects taught us about content models that survive redesigns

Israel Vásquez — Thu, 24 Sep 2026 22:18:01 +0000

Across our Sanity projects, the most reusable technical decision was not a clever GROQ query. It was defining the content before defining the pages.

For Kasada, Orb, Contextual AI, Zenity, Hy-Vee, Newfront, and our own M8 site, the schema determined whether content could be filtered, cross-linked, migrated, previewed, and rendered beyond one page template. The patterns below come from our documented Sanity implementations, where the same platform supported marketing sites, internal tools, legacy migrations, and multi-site content operations.

Model entities, not URLs

Structured content stores meaning and relationships independently from its final presentation. A case study remains a case study whether it appears as a card, a hero, or an entry in a filtered index.

That distinction sounds small, but it changes how a system responds to redesigns. A schema named after a URL or page layout tends to inherit assumptions from that interface. When the layout changes, the schema often has to change with it.

We instead model stable entities. Authors, clients, technologies, and sectors become documents rather than repeated strings. Other content references those documents. This makes filtering, cross-linking, and resurfacing possible without adding another manually maintained field for every new use.

Editors still need control, but not necessarily control over every visual property. We use a fixed set of design system blocks so editors can determine sequence and emphasis while the application owns spacing, color, and responsive behavior. On M8, modular sections can be reordered and recombined while preserving the exposed grid and visual system.

The tradeoff is intentional. Editors cannot construct arbitrary layouts inside the Studio, but the pages they publish remain inside the tested design system.

Map a migration before moving content

A CMS migration is primarily a mapping problem.

Our process begins with an audit of the legacy archive. We decide what should carry over, what needs rewriting, and what should be retired. Only after those decisions are reflected in the target content model do we move the content.

For Contextual AI, this meant leaving the legacy system with the existing content intact while introducing a model capable of supporting what came next. The archive survived, but the limitations of its previous representation did not have to become permanent features of the new schema.

Starting with an export reverses that sequence. It encourages the team to reproduce the source CMS inside Sanity, including page-shaped records, duplicated strings, and fields that existed only to satisfy an old template.

A useful migration plan therefore describes both sides of every mapping: what the legacy field represented and which new document, reference, or structured field should own that meaning. The migration script should implement an already settled model, not become the place where the model is improvised.

Let query complexity expose modeling problems

Sanity uses GROQ to query and transform structured content. GROQ is expressive enough to compensate for many schema problems, which is exactly why query complexity deserves attention.

Our rule of thumb is concrete: if rendering one card requires 3 nested joins, inspect the schema before tuning the query. The issue may be that data used together is modeled too far apart, or that presentation concerns have leaked into the content structure.

Portable Text follows the same principle. We treat it as structured rich text, not stored HTML. The application decides how the structure should render on a web page, in a feed, or for a machine reader. That avoids maintaining separate copies of the same body for each surface.

This becomes increasingly important when content serves both people and machines. On Kasada, structured content, explicit references, and machine-readable markup supported enterprise buyers and automated consumers from the same model. The structure was part of the architecture, not something added later through a plugin.

On M8, we also use Sanity Typegen to generate TypeScript types from GROQ queries. This keeps the relationship between the content model, query result, and application code visible to the type system instead of relying on manually maintained interfaces.

Treat the editor workflow as an application

A customized Sanity Studio is a frontend application for a specific group of users. Its information architecture, input components, previews, and validation deserve the same attention as the public interface.

The Hy-Vee KidsFit implementation is a useful example. Editors add a YouTube URL in Sanity, and the integration retrieves the video thumbnail, title, and description through the YouTube API. GROQ also powers dynamic resource filters on the site.

That input does more than store a URL. It turns a repeated editorial task into a consistent integration point. Editors spend less time transferring metadata manually, and the public interface receives a predictable content shape.

For Zenity, the structured model gave the marketing team enough flexibility to update content and create pages without developer support. Sanity's collaborative editing also allowed multiple team members to work simultaneously.

Live previews on M8 complete the feedback loop by showing creators how changes will appear before publication. If an editor regularly needs a developer to interpret a field or publish a routine page, we treat that as a modeling or Studio design problem.

Balance publishing speed with maintenance

Our Sanity sites commonly use Next.js on Vercel. Server Components read from Sanity directly, and publication triggers on-demand revalidation. Editors see updates without waiting for a full rebuild, while readers receive static performance.

That architecture separates publishing from deployment, but it does not eliminate maintenance. A customized Sanity Studio has dependencies, upgrades, custom components, and its own release surface. Teams choosing Sanity should account for that application alongside the public site.

This leads to a practical selection criterion. Sanity fits when the content contains stable entities and relationships worth preserving, editors benefit from a purpose-built workflow, and the team can maintain the Studio as software. If the main requirement is a hosted editing interface with nothing to customize or upgrade, a simpler CMS may be a better fit.

When those conditions do hold, the durable asset is not any individual page. It is a content model that can survive the next layout, integration, publishing workflow, and consuming application.

What two Contentful builds taught us about content models and publishing boundaries

Israel Vásquez — Wed, 23 Sep 2026 22:56:51 +0000

For Addiction Education Society (AES), we modeled resources, stories, and core pages in Contentful so staff could update the public site without developer support. The team now publishes changes in minutes rather than days. For CrewAI, we mapped reusable page components to Contentful models, then built a separate handoff that turns published Ghost articles into Contentful drafts. Both projects used the same content management system (CMS). The useful engineering decision was where to place editorial control, rendering rules, and the final publication gate.

A headless CMS stores structured content separately from the code that renders it. That separation helps only when the content model reflects what editors need to create and what engineers need to keep consistent. These projects exposed 3 questions worth answering before migrating a page: What is reusable content? Which system owns each step of publication? Which existing systems should stay in place?

Model the things editors actually manage

At AES, the main requirement was editorial independence. Staff needed to maintain information as the nonprofit's education and outreach work grew. We created Contentful models for resources, stories, and core pages. Those categories gave the team structured material to build and update pages independently, including for time-sensitive initiatives. The AES case study reports that updates which took days can now be published in minutes.

CrewAI needed a different level of composition. Its previous CMS could no longer support the website it needed. We designed reusable components in Figma, including hero layouts, pricing sections, feature grids, statistics modules, and tabbed content. Each component mapped to a structured Contentful model. Marketing could assemble pages from those blocks, while developers rendered the layouts through GraphQL with strong type safety. In the CrewAI case study, the shift from individual templates to reusable sections is the mechanism behind that division of work.

The model therefore served as a contract. An editor could choose and populate a supported section; engineering owned how that section rendered and how it fit with analytics, search optimization, and integrations. Contentful's reference based type system makes these relationships possible, but the types still have to be designed. A collection of loosely related entries would give editors fields to fill without giving the site a dependable composition system. That is the modeling cost we account for up front.

Draw the migration boundary around the work

AES also shows why a CMS migration does not have to mean replacing every application under a domain. Its WordPress learning system supported active educational programs. Rebuilding that system along with the marketing site would have risked downtime or lost functionality. We launched a new marketing platform using Astro and Contentful alongside the existing learning system. Public pages, articles, and donation flows were modernized while the programs continued. Visitors still had an experience under one domain, and AES retained a path to migrate more functionality later.

That boundary mattered operationally. Staff gained control of public content without waiting for the learning platform to be rebuilt. The Contentful work solved a publishing problem; the surrounding architecture kept an existing program available. The project also used Contentful's pro bono program, which made the platform accessible to this qualifying nonprofit. Pricing was part of the platform decision, even though the implementation followed the same content modeling discipline as a larger site.

CrewAI crossed a different boundary. We moved its site away from page specific templates toward structured sections in Contentful, but kept Ghost for article writing. Treating every existing tool as something to replace would have disrupted a workflow the editorial team already valued. The two projects suggest a practical migration exercise: list the tasks people perform in each system, then move only the tasks the new architecture is meant to improve.

Make the publication gate explicit

CrewAI's writers publish articles in Ghost. An automated workflow converts each published Ghost article into a draft in Contentful. It stops there deliberately. Marketing then reviews the content, adds metadata, adjusts page composition, schedules it, and publishes it to the public site. Ghost owns the writing experience; Contentful owns the website release decision.

The draft state is the critical part of that integration. If the handoff went straight to public publication, the team would lose the documented review and enrichment step. If writers had to repeat their work manually in Contentful, the handoff would become another editorial bottleneck. The Contentful project page describes this pipeline as the integration point; the case study shows the specific roles on each side of it.

This is a useful question for any CMS integration: does an upstream “published” event mean ready for the public website, or ready for the next team to review? In CrewAI's workflow, those are separate states. Naming them in the architecture gives both teams a clear handoff and lets each keep the tool suited to its work.

Choose the platform with its constraints visible

Contentful gave AES structured content and staff publishing control; it gave CrewAI reusable page composition and a controlled handoff from Ghost. Neither outcome came from installing a CMS alone. We had to define content types, decide which existing systems would remain, and specify the point where a draft becomes public.

The platform also has costs. Its type system rewards careful modeling, and the Contentful project page notes both the budget impact and the absence of a built-in visual canvas for page building. For a small team with simple pages, that may be more structure than the work requires. For a site where several teams need to assemble pages and govern releases, the structure has a clear job. Start the architecture discussion with the editorial handoffs and the content that must be reused; they reveal whether the CMS belongs at the center, alongside another tool, or only on one part of the site.

What a Next.js and Vercel rebuild has to solve beyond deployment

Israel Vásquez — Tue, 22 Sep 2026 18:53:22 +0000

On Contextual AI’s website rebuild, we defined Sanity schemas before moving content out of WordPress. The schemas had to represent blog posts, research articles, and use cases while the new Next.js site was still taking shape. That sequence made the migration possible alongside development, but it also exposed a practical limit: content from the old site did not always fit the new layouts. Scripts could carry structure and metadata across; people still had to review and adapt the result.

That is a useful way to think about a Next.js and Vercel implementation. The framework and hosting platform matter, but the project’s value depends on how content, review, design, and integrations work together after launch. Our work on Contextual AI, Zenity, and Orb shows where those decisions become concrete.

Model the content before moving it

Contextual AI needed a new visual identity and a site that could accommodate changing content needs. We started development with an intensive pass on the Sanity schema, refining it for the different types of data that would come from WordPress. Once the schemas were ready, migration began while the Next.js architecture and new sections were still being built. Most of the import used automated scripts to preserve structure and metadata; manual review caught content issues and helped adapt material to layouts that did not match the old site one to one.

The distinction matters in a rebuild. A script can transfer fields, but a new design may change what each field is supposed to communicate. If a research article, use case, and blog post all have different editorial jobs, treating them as interchangeable pages can leave editors with awkward workarounds. Contextual AI’s documented approach began with those content types, then combined automation with review where the new experience called for judgment.

We also added real-time content previews to make publishing more intuitive. That connected the content model to the publishing experience. A technically correct import is only part of the job if the team cannot confidently maintain the site afterward. The Contextual AI project documents the migration and the continued work after its launch deadline.

Make review part of the deployment workflow

On Zenity’s site, Vercel Preview Deployments gave product, marketing, and leadership teams a place to review work directly on staging links. Built-in comments let them point to the interface they were discussing. In the Zenity project, we describe shorter feedback loops, fewer meetings, and faster progress toward launch from that workflow.

This is an operational use of deployment previews, beyond checking that a build runs. A design review is more useful when the reviewer can see the page in a browser and attach feedback to the actual state of the interface. It also gives engineers a concrete reference for a requested change. The decision to include nonengineering stakeholders in preview review affects how a team ships, even though it does not change the application’s runtime architecture.

Zenity’s content workflow used Sanity for a related reason. Its structured model let the team update and create pages without developer support, while collaborative editing allowed multiple people to work in the CMS. Preview links and editable content served different parts of the same publishing process: one supported review of the implementation; the other supported ongoing changes to the site.

Keep the interface connected to the systems behind it

Zenity also needed forms that matched its interface while sending submissions into HubSpot. We built custom forms using HubSpot’s forms API, connecting the site’s presentation layer to the client’s customer relationship management system. That work is easy to overlook in a discussion of Next.js performance, yet it determines whether a polished page can support the team’s follow-up workflow. The case study describes the integration and its intended lead capture flow; it does not publish conversion results.

For Orb, the maintenance question appeared in the content model. Sanity organizes pages, modules, and messaging as reusable blocks, giving the team a way to update campaigns and product messaging without asking a developer to edit every page. Relume helped explore layouts and content hierarchy early, before final design. We then used Next.js and Tailwind CSS for the site, with Vercel handling deployment and global delivery. The Orb project describes hybrid rendering as part of the approach to keeping content fast while retaining flexibility.

Orb’s visual requirements added another layer. We integrated Unicorn Studio for shader effects and interactive motion using WebGL. That choice shows why performance work cannot be reduced to selecting a framework: visual assets and interaction techniques are also part of what the browser must render. The project describes the motion and the site’s performance qualitatively, without publishing a measurement that would let us isolate the effect of the animation.

Treat the stack as responsibilities, not a checklist

Across these projects, Next.js provides the site architecture; Vercel provides deployment and preview workflows; Sanity gives editors structured content. Those names describe different responsibilities. Their value comes from the seams between them: schema design before migration, content previews during editing, deployment previews during review, and API integrations where the website has to support business operations.

That is also a practical way to scope a rebuild. Identify the content types and who will maintain them. Decide how old content will be mapped and reviewed. Put the people who approve the interface into a preview workflow. List the external systems a page must send data to or receive data from. Then make rendering and media choices in the context of the actual experience being built. Our Next.js and Vercel development page names the broader services; these project decisions show what that work looks like in practice.

Designing a Transactional Voice Agent for Latency, Safety, and Human Handoff

Israel Vásquez — Thu, 17 Sep 2026 19:51:24 +0000

A production voice agent has to do more than understand a request. It must respond within the rhythm of a phone conversation, coordinate live APIs, protect sensitive data, and transfer the caller without losing context when automation reaches its limit. In our travel booking implementation, that meant completing the full shop, select, pay, book, and confirm workflow through an existing Amazon Connect environment. The system reached a 92% successful end-to-end transaction rate using live APIs and saved 10 to 15 minutes per successful automated booking.

The useful lesson was not simply that a large language model could handle a booking. The result depended on decisions around the model: how work was divided, how latency was masked, where sensitive operations stopped, and when a person took over.

Add AI as a path through the existing system

The client already had Amazon Connect and the APIs required to complete a reservation. We added conversational AI as a new path inside that environment instead of replacing the contact center or recreating its transactional systems.

Amazon Lex transcribes the caller's freeform request and identifies the basic intent. Claude Haiku handles quick classification, while Claude Sonnet performs the deeper reasoning required to compare options and run the booking APIs. The agent can search live inventory, discuss alternatives, adjust to caller preferences, send a secure payment link by text, confirm the reservation through the client's Payment API, and trigger the confirmation email while the caller remains on the line.

This approach keeps the AI layer focused on conversation and coordination. Existing systems remain responsible for the operations they already perform. For teams working with mature contact center infrastructure, that is a useful boundary: first identify the smallest new path that can orchestrate the current systems, then replace infrastructure only when the existing components block the required workflow.

Treat latency as an end-to-end budget

Voice exposes delays that would be less disruptive in a text interface. Our end-to-end response latency target was less than 2 seconds, but transcription and text-to-speech already consumed 1 to 2 seconds before the model had much room to respond. The implemented response time was 2 to 3 seconds.

True streaming was not ready for this implementation, so we used a practical workaround. The model streamed its reply into DynamoDB. Amazon Lex checked the stored output every few hundred milliseconds and began speaking once it found complete sentences. This was not true streaming, but it reduced the silence the caller experienced and allowed speech to sound natural.

The transferable principle is to budget latency across the whole turn, not just model inference. Measure speech recognition, orchestration, API calls, model generation, buffering, and speech synthesis as one user experience. When a platform capability is missing, optimize the part the caller perceives. Here, sentence-level delivery mattered more than waiting for a technically ideal streaming path.

Choose orchestration and models by workload

We began with LangChain because it handled tool calling and context well. When AWS introduced Bedrock AgentCore as part of the Strands framework, we moved the orchestration into that stack. Its built-in memory management simplified session handling, and tighter integration with Amazon Connect, AWS Lambda, and DynamoDB reduced the infrastructure the team had to maintain. Keeping orchestration within AWS also matched the client's requirement that the system stay in its AWS environment.

Model selection followed the same workload-specific reasoning. We tested newer AWS models for speed and integration, but they did not provide the quality required for complex, multistep booking logic. Claude Haiku remained the fast classifier, and Claude Sonnet handled option selection and API execution.

This split is more useful than choosing one model for every turn. Classification and transactional reasoning have different latency and quality requirements. Test them as separate workloads, then assign each to the least expensive or fastest model that still meets its acceptance criteria. A newer model or a tighter platform integration is not an upgrade if it weakens the critical path.

Keep sensitive operations outside the AI boundary

The agent creates a payment link, but it never handles payment data. Customer identity is verified through the client's own lookup system, and payment remains behind the existing secure Payment API. Prompt guardrails prevent the model from storing or repeating sensitive details beyond the current call. Conversation state is stored in DynamoDB, while policies and rules are retrieved from a Bedrock Knowledge Base.

That separation narrows what the model can expose or mishandle. The model can guide the transaction and invoke approved operations without becoming the system of record for identity or payment. For transactional agents, define this boundary before refining prompts: list what the model may interpret, what it may initiate, and what must remain entirely inside trusted systems.

Automation also needs a designed exit. If the agent could not resolve a request within 2 turns, it transferred the call to a human agent with the conversation context and identified customer account intact. The caller did not have to repeat the request.

This makes escalation part of the architecture rather than an exception. A useful handoff policy needs a trigger, the context to transfer, and a destination capable of continuing the workflow. Together, those pieces let routine calls stay automated while disputes, irregular operations, and distressed customers reach people equipped to help.

[Boost]

Israel Vásquez — Tue, 15 Sep 2026 16:55:56 +0000

Israel Vásquez for Monogram

Sep 14

How to Engineer a Multi-Agent Pipeline for Production Reliability

#ai #llm #architecture #googlecloud

4 min read

How to Engineer a Multi-Agent Pipeline for Production Reliability

Israel Vásquez — Mon, 14 Sep 2026 21:08:21 +0000

We built a nine-agent pipeline that turns a natural language use case into working code, an interactive preview, and an implementation guide in under 4 minutes. It serves roughly 800 to 1,000 users per day, but its first production version consumed about 30,000 tokens and made 15 to 20 Pro-tier model calls per request. The useful lesson from this multi-agent AI implementation is not simply that more agents can solve a larger problem. It is that production reliability comes from deciding which work belongs to a model, what context must be mandatory, and where strict contracts should replace freeform generation.

Decompose work around failure boundaries

The system needed to analyze a requested use case, find relevant APIs, generate visual configuration, write code, inspect that code, repair defects, and produce implementation documentation. Putting all of those responsibilities into one prompt would have created an oversized context, less predictable output, and no clean place to isolate failures.

We divided the workflow into nine agents. A root agent manages session state, routing, and runtime instruction composition. Specialized agents handle analysis, styling, code generation, evaluation, refinement, and documentation. Each receives a narrow responsibility, an explicit input, and an expected output.

That separation matters because it creates operational boundaries. A style configuration can be validated without rerunning code generation. An evaluator can reject unsafe code without rewriting unrelated sections. Each stage can be tested, tuned, and assigned a model based on the work it performs.

Decomposition also carries a cost. Every boundary introduces orchestration, state transfer, and another potential failure point. The architecture was appropriate because the workflow already contained distinct tasks with different validation rules. Splitting an arbitrary prompt into several agents would only distribute the ambiguity.

A practical criterion is to look for independently testable failure modes. If a task can be given a clear contract and evaluated without judging the entire result, it may deserve its own stage.

Use deterministic code when probability adds no value

Two stages, the Schema Validator and Documentation Builder, make no large language model calls.

The Schema Validator performs deterministic checks on generated configuration. The Documentation Builder detects APIs in the final code and maps them to documentation URLs from a canonical registry. Neither task benefits from linguistic creativity. Implementing them programmatically removes 2 possible sources of hallucination and reduces token consumption for those stages to zero.

The same principle shaped the boundaries between code generation, evaluation, and refinement. Asking one agent to generate code and then approve its own output creates a weak feedback loop. We separated the evaluator from the refiner. The evaluator applies strict rules and security scanning; the refiner receives specific findings and makes targeted corrections.

Every agent that generates code or configuration must also satisfy a structured output schema. This prevents freeform commentary from leaking into machine-consumed output, reduces unnecessary tokens, and gives downstream agents parseable input without an extraction step. More importantly, each handoff becomes independently validatable.

Models are useful where interpretation or synthesis is required. Validation, lookup, and contract enforcement should remain ordinary software when their rules can be expressed directly.

Make critical context mandatory

The Model Context Protocol, or MCP, gives the analysis, code generation, and code evaluation agents access to live API documentation through an indexed knowledge base. Query results are capped to control context growth, and the MCP server runs as an isolated service so documentation retrieval does not compete with model calls for API quota.

This keeps knowledge separate from agent code. Documentation can change without requiring the agents to be redeployed.

Retrieval alone did not solve the correctness problem. MCP tool calls are initiated by the agent, so a confident model can decide that it already knows an API and skip the lookup. That creates a path to plausible but invalid code.

We addressed that gap with dynamic instruction injection. The analysis stage identifies which APIs a request needs. Canonical examples for those APIs are then inserted directly into downstream instructions. Open-ended questions still use MCP retrieval, while required implementation patterns arrive as mandatory context.

This distinction is broadly useful. Use optional retrieval for information the model may need to explore. Inject context directly when correctness depends on the model seeing it. After moving from static examples in every prompt to relevant examples selected at runtime, the system reduced per-request token consumption by 73%.

Optimize from production evidence

The first release used Gemini 2.5 Pro for code generation, evaluation, and refinement. It worked, but using the most capable model throughout the pipeline made both cost and latency unsustainable at production volume.

The system evolved through five engineering phases. Dynamic instruction injection removed irrelevant prompt material. Model tiering assigned work according to task requirements and made evaluation cycles roughly 70% faster, with no measured loss of quality. Caching and budget controls reduced repeated work, while BigQuery telemetry and structured observability made the effects measurable.

Reliability work extended beyond average latency. The pipeline includes security auto-fix loops, graceful fallback to the previous valid output, and three-layer retry behavior. Its environments are managed with Terraform across development, staging, and production, with automated CI/CD gates and Google Cloud services including Vertex AI Agent Engine, Cloud Run, Secret Manager, and Cloud Logging.

The transferable method is to begin with explicit stage contracts, observe production behavior, and optimize the stages that evidence identifies. Model choice, prompt context, retries, caching, and deterministic code are all controls within the same system. Treating them that way turns an agent demo into a service that can deliver consistent results at daily production volume.

How we made 2,000 customer conversations queryable in a few hours

Israel Vásquez — Tue, 08 Sep 2026 20:32:43 +0000

A scheduled run processed roughly 2,000 airline support conversations in a few hours. Once processing finished, leadership could ask questions in plain language and receive answers in seconds without rerunning the analysis.

That separation between evaluation and retrieval was the central architectural decision.

In this customer journey intelligence project, we were working with chat logs, voice recordings, routing events, conversation attributes, and agent interaction logs. The data covered the same customer journeys, but it lived in different systems and did not share a reliable schema.

Normalize the journey before evaluating it

The model could not evaluate a conversation until the system knew which records belonged together.

Each environment used a different name for the shared conversation identifier. Timestamps did not always align, and voice and chat records arrived at different levels of detail. We built a normalization layer that mapped each environment into one journey model, then ordered every message, transfer, hold, and resolution chronologically.

This work was programmatic. The LLM received a coherent customer journey rather than being asked to infer relationships between disconnected records.

That distinction matters. A capable model can still produce a confident assessment of incomplete evidence.

Separate analysis from querying

The platform has 2 execution paths.

The scheduled path joins the source data, transcribes voice with Amazon Transcribe, removes personally identifiable information, and sends the reconstructed journey to Claude Sonnet for evaluation. The results are then stored for later use.

The interactive path reads those stored results. The dashboard, data explorer, and conversational agent can search and synthesize the analysis, but they do not rescore the original conversations.

This design fits reporting, incident investigation, and trend analysis, where per-second processing is unnecessary. Batch execution also makes compute usage and LLM costs easier to predict. A team can scope a run to a 50-conversation incident review or a 2,000-conversation monthly analysis.

Real-time processing would introduce more infrastructure and cost without improving these workflows.

Keep evaluation criteria outside the model

The system evaluates intent, sentiment, urgency, resolution, compliance, and specialist performance using the client's existing quality rubrics.

We considered the rubrics part of the application configuration. When the quality team changes a criterion, it can update the prompt instead of collecting new training data and fine-tuning another model.

Claude Sonnet was selected after smaller models lost accuracy on nuanced criteria, including sentiment across multi-turn conversations and whether an escalation was appropriate. The cheaper model was not cheaper if analysts could not trust its judgments.

This approach keeps the evaluation process adjustable, but it also requires prompt governance. Rubric changes should be versioned and tested because they change the meaning of every score produced afterward.

Remove PII before inference

Redaction runs after journey assembly and before the model receives any content.

The redaction step is deterministic, applies to every conversation, and consumes no model tokens. Sensitive information does not enter the prompt or appear in stored evaluation results.

Post-processing model output would have been too late. By that point, the sensitive data would already have crossed the inference boundary.

Calibrate against human decisions

An LLM evaluator needs a reference set. We compared its output with conversations the client's analysts had already scored and refined the rubrics where the two diverged.

Exact agreement reached about 85%. Agreement exceeded 90% when scores within 1 point on the 1-to-10 scales counted as matches.

Both measurements are useful. Exact agreement reveals consistency, while the tolerance measurement shows whether disagreements are materially different or simply adjacent values on a subjective scale. The acceptable tolerance should be defined before evaluating the model, not chosen afterward to improve the result.

What to carry into your own system

For similar analysis platforms:

Reconstruct the complete domain object before sending it to an LLM.
Use deterministic code for joins, ordering, validation, and redaction.
Separate expensive evaluation from latency-sensitive retrieval.
Keep changing business criteria in versioned rubrics.
Test model quality against examples scored by domain experts.
Choose batch or real-time execution from the user workflow, not from the novelty of the architecture.

The conversational agent is the visible part of the product. Its reliability comes from the normalized, redacted, calibrated evidence underneath it.

[Boost]

Israel Vásquez — Thu, 03 Sep 2026 15:30:04 +0000

Israel Vásquez for Monogram

Sep 2

How to Build a Website for Browsers and AI Agents From a Shared Content Model

#webdev #nextjs #ai #architecture

5 min read

How to Build a Website for Browsers and AI Agents From a Shared Content Model

Israel Vásquez — Wed, 02 Sep 2026 20:08:22 +0000

The Walden Robotics website uses 19 reusable content components and publishes 6 discovery surfaces for automated clients. Both start with the same architectural decision: content is modeled once, then delivered in the form each client needs.

We built the platform with Prismic and the Next.js App Router. Editors assemble pages from structured components, visitors receive generated HTML, and clients that prefer structured text can request Markdown. Discovery endpoints tell automated systems where to find APIs, authorization metadata, agent documentation, and news content.

This architecture connects content modeling, delivery, and discovery. HTML and Markdown remain tied to the same records, while the discovery layer identifies the resources available to automated clients.

Build the content model before the delivery formats

Every page on the Walden Robotics site is assembled from structured content in Prismic. We created 19 Slice Machine components covering hero sections, calls to action, lead forms, content blocks, navigation, announcements, and supporting layouts.

A slice defines the fields editors control, while the application controls how those fields render. Editors can combine sections into new pages without changing the component code. Developers retain control over rendering, accessibility, search optimization, and visual consistency.

The platform also separates page content from shared records. Navigation, footer content, SEO defaults, and global announcements each have a single source of truth. A change to a global announcement is made once in Prismic and reflected wherever the application renders that record.

That distinction gives the content model clear ownership:

Page content: Sections editors arrange within an individual page.
Global content: Navigation, defaults, and announcements shared across routes.
Behavioral configuration: Form definitions and messages that affect application behavior.

These categories can live in the same CMS, but they do not carry the same risk. Changing a headline affects presentation. Changing a validation rule affects how the application accepts input.

Keep publication and performance in the same workflow

The site uses the Next.js App Router with static generation, caching, and automatic revalidation when content changes in Prismic. Visitors receive generated pages, while editors can publish updates without requesting a manual rebuild.

Draft Mode provides a separate path for unpublished work. Authorized editors can preview a page before publication without changing how public traffic is served. Preview remains an editorial state, and the public site keeps the performance characteristics of static delivery.

Static generation makes cache freshness an explicit part of publishing. The platform connects Prismic content changes to revalidation, so editors work in the CMS while the application updates its generated output.

Treat CMS-configured forms as untrusted input

Walden's marketing team defines forms in Prismic, including field types, validation rules, placeholder text, and success messages. The same system supports contact forms, campaign landing pages, and promotional experiences.

Once CMS data controls application behavior, the browser becomes an untrusted boundary. Browser validation improves the user experience, but a user or script can modify a request before submission. The platform signs every form field definition with HMAC before the form reaches the browser and validates the submitted data on the server.

HMAC, or hash-based message authentication code, uses a secret key to produce a signature for a message. The server can recompute that signature to determine whether the field definition changed after it was issued. HMAC verifies integrity and authenticity; it does not encrypt the definition.

The form pipeline also accepts requests only from approved Basin endpoints. Honeypot fields, request timeouts, and redirect controls provide additional protection against automated abuse and malicious requests. Each control addresses a separate failure mode.

The reusable pattern is a trust boundary around behavioral configuration:

Sign configuration that crosses an untrusted client.
Validate submitted data again on the server.
Restrict the services and destinations that may receive requests.
Bound external interactions with timeout and redirect controls.
Combine abuse detection with validation and endpoint restrictions.

Editorial flexibility expands what a marketing team can configure; server-side enforcement keeps that flexibility inside defined limits.

Keep operational data in its operating system

The same ownership principle applies to career listings. Available positions come from Rippling's public applicant tracking system, where recruiters already manage openings. The Careers section stays current without requiring marketing or engineering to copy each role into Prismic.

Prismic remains authoritative for the editorial experience around those listings. Rippling remains authoritative for recruiting data. Next.js combines both sources in the rendered page, preserving the workflow of the team responsible for each type of information.

Serve HTML and Markdown from the same records

The platform supports HTTP content negotiation, allowing a client to request Markdown when structured text is preferable to HTML. News articles, CMS pages, and supporting content are also available through familiar .md URLs.

HTTP content negotiation allows a client and server to select a representation of a resource. On Walden's platform, human visitors can receive the full website experience, while automated clients can receive Markdown generated from the same underlying content.

Markdown does not replace HTML. HTML carries document semantics, links, interface behavior, and presentation for browsers. Markdown gives clients that primarily need headings, paragraphs, and lists a direct representation of the content. Generating both from Prismic avoids a second editorial repository and keeps the representations tied to the same source records.

How will automated clients find the content?

A clean representation has limited value if software has to guess where it lives. Walden's platform publishes 6 discovery surfaces:

/.well-known/api-catalog
/.well-known/mcp/server-card.json
/.well-known/agent-skills/
/.well-known/oauth-authorization-server
sitemap.md with Link headers
/api/news/articles with an OpenAPI 3.1 description

Discovery and representation solve different problems. Markdown provides content in a form an automated client can process. Catalogs, descriptions, and Link headers help that client locate the available resources and understand how to use them.

The /.well-known/ prefix has a specific role. RFC 8615 defines it as the path prefix for registered well-known locations. Walden's /.well-known/api-catalog endpoint follows RFC 9727, which defines a discoverable location and link relation for an API catalog.

The news API uses OpenAPI 3.1, a language-agnostic interface description for HTTP APIs. It allows people and software to understand a service without inspecting its source code or observing network traffic.

Other resources in the list, including the MCP server card and agent skill documentation, should be described according to the implementation or convention they follow. Placing a resource under /.well-known/ does not automatically make its contents an official standard.

What other teams can reuse

The Walden Robotics platform establishes 4 principles that apply beyond this project:

Keep ownership explicit: Store each type of information in the system where the responsible team maintains it.
Generate representations from shared records: Use structured content to produce HTML, Markdown, or other formats without duplicating editorial work.
Apply trust at behavioral boundaries: Sign configuration, validate requests on the server, and restrict external interactions.
Separate delivery from discovery: Provide useful representations, then publish the catalogs, descriptions, and links that make them findable.

At Monogram, we treat these as parts of the same platform architecture. Content ownership determines what the system can render, validation determines which instructions it can trust, and discovery determines which clients can use the result.

Use formal standards where they exist, document the remaining surfaces precisely, and keep every representation tied to its authoritative source.

AEO for developers: treat content as infrastructure

Israel Vásquez — Wed, 02 Sep 2026 19:36:40 +0000

A 2025 Pew Research Center study found that users clicked traditional results in 8% of Google visits with an AI summary, compared with 15% when no summary appeared. Links cited inside those summaries received clicks in only 1% of visits.

That changes what visibility means. A page can influence a decision without receiving the visit and developers increasingly control whether its information can be found, interpreted, and cited correctly.

AEO is not a replacement for SEO; it is an additional requirement on the publishing system.

Start with the retrieval path

Google says its generative search features still depend on core Search ranking systems and indexed web pages. It also says there are no special technical requirements, machine-readable files, or schema types required for inclusion.

The familiar foundation still matters: crawlable URLs, accessible text, internal links, canonical tags, accurate metadata, and structured data that matches the visible page. AI Mode and AI Overviews may issue several related searches before assembling a response, so a single prompt can produce multiple retrieval paths through a site.

For developers, the first AEO audit is straightforward: verify that important information can be reached and rendered without relying on blocked scripts, private APIs, or ambiguous redirects. Google documents the requirements for AI features in Search.

Model facts before adding markup

Schema can help search engines interpret a page, but it cannot repair inconsistent source content.

Product specifications, authors, publication dates, locations, prices, and availability should be governed fields in the CMS—not details copied independently into page content, metadata, JSON-LD, feeds, and APIs. A shared content model lets each delivery layer draw from the same source.

This architecture reduces a common failure mode: the page says one thing, the structured data says another, and an external profile contains an older value. AI systems can retrieve every version; they cannot resolve an inconsistency the platform created.

The practical pattern is simple: model a fact once, validate it, then publish it through the formats that need it.

Make crawler access an explicit policy

AI platforms use different user agents for different purposes. Search retrieval, user-requested access, and model training should not be treated as a single permission.

OpenAI, for example, distinguishes OAI-SearchBot, which supports discovery in ChatGPT search, from GPTBot, which publishers can block when excluding pages from potential model training. Perplexity separately documents PerplexityBot for search results and Perplexity-User for user-triggered access. (OpenAI publisher guidance, Perplexity crawler documentation)

Encode those decisions across robots.txt, CDN rules, firewalls, and page-level controls. Then check server logs. A policy file describes the intended behavior; request logs show whether the intended crawler can reach the intended content.

Measure representation, not only traffic

Rankings and referral sessions remain useful, but they do not show whether an AI-generated response represented the organization accurately.

Bing Webmaster Tools now reports citations, cited pages, grounding queries, and visibility trends across supported AI experiences. Google has also introduced dedicated generative AI performance reports for a subset of Search Console properties. (Bing AI Performance, Google Search Console announcement)

Pair those platform reports with a controlled prompt set. Track which pages are cited, whether important entities and specifications remain accurate, and where responses repeatedly retrieve outdated or incomplete information.

AEO becomes manageable when it is treated like the rest of the platform: structured inputs, explicit access rules, observable outputs, and regular verification.

Keeping a Next.js Marketing Site Fast Without Slowing Down Marketing

Israel Vásquez — Mon, 31 Aug 2026 16:19:04 +0000

A fast marketing site is relatively easy to build when very little is happening on it.

The harder version has large hero media, analytics, advertising tools, CRM forms, a headless CMS, content previews, and a marketing team that needs to publish without waiting for an engineer.

That was part of the challenge when Monogram built Kasada's new website.

Kasada is a cybersecurity company, so the platform also needed to communicate technical credibility. But performance could not come at the cost of the workflows required to actually run the website.

The interesting engineering problem became: how do you protect frontend performance without turning performance into a restriction for marketing?

Performance is an architectural problem

It is easy to treat web performance as an optimization phase.

Compress some images. Lazy load a few resources. Run Lighthouse. Fix the obvious problems.

That approach becomes fragile on a marketing platform because the website keeps changing.

New campaigns appear. Analytics tools are added. Advertising scripts change. Editors replace media. Landing pages are assembled differently. Forms appear in new places.

Performance therefore cannot depend entirely on engineers manually reviewing every page.

For Kasada, Monogram built the site with the Next.js App Router and created caching and revalidation strategies around its content heavy architecture. Published pages could benefit from static generation and optimized asset delivery, while editors still had draft and live preview workflows inside Sanity.

The important distinction is that editorial flexibility and production rendering do not have to follow the same execution model.

An editor needs current draft content.

A visitor usually needs the latest published version, delivered as efficiently as possible.

Treating those as separate requirements gives the application more room to optimize both.

Rich media needs guardrails too

Scripts are not the only source of performance problems on marketing sites.

Hero sections are particularly dangerous because they combine two competing goals. They are usually the most visually important content on the page, but they can also contain some of its heaviest assets.

Kasada's implementation included specific performance work around hero media rather than treating media optimization as a generic concern.

That suggests a useful design system principle:

Performance constraints should live as close as possible to the components capable of creating performance problems.

If every editor can independently decide how a large hero asset is loaded, performance becomes dependent on individual publishing decisions.

If the component owns the important delivery behavior, editors can change content without also needing to become performance engineers.

This is one of the less visible advantages of connecting a CMS to a controlled component system.

The CMS provides flexibility over content.

The component defines the technical boundaries around how that content reaches the browser.

Third party scripts are part of your performance budget

Marketing platforms rarely control every resource they execute.

Analytics and advertising tools introduce JavaScript from systems outside the application's core bundle. Those tools may be necessary for attribution, campaign measurement, or other business requirements, so simply removing them is often not a realistic engineering recommendation.

Kasada had this exact constraint.

Monogram implemented audit aware loading for production analytics and advertising integrations, specifically to prevent those tools from unnecessarily degrading performance measurements and Core Web Vitals.

The broader lesson is more important than the specific implementation.

A third party script should not become invisible to the architecture simply because your team does not own its code.

It still consumes browser resources.

It can still compete with rendering work.

It can still affect Core Web Vitals.

And it can still change independently of your application.

That means third party integrations deserve the same architectural questions as other dependencies:

What does this script need to do? When does it need to execute? Which pages actually require it? What happens to the user experience when it becomes expensive?

Performance engineering on a marketing website is partly dependency management.

Marketing capabilities should reuse infrastructure

Protecting performance does not mean preventing marketing from adding functionality.

The better approach is to make common capabilities reusable.

Kasada's HubSpot integration demonstrates this on the lead generation side. Forms were incorporated into the reusable content section library instead of being implemented independently across landing pages and campaigns. Form validation and visual behavior could therefore remain aligned with the rest of the platform while submissions flowed into Kasada's CRM.

This is useful beyond forms.

If a capability will appear repeatedly across a website, implementing it as shared infrastructure creates a place where engineering constraints can be enforced once.

That might include rendering behavior, validation, accessibility, script loading, or integration logic.

The alternative is allowing each page to solve the same problem independently.

That gives teams flexibility quickly, but it also creates multiple places where performance and maintainability can deteriorate.

Optimize the system, not just the launch

The most important performance decision in a large marketing project often happens before anyone opens a performance profiler.

It is deciding who controls what.

On Kasada's website, Sanity gives marketers control over structured content and page composition. The component system gives engineering control over rendering behavior and design consistency. Next.js provides the production rendering, caching, and revalidation layer. External marketing systems remain integrated without being allowed to define the architecture of the application.

That division of responsibility matters because the website is not frozen after launch.

Editors will publish.

Campaigns will change.

New integrations will appear.

Media will be replaced.

If performance depends on everything remaining exactly as it was on launch day, the architecture has already failed.

A more durable approach is to build performance expectations into the places where future changes will happen.

For a CMS driven Next.js site, that means thinking beyond Lighthouse scores and asking:

Which content can be delivered statically?
What actually needs revalidation?
How will editors preview unpublished changes?
Which components can introduce expensive media?
Which external scripts can affect rendering?
Which recurring marketing capabilities should become shared infrastructure?

Those questions are less exciting than choosing a new framework.

They are also much more likely to determine whether the site is still fast six months after launch.

That was the useful lesson from Kasada: performance works better as a constraint of the platform than as a cleanup task for individual pages.