<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Emily Hilton</title>
    <description>The latest articles on DEV Community by Emily Hilton (@emily_hilton_46dff14ebb4c).</description>
    <link>https://dev.to/emily_hilton_46dff14ebb4c</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4181263%2Fe8e09e6c-0d29-48e5-a485-5b115d6cf268.png</url>
      <title>DEV Community: Emily Hilton</title>
      <link>https://dev.to/emily_hilton_46dff14ebb4c</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/emily_hilton_46dff14ebb4c"/>
    <language>en</language>
    <item>
      <title>A RAG Index Cutover Drill for Client-Facing AI Systems</title>
      <dc:creator>Emily Hilton</dc:creator>
      <pubDate>Tue, 17 Nov 2026 09:49:05 +0000</pubDate>
      <link>https://dev.to/emily_hilton_46dff14ebb4c/a-rag-index-cutover-drill-for-client-facing-ai-systems-4b0p</link>
      <guid>https://dev.to/emily_hilton_46dff14ebb4c/a-rag-index-cutover-drill-for-client-facing-ai-systems-4b0p</guid>
      <description>&lt;p&gt;A retrieval index update is a production change, even when the model and prompt stay fixed. A new chunker, embedding model, document set, or access filter can change what the assistant sees. In a client workflow, that may change answers, citations, and who can read a source. This drill makes an index cutover a testable release rather than a silent re-embed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Define the two index versions
&lt;/h2&gt;

&lt;p&gt;Keep the current index readable while building the candidate. Give each index an immutable identifier and record the source snapshot, ingestion time, parser, chunking settings, embedding model, metadata schema, and access-control mapping. A label like “latest” is not enough for a rollback.&lt;/p&gt;

&lt;p&gt;A small release manifest might contain:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;release&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;client-policy-2026-11-17&lt;/span&gt;
&lt;span class="na"&gt;active_index&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;policy-v17&lt;/span&gt;
&lt;span class="na"&gt;candidate_index&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;policy-v18&lt;/span&gt;
&lt;span class="na"&gt;source_snapshot&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;approved-docs-2026-11-16&lt;/span&gt;
&lt;span class="na"&gt;chunker&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;semantic-v3&lt;/span&gt;
&lt;span class="na"&gt;embedding_model&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;embed-v2&lt;/span&gt;
&lt;span class="na"&gt;acl_mapping&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;tenant-and-role-v5&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Build the candidate without changing live traffic. Check document counts, failed ingestions, duplicate document IDs, missing metadata, and records removed since the previous snapshot. If source material contains customer data, use approved test fixtures and preserve the same access rules in the test environment.&lt;/p&gt;

&lt;h2&gt;
  
  
  Run a paired canary set
&lt;/h2&gt;

&lt;p&gt;Choose cases from the actual workflow: a common question, a rare exception, an amended policy, a revoked document, a tenant-specific document, a request with no approved answer, and a question whose answer spans two documents. Run each case against both index versions with the model, prompt, tools, and evaluation settings pinned. Save the retrieved document IDs and ranks as well as the final answer.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Case&lt;/th&gt;
&lt;th&gt;Candidate should show&lt;/th&gt;
&lt;th&gt;Stop condition&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Current policy&lt;/td&gt;
&lt;td&gt;New approved source&lt;/td&gt;
&lt;td&gt;Old superseded source is cited&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Revoked document&lt;/td&gt;
&lt;td&gt;No revoked chunk&lt;/td&gt;
&lt;td&gt;Revoked text remains retrievable&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Tenant boundary&lt;/td&gt;
&lt;td&gt;Only authorized chunks&lt;/td&gt;
&lt;td&gt;Cross-tenant result appears&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Multi-document answer&lt;/td&gt;
&lt;td&gt;All needed sources&lt;/td&gt;
&lt;td&gt;A necessary condition disappears&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Unanswerable request&lt;/td&gt;
&lt;td&gt;Abstention or clarification&lt;/td&gt;
&lt;td&gt;Unsupported answer appears&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Rare exception&lt;/td&gt;
&lt;td&gt;Exception clause retained&lt;/td&gt;
&lt;td&gt;Generic rule overrides it&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Compare the &lt;em&gt;business outcome&lt;/em&gt; and source support, not only text similarity. A fluent answer can still be wrong if it cites the wrong version. Investigate any regression before routing live requests to the candidate.&lt;/p&gt;

&lt;h2&gt;
  
  
  Cut over with an exit path
&lt;/h2&gt;

&lt;p&gt;Start with an internal canary or a small authorized traffic slice. Tag every request with the index version so traces and user reports can be separated. Watch unsupported-answer reports, retrieval misses, citation quality, authorization failures, latency, and the number of cases sent to a human. Set the thresholds and decision owner before the canary starts.&lt;/p&gt;

&lt;p&gt;If a stop condition occurs, restore the prior index alias and verify the whole journey: retrieval, answer, citation, and tool routing. Keep the candidate index for diagnosis, but prevent it from serving user traffic. Reconcile any cases processed during the canary; changing the alias does not undo a bad answer already delivered.&lt;/p&gt;

&lt;h2&gt;
  
  
  Retain the release evidence
&lt;/h2&gt;

&lt;p&gt;Store the manifest, source approvals, paired case results, canary window, observed metrics, decision, and rollback test. This lets a later engineer explain why the index changed and reproduce the result. It also makes the next cutover faster: the same canary set can be rerun with a new source snapshot and explicit changed expectations.&lt;/p&gt;

&lt;p&gt;For engineers preparing to own this kind of client deployment work, &lt;a href="https://www.gsdcouncil.org/certified-forward-deployed-engineer" rel="noopener noreferrer"&gt;Forward Deployed Engineer Certification&lt;/a&gt; provides role context alongside practical delivery skills.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>rag</category>
      <category>testing</category>
    </item>
    <item>
      <title>Forward Deployed Engineering: From Client Discovery to a Handover-Ready AI Prototype</title>
      <dc:creator>Emily Hilton</dc:creator>
      <pubDate>Tue, 13 Oct 2026 06:52:47 +0000</pubDate>
      <link>https://dev.to/emily_hilton_46dff14ebb4c/forward-deployed-engineering-from-client-discovery-to-a-handover-ready-ai-prototype-15ne</link>
      <guid>https://dev.to/emily_hilton_46dff14ebb4c/forward-deployed-engineering-from-client-discovery-to-a-handover-ready-ai-prototype-15ne</guid>
      <description>&lt;p&gt;A forward deployed engineer works close to the environment where software must actually run. The job is to turn a client's ambiguous need into a small, testable system, connect it to the client's data and workflows, and leave the team able to operate it. A model demo is only one part of that work.&lt;/p&gt;

&lt;p&gt;This article is a proposed workflow for an AI-enabled document question-answering project. The examples are illustrative, not a report of a production deployment.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Define the workflow before choosing the model
&lt;/h2&gt;

&lt;p&gt;Start with a conversation about the decision the client needs to make. For a support team, the question might be: “Can an agent find the approved escalation path in the current handbook?” That is more useful than “Build a chatbot.” Record the intended users, the documents they may access, the acceptable response time, and the point where a human must take over.&lt;/p&gt;

&lt;p&gt;Write an acceptance check in plain language: given an approved handbook question, the service returns a cited answer; given insufficient evidence, it abstains; given an unauthorized document request, it denies access before retrieval. These three paths give the client and engineering team a common definition of working software.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Map the data and access boundaries
&lt;/h2&gt;

&lt;p&gt;Inventory the documents, their owners, update cadence, and access rules. Decide how a new version enters the index and how an old version is removed. Keep a document identifier and version with each retrieved passage so a reviewer can trace an answer back to its source.&lt;/p&gt;

&lt;p&gt;A client-supplied document scope is a request, not an authorization decision. Check the caller's permissions on the server before retrieval. If the prototype has no access-control integration, use non-sensitive test documents and say clearly that the feature is incomplete.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Make the API contract inspectable
&lt;/h2&gt;

&lt;p&gt;A narrow request can carry the question, requested scope, and a trace ID:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"question"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"What is the escalation path?"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"document_scope"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"support-handbook-v3"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"trace_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"demo-0042"&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The response should make the outcome and evidence visible:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"status"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"answered"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"answer"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Escalate to the on-call support lead."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"sources"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[{&lt;/span&gt;&lt;span class="nl"&gt;"document_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"support-handbook-v3"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"section"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"2.4"&lt;/span&gt;&lt;span class="p"&gt;}],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"trace_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"demo-0042"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"model_version"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"prototype-1"&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A citation helps a reviewer find the passage. It does not prove the answer is correct. If retrieval does not supply enough evidence, return a defined abstention state rather than inventing a source.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Test failure paths with the client
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Case&lt;/th&gt;
&lt;th&gt;Expected behavior&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Empty question&lt;/td&gt;
&lt;td&gt;Return a validation error&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Unauthorized document scope&lt;/td&gt;
&lt;td&gt;Deny before retrieval&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;No relevant passage&lt;/td&gt;
&lt;td&gt;Abstain without a fabricated citation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Retrieval service unavailable&lt;/td&gt;
&lt;td&gt;Return a service error and trace ID&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Changed source document&lt;/td&gt;
&lt;td&gt;Record the new version and rerun evaluation&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;For each case, save the input, expected result, actual result, and the reason the check matters. Keep evaluation examples separate from prompt-tuning examples. Review the output with the people who will use it; their corrections often reveal missing workflow rules that an isolated benchmark misses.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Leave an operating path
&lt;/h2&gt;

&lt;p&gt;The handover note should identify who owns the document index, how updates are deployed, which logs reconstruct a failed request, the cost and rate limits, and the rollback option. Another engineer should be able to reproduce one valid answer, one abstention, and one authorization failure without the original builder present.&lt;/p&gt;

&lt;p&gt;That is a useful standard for forward deployed engineering: the solution should fit the client's workflow, expose its limits, and survive handover. For readers exploring a structured learning path, the &lt;a href="https://www.gsdcouncil.org/certified-forward-deployed-engineer" rel="noopener noreferrer"&gt;Certified Forward Deployed Engineer syllabus&lt;/a&gt; lists the certification topic and official details.&lt;/p&gt;

</description>
      <category>api</category>
      <category>architecture</category>
    </item>
  </channel>
</rss>
