<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Margaret Kashuba</title>
    <description>The latest articles on DEV Community by Margaret Kashuba (@margaret_kashuba_e3f418ce).</description>
    <link>https://dev.to/margaret_kashuba_e3f418ce</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3956921%2F9197529f-b91f-4662-8224-55e15eeb3f66.jpeg</url>
      <title>DEV Community: Margaret Kashuba</title>
      <link>https://dev.to/margaret_kashuba_e3f418ce</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/margaret_kashuba_e3f418ce"/>
    <language>en</language>
    <item>
      <title>HubSpot + OpenAI integration patterns: webhooks, properties, and the failure modes nobody tells you about</title>
      <dc:creator>Margaret Kashuba</dc:creator>
      <pubDate>Tue, 02 Jun 2026 16:20:43 +0000</pubDate>
      <link>https://dev.to/margaret_kashuba_e3f418ce/hubspot-openai-integration-patterns-webhooks-properties-and-the-failure-modes-nobody-tells-you-3373</link>
      <guid>https://dev.to/margaret_kashuba_e3f418ce/hubspot-openai-integration-patterns-webhooks-properties-and-the-failure-modes-nobody-tells-you-3373</guid>
      <description>&lt;p&gt;HubSpot's native AI features are a starter kit. They are fine for "summarize this email" and "draft a follow-up". The moment you want production AI behaviour inside a HubSpot workflow — multi-step reasoning, custom retrieval, write-back to structured properties, with proper retries and audit logging — you are off the marketing brochure and into engineering territory. This post is what I wish I had read before we shipped our first HubSpot + OpenAI integration for a B2B SaaS RevOps team.&lt;/p&gt;

&lt;p&gt;If you want the broader business case, &lt;a href="https://altheracode.com" rel="noopener noreferrer"&gt;AltheraCode&lt;/a&gt; (the studio I work with) has a published case study on this. I'm going to skip the business case here. This is engineer-to-engineer.&lt;/p&gt;

&lt;h2&gt;
  
  
  The four surfaces where AI plugs into a HubSpot stack
&lt;/h2&gt;

&lt;p&gt;You have exactly four real integration surfaces. Everything you read about HubSpot AI patterns reduces to one of these.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Workflow custom code actions.&lt;/strong&gt; Node.js, runs serverless inside HubSpot, 20-second hard timeout, 100 MB memory cap. The most common path. Good for synchronous enrichment that fits in 20 seconds.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Webhooks out of HubSpot.&lt;/strong&gt; Workflow → POST to your service → your service does whatever → writes back via HubSpot CRM API. The right choice when you need more than 20 seconds, more than 100 MB, or anything that needs queueing.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The CRM API directly, polled or event-driven via app subscriptions.&lt;/strong&gt; You operate outside HubSpot entirely and treat HubSpot as a database with REST endpoints. Best for high-volume bulk operations.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Conversations API and timeline events.&lt;/strong&gt; Underused. Lets you write AI-generated content into the contact or deal timeline without touching structured properties. Excellent for "summaries" that should be auditable but not searchable.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Pick the surface that fits the SLA you need, not the one that feels familiar.&lt;/p&gt;

&lt;h2&gt;
  
  
  Pattern 1: lead-intent enrichment as a workflow custom code action
&lt;/h2&gt;

&lt;p&gt;Most common pattern. Contact enters a workflow on some engagement threshold; you call OpenAI; you write a summary property back. Looks like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nx"&gt;exports&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;main&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;async &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;event&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;callback&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;hubspot&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;require&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;@hubspot/api-client&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;OpenAI&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;require&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;openai&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;client&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nx"&gt;hubspot&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Client&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;accessToken&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;process&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;env&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;HS_PRIVATE_APP_TOKEN&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;openai&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;OpenAI&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;apiKey&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;process&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;env&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;OPENAI_API_KEY&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;

  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;contactId&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;event&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;object&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;objectId&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;contact&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;crm&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;contacts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;basicApi&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getById&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="nx"&gt;contactId&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;email&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;company&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;jobtitle&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;linkedin_url&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;last_seen_url&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
  &lt;span class="p"&gt;);&lt;/span&gt;

  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;prompt&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;`You write a single 4-sentence pre-call summary for an account executive.
Rules:
- Do NOT describe the company in marketing language.
- Reference exactly one specific, dated fact from the company's public footprint.
- Note the prospect's likely role in a buying committee.
- End with one concrete question the AE should ask first.

Inputs:
&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;JSON&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;stringify&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;contact&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;properties&lt;/span&gt;&lt;span class="p"&gt;)}&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;completion&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;openai&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;chat&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;completions&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;create&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
    &lt;span class="na"&gt;model&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;gpt-4o&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;messages&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[{&lt;/span&gt; &lt;span class="na"&gt;role&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;user&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;content&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;prompt&lt;/span&gt; &lt;span class="p"&gt;}],&lt;/span&gt;
    &lt;span class="na"&gt;temperature&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;0.2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;max_tokens&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;200&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="p"&gt;});&lt;/span&gt;

  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;summary&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;completion&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;choices&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="nx"&gt;message&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;content&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

  &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;crm&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;contacts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;basicApi&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;update&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;contactId&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;properties&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;ai_pre_call_summary&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;summary&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;});&lt;/span&gt;

  &lt;span class="nf"&gt;callback&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;outputFields&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;summary&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Three notes that cost me time.&lt;/p&gt;

&lt;p&gt;You &lt;strong&gt;must&lt;/strong&gt; add the &lt;code&gt;@hubspot/api-client&lt;/code&gt; and &lt;code&gt;openai&lt;/code&gt; packages explicitly in the custom code action's package list. The HubSpot UI does this for you only if you discover the right dropdown. I missed it for two hours.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;event.object.objectId&lt;/code&gt; is reliable. &lt;code&gt;event.inputFields&lt;/code&gt; is not — workflow inputs that look populated in the UI sometimes arrive as &lt;code&gt;undefined&lt;/code&gt;, especially after a property rename. Always re-fetch from the CRM API, do not trust the inputs.&lt;/p&gt;

&lt;p&gt;Use &lt;code&gt;temperature: 0.2&lt;/code&gt; or lower for any structured writeback. At 0.7 you will get cute summaries with one outlier sentence per week that breaks downstream parsing.&lt;/p&gt;

&lt;h2&gt;
  
  
  Pattern 2: AI-summarized deal notes with property writeback
&lt;/h2&gt;

&lt;p&gt;A call lands in Gong (or Fathom, or Granola — same shape). You want the prospect's stated objection, timeline, and decision criteria written into structured properties on the HubSpot deal.&lt;/p&gt;

&lt;p&gt;The naive version writes a free-text summary into a single Note property. The production version uses three separate custom properties and a function-calling response so you can actually run pipeline analytics on them later.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;tool&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;function&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;function&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;record_deal_signals&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;description&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Record structured signals from a sales call.&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;parameters&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;object&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="na"&gt;properties&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="na"&gt;stated_objection&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;string&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
        &lt;span class="na"&gt;stated_timeline&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
          &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;string&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
          &lt;span class="na"&gt;enum&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;no_timeline&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;0_30_days&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;30_90_days&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;90_plus_days&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
        &lt;span class="p"&gt;},&lt;/span&gt;
        &lt;span class="na"&gt;decision_criteria&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
          &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;array&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
          &lt;span class="na"&gt;items&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;string&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
      &lt;span class="p"&gt;},&lt;/span&gt;
      &lt;span class="na"&gt;required&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;stated_timeline&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;decision_criteria&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;completion&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;openai&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;chat&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;completions&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;create&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
  &lt;span class="na"&gt;model&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;gpt-4o&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;messages&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;role&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;system&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;content&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;You extract structured signals from sales call transcripts. Do not paraphrase. Quote.&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;role&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;user&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;content&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;transcript&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;],&lt;/span&gt;
  &lt;span class="na"&gt;tools&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;tool&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
  &lt;span class="na"&gt;tool_choice&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;function&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;function&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;record_deal_signals&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="na"&gt;temperature&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;0.1&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;signals&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;JSON&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;parse&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;completion&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;choices&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="nx"&gt;message&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;tool_calls&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="kd"&gt;function&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;arguments&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;crm&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;deals&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;basicApi&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;update&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;dealId&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;properties&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;stated_objection&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;signals&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;stated_objection&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="dl"&gt;''&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;stated_timeline&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;signals&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;stated_timeline&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;decision_criteria&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;signals&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;decision_criteria&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="p"&gt;[]).&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt; | &lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Two production lessons here.&lt;/p&gt;

&lt;p&gt;Force &lt;code&gt;tool_choice&lt;/code&gt; to a specific function rather than &lt;code&gt;auto&lt;/code&gt;. Auto will occasionally return a chat message with no tool call when the model decides the input does not warrant one. You can not tolerate that flakiness inside a workflow.&lt;/p&gt;

&lt;p&gt;For the &lt;code&gt;decision_criteria&lt;/code&gt; array, store as a pipe-delimited string in a single HubSpot property rather than trying to model a multi-select. HubSpot multi-select properties are limited and brittle for free-text values.&lt;/p&gt;

&lt;h2&gt;
  
  
  Pattern 3: timeline events instead of properties
&lt;/h2&gt;

&lt;p&gt;People reach for properties because they are the obvious unit. For AI-generated content that is interesting context but not data you'll filter pipeline reports on, write to the timeline instead.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;crm&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;timeline&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;eventsApi&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;create&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
  &lt;span class="na"&gt;eventTemplateId&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;YOUR_TEMPLATE_ID&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;email&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;contact&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;email&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;tokens&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;summary&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;aiGeneratedNote&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;source&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;gong_call_2026_05_28&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;confidence&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;0.92&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Timeline events are searchable, sortable by time, and don't pollute your property list. We use timeline events for "research summaries", "previous-call recap on next-call open", and "agent-flagged risk signals". Anything that's "context for a human" goes here. Anything you'll run a report on goes into a property.&lt;/p&gt;

&lt;h2&gt;
  
  
  Failure modes, by frequency
&lt;/h2&gt;

&lt;p&gt;In rough order of how often each one bit me:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Workflow custom code 20-second timeout.&lt;/strong&gt; If your AI call plus enrichment plus writeback can not finish in 20 seconds, you must move to pattern 2 (webhook out). I have seen people split a call across two custom code actions to "fit". Do not do this. The state management between them is a nightmare. Bite the bullet and externalise.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;HubSpot rate limits.&lt;/strong&gt; 100 requests per 10 seconds per portal for the v3 API. If your AI agent is enriching new contacts in bulk, you will hit this. Implement a leaky-bucket queue in front of your writes.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Property rename detection.&lt;/strong&gt; A marketing ops person renames &lt;code&gt;Lead_Score_v2&lt;/code&gt; to &lt;code&gt;Lead Score (V2)&lt;/code&gt;. Half your code breaks. Build a nightly job that fetches property metadata and diff-checks against your code's expected schema.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Custom code action package versions.&lt;/strong&gt; HubSpot pins package versions on the runtime side. &lt;code&gt;openai@4.x&lt;/code&gt; works; &lt;code&gt;openai@5.x&lt;/code&gt; does not at the time of writing. Pin explicitly.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Idempotency on retries.&lt;/strong&gt; Workflows retry on failure. Your AI action must be idempotent — same input, same output property update — or you will have duplicate timeline events and AE confusion. Use a request-ID derived from the contact ID and a hash of the prompt inputs.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Notes that turn out to be portal-visible.&lt;/strong&gt; HubSpot has a "private note" UI affordance and a "shared portal note" property — they are not the same thing. Read the docs carefully, and default to draft-mode for any AI writeback that touches a customer-shared object.&lt;/p&gt;

&lt;h2&gt;
  
  
  Logging and observability
&lt;/h2&gt;

&lt;p&gt;The unfun part. Production AI inside HubSpot needs three things you will not have on day one:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Every model call logged to a side store (we use a simple Cloudflare D1 instance with &lt;code&gt;request_id, contact_id, model, prompt_hash, response, latency_ms, cost_cents&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;A &lt;code&gt;prompt_version&lt;/code&gt; property written alongside every AI-generated value so you can A/B prompts in production without losing history.&lt;/li&gt;
&lt;li&gt;A weekly cost-and-quality report. Cost is easy. Quality is a small spreadsheet where the sales ops lead grades 20 sampled outputs each week. You do not skip this.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Closing
&lt;/h2&gt;

&lt;p&gt;Real HubSpot AI engineering is more workflow-mechanics than model-engineering. The model is the easy part. The retries, the property hygiene, the 20-second budget, the rate limits, the portal-visibility surprises — that is where the work lives.&lt;/p&gt;

&lt;p&gt;If you ship a HubSpot AI integration this quarter, the thing that will save you a sprint is reading the workflow custom code action docs cover to cover before you write your first prompt.&lt;/p&gt;

&lt;p&gt;I'd be curious how others are handling property-rename detection. We brute-force it with a daily diff. There must be a smarter way.&lt;/p&gt;

</description>
      <category>hubspot</category>
      <category>ai</category>
      <category>openai</category>
      <category>javascript</category>
    </item>
    <item>
      <title>Slack AI assistant for engineering: RAG, pgvector, and the parts that broke</title>
      <dc:creator>Margaret Kashuba</dc:creator>
      <pubDate>Fri, 29 May 2026 13:49:06 +0000</pubDate>
      <link>https://dev.to/margaret_kashuba_e3f418ce/slack-ai-assistant-for-engineering-rag-pgvector-and-the-parts-that-broke-55l9</link>
      <guid>https://dev.to/margaret_kashuba_e3f418ce/slack-ai-assistant-for-engineering-rag-pgvector-and-the-parts-that-broke-55l9</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;TL;DR — Built a Slack-native AI assistant that answers engineering questions from Confluence, GitHub wikis, Notion, and PDFs. RAG with pgvector + hybrid retrieval + permissions enforced at retrieval (not at the LLM). Two mistakes cost us about two weeks. Here is the field report so you don't repeat them.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Sometime around the third sprint of this project I stopped believing that "internal AI assistant" was a real product category and started believing it was an interface problem dressed up as an AI problem. I want to write down why, because most of the posts I see about AI assistants focus on the wrong layer.&lt;/p&gt;

&lt;p&gt;The team I worked with — a small AI engineering studio called &lt;a href="https://altheracode.com" rel="noopener noreferrer"&gt;AltheraCode&lt;/a&gt; — builds software for the construction-engineering world. Not the sexy bit — not BIM, not generative design — the boring middle. Specifications. Standards. Internal rules nobody reads until they have to. The kind of knowledge that lives in three different Confluence spaces, four shared drives, and the head of a senior engineer who's been there nine years and is one bad week away from going on sabbatical.&lt;/p&gt;

&lt;p&gt;The brief was: build something so that when a junior engineer has a question on a Thursday afternoon, they don't have to DM that senior engineer and feel guilty about it.&lt;/p&gt;

&lt;p&gt;Sounds simple. It isn't.&lt;/p&gt;

&lt;h2&gt;
  
  
  The thing nobody tells you about engineering knowledge
&lt;/h2&gt;

&lt;p&gt;Wikis don't fail because the search is bad. They fail because asking is easier than searching, and humans pick the easier path every single time. Once a team has more than maybe 25 engineers, the dominant pattern for "how do I do X" is to ask in a channel, not to type into a search bar. The whole internal-search-engine industry has been quietly losing this fight for fifteen years.&lt;/p&gt;

&lt;p&gt;So when you ship an AI assistant, you are not competing with the wiki. You are competing with &lt;code&gt;#eng-help&lt;/code&gt;. The assistant has to live where engineers already are, answer faster than a human can type a reply, and be wrong less often than the wiki is stale. That last bar is lower than you'd hope and a lot harder than it sounds.&lt;/p&gt;

&lt;p&gt;We picked Slack as the surface and stuck with it. There was an early conversation about also building a web UI "for the cases where Slack feels wrong" — we killed that idea in week two and I'm glad we did. Two interfaces would have meant half the attention on either of them.&lt;/p&gt;

&lt;h2&gt;
  
  
  What we actually built
&lt;/h2&gt;

&lt;p&gt;In rough strokes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Slack Bolt&lt;/strong&gt; bot running in the client's AWS account, surfaced as a slash command, an &lt;code&gt;@mention&lt;/code&gt; in any channel, and a DM.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Ingestion pipeline&lt;/strong&gt; pulling from Confluence, GitHub wiki, repo READMEs and &lt;code&gt;/docs&lt;/code&gt;, Notion, and a curated set of S3-hosted PDFs (technical specs and internal standards).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Embedding store&lt;/strong&gt; — &lt;code&gt;pgvector&lt;/code&gt; inside the existing PostgreSQL deployment.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Retriever layer&lt;/strong&gt; with hybrid search: dense embeddings + BM25 keyword scoring + a re-ranker + per-document permission filter keyed on the user's Slack identity.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;LLM layer&lt;/strong&gt; — OpenAI's GPT-4 generation at the time, strictly grounded on retrieved chunks, with an explicit "I don't know" fallback path.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Citation surface&lt;/strong&gt; — every answer in Slack ships with inline source links and a thumbs-up/down so we can grade quality in production.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Audit log&lt;/strong&gt; — every query, retrieval, answer, and feedback signal in PostgreSQL with a 90-day retention window.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The architecture is not novel. The novelty is in two choices.&lt;/p&gt;

&lt;h3&gt;
  
  
  pgvector instead of a dedicated vector DB
&lt;/h3&gt;

&lt;p&gt;People love to argue about this. Here is the version I will defend: if your corpus is in the hundreds of thousands of chunks and you already run a healthy Postgres, pgvector is fine. Better than fine — it saves you a piece of infrastructure that has a permanent operational tax. Pinecone and friends are wonderful at hundreds of millions of chunks. You probably don't have hundreds of millions of chunks. I'd rather optimise pgvector for another six months than babysit a new managed service.&lt;/p&gt;

&lt;h3&gt;
  
  
  Permissions enforced at retrieval, not in the prompt
&lt;/h3&gt;

&lt;p&gt;This is the one I will die on. You cannot tell an LLM "don't reveal X" and trust it. Models will leak things you asked them not to leak, especially when a clever user asks the right way. The only correct answer is to not put the restricted thing in the context window in the first place.&lt;/p&gt;

&lt;p&gt;We mapped Slack identity → SSO identity → per-document ACL, applied as a row-level filter on &lt;code&gt;pgvector&lt;/code&gt; before the LLM ever sees anything. It looks like this at the SQL level:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;chunk_text&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;document_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ts_rank&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;embedding&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&amp;gt;&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;dist&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;chunks&lt;/span&gt; &lt;span class="k"&gt;c&lt;/span&gt;
&lt;span class="k"&gt;JOIN&lt;/span&gt; &lt;span class="n"&gt;document_acl&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;document_id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;document_id&lt;/span&gt;
&lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;principal_id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;ANY&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nb"&gt;text&lt;/span&gt;&lt;span class="p"&gt;[])&lt;/span&gt;           &lt;span class="c1"&gt;-- user's SSO groups&lt;/span&gt;
&lt;span class="k"&gt;ORDER&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;dist&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;ts_rank&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;
&lt;span class="k"&gt;LIMIT&lt;/span&gt; &lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It's more work upfront and it pays back forever.&lt;/p&gt;

&lt;h2&gt;
  
  
  The bits we got wrong
&lt;/h2&gt;

&lt;p&gt;There were two real mistakes. I'll spare you the small ones.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Ingestion pipeline was undersized.&lt;/strong&gt; The first version re-embedded the entire corpus every night. That was fine at ten thousand chunks. It fell over around two hundred thousand. We rewrote the pipeline to hash content and only re-embed pages that actually changed, plus carved out a separate fast-path for new pages arriving between nightly runs. That cost us about ten days. We should have caught it in design review.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Hallucination guardrail was too permissive.&lt;/strong&gt; Our first prompt told the model to "answer based on the retrieved documents and acknowledge when you don't have enough information." Reasonable on paper. In practice the model was acknowledging far less than it should. Engineers were getting confident-sounding wrong answers and they noticed fast. Trust in the bot started slipping in the second week.&lt;/p&gt;

&lt;p&gt;We rewrote the system prompt to require an explicit citation per factual claim, and to refuse rather than guess when retrieval returned nothing meaningful. Roughly:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;You answer engineering questions using ONLY the provided context chunks.
Rules:
1. Every factual claim must cite a chunk by ID, in the form [doc-XX].
2. If the retrieved chunks do not contain a confident answer, say:
   "I don't have a confident answer for that — try #eng-help."
3. Never invent function names, endpoints, JIRA tickets, or process steps.
4. If the question is ambiguous, ask one clarifying question instead of guessing.

Context chunks:
{retrieved_chunks}

Question:
{user_question}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Refusal rate went up. Trust came back. The lesson here is that "I don't know" is a feature, and you should treat it like one.&lt;/p&gt;

&lt;p&gt;If I were starting again I'd skip those two mistakes and reclaim about two weeks of calendar.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it does on a normal Tuesday
&lt;/h2&gt;

&lt;p&gt;The assistant answers questions like:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;"What's our naming convention for new microservices?"&lt;/li&gt;
&lt;li&gt;"Who owns the billing pipeline on-call rotation?"&lt;/li&gt;
&lt;li&gt;"What's the difference between the v3 and v4 telemetry schemas?"&lt;/li&gt;
&lt;li&gt;"How do we handle multi-tenant tenancy isolation in the report generator?"&lt;/li&gt;
&lt;li&gt;"Where do I file a vendor-onboarding request and who approves it?"&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These are answers that used to take five minutes of a senior engineer's time, or two days of a junior engineer's polite-but-frustrated asking around. The assistant answers most of them in under three seconds, with sources cited.&lt;/p&gt;

&lt;p&gt;It doesn't generate code. It doesn't editorialise. It doesn't try to give opinions about how the architecture should work. It retrieves and rephrases, with sources cited. That boundary is what makes it trustworthy. Every time we have been tempted to widen the scope — "what if it could draft the PR description too" — we have reminded ourselves that the moment it starts inventing things, it stops being a knowledge surface and becomes a thing engineers have to double-check, which is the opposite of what we set out to build.&lt;/p&gt;

&lt;h2&gt;
  
  
  A few things I'd take to the next project
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;The interface is the product.&lt;/strong&gt; A great RAG pipeline behind a mediocre Slack experience is a mediocre product. Invest disproportionately in the conversation design.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Refuse by default.&lt;/strong&gt; "I'm not confident" beats a confident wrong answer every single time. Engineers learn fast which tools they can trust, and they don't come back when they get burned.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Permissions at retrieval, not in the prompt.&lt;/strong&gt; Saying it twice because it's the single most common mistake I see in AI assistant projects, and the failure mode ends up in a postmortem.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Hybrid retrieval, not pure semantic.&lt;/strong&gt; Engineering questions are full of exact identifiers — function names, error codes, internal acronyms. Pure embedding search misses these. Add BM25 and a re-ranker on day one.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Ship the boring parts first.&lt;/strong&gt; Audit logging, feedback collection, citation rendering — all the stuff that doesn't feel like "the AI part" — is what makes the system survive contact with a real team. Skip it and you'll be retrofitting it under deadline pressure six months later.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Why I'm writing this
&lt;/h2&gt;

&lt;p&gt;Internal AI assistants are at an awkward stage. Every B2B tech company is building one. Most of them are quietly underperforming. Almost nobody writes honestly about why. The result is a lot of LinkedIn posts about "transforming knowledge work with AI" and not enough postmortems about the part where retrieval permissions almost leaked a sensitive document.&lt;/p&gt;

&lt;p&gt;If you're scoping a similar project, the most useful thing I can offer is the list of mistakes above. Skip them. Spend the saved weeks on the parts of your own domain that I don't know about.&lt;/p&gt;

&lt;p&gt;I'd love to read your version of this post when you ship.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>rag</category>
      <category>slack</category>
      <category>softwaredevelopment</category>
    </item>
  </channel>
</rss>
