DEV Community: Tessa Kriesel

How to Set Up Signup Attribution in PostHog Behind a Cookie Banner

Tessa Kriesel — Mon, 12 Oct 2026 18:39:17 +0000

When someone signs up for your product, you want to know what brought them in. PostHog can tell you. It takes some extra setup when a cookie banner sits in front of your website and signup happens in a separate app. Without it, nearly every signup shows up as direct.

This tutorial follows one visitor from their first click to their account and fixes each place where their source gets lost.

The path one visitor takes

Someone clicks a link to your website in a Reddit post.
They land on your site. A cookie banner asks if analytics is okay.
They read a couple of pages, then accept.
They click Sign up. That takes them off your website and into your app, which often lives at a different address.
They create an account.

You want the account at the end of that path to say "Reddit." The source drops off at two points. Both happen because analytics waits for the visitor's consent.

At the banner. On a site with a cookie banner, PostHog is usually set to stay off until the visitor accepts. The landing from Reddit happens before that, so it isn't recorded.
When they move to the app. The app is often a separate site, and the visitor hasn't accepted cookies there yet. So it starts them as a new visitor with no history.

Steps 1 and 2 fix those two points. Step 3 gets the source into your CRM. Step 4 checks that it all works.

What you need

PostHog's JavaScript library on the website and the app, using the same project
A cookie banner that controls whether analytics runs
The website and the app on the same parent domain, with the app on a subdomain of it

Step 1: Remember how the visitor arrived

The visitor lands from Reddit before they've accepted cookies, so PostHog isn't running yet. By the time they accept, the landing is over and there's no record of where it came from.

Start PostHog on the website so it captures and stores nothing until the visitor accepts.

const accepted = visitorHasAccepted()

posthog.init(POSTHOG_KEY, {
  api_host: POSTHOG_HOST,
  opt_out_capturing_by_default: !accepted, // capture nothing until the visitor accepts
  persistence: accepted ? 'localStorage+cookie' : 'memory', // store nothing until the visitor accepts
  person_profiles: 'always', // keep a profile for visitors before they sign up
  capture_pageview: false, // you send the pageview yourself, below
})

visitorHasAccepted() stands for however your cookie banner remembers the visitor's choice.

person_profiles: 'always' matters here. The first source is stored on the visitor's profile, so the visitor needs a profile before they have an account.

Then save the landing yourself. When the first page loads, put the page's address and the referrer into the browser tab's session storage. The address carries any campaign tags (the utm_ parameters) and ad click IDs. Nothing leaves the browser.

const ENTRY_KEY = 'entry_context'

function stashEntryContext() {
  try {
    if (sessionStorage.getItem(ENTRY_KEY)) return // only the first page of the visit
    sessionStorage.setItem(ENTRY_KEY, JSON.stringify({
      url: location.href,
      referrer: document.referrer,
    }))
  } catch {}
}

stashEntryContext() // run this on every page load

When the visitor accepts cookies, turn PostHog on and send one pageview that carries what you saved.

// The campaign tags and ad click IDs PostHog records as first touch, as of posthog-js 1.434.
const CAMPAIGN_PARAMS = [
  'utm_source', 'utm_medium', 'utm_campaign', 'utm_content', 'utm_term',
  'gad_source', 'mc_cid', 'gclid', 'gclsrc', 'dclid', 'gbraid', 'wbraid',
  'fbclid', 'msclkid', 'twclid', 'li_fat_id', 'igshid', 'ttclid', 'rdt_cid',
  'epik', 'qclid', 'sccid', 'irclid', '_kx',
]

// Returns the saved landing once per visit. After that it returns null.
function takeEntryContext() {
  try {
    const saved = sessionStorage.getItem(ENTRY_KEY)
    if (!saved || sessionStorage.getItem(ENTRY_KEY + '_sent')) return null
    sessionStorage.setItem(ENTRY_KEY + '_sent', '1')
    return JSON.parse(saved)
  } catch {
    return null
  }
}

function hostOf(url) {
  try {
    return new URL(url).host
  } catch {
    return '$direct'
  }
}

function sendPageview() {
  const entry = takeEntryContext()
  if (!entry) {
    posthog.capture('$pageview')
    return
  }

  // The landing page, in the fields PostHog uses.
  const url = new URL(entry.url)
  const landing = {
    $current_url: entry.url,
    $host: url.host,
    $pathname: url.pathname,
    $referrer: entry.referrer || '$direct',
    $referring_domain: entry.referrer ? hostOf(entry.referrer) : '$direct',
  }
  for (const param of CAMPAIGN_PARAMS) {
    landing[param] = url.searchParams.get(param) || null
  }

  // PostHog stores first touch as the same fields with an $initial_ prefix.
  const firstTouch = { ...landing }
  for (const [key, value] of Object.entries(landing)) {
    firstTouch['$initial_' + key.replace('$', '')] = value
  }

  // The pageview itself reports the landing's referrer and campaign tags.
  const props = { $referrer: landing.$referrer, $referring_domain: landing.$referring_domain }
  for (const param of CAMPAIGN_PARAMS) {
    if (landing[param]) props[param] = landing[param]
  }

  posthog.capture('$pageview', props, { $set_once: firstTouch })
}

Then connect it to your cookie banner. Turn PostHog on and send the pageview when the visitor accepts, and turn it back off if they withdraw.

// When the visitor clicks Accept on your banner:
function onAccept() {
  posthog.set_config({ persistence: 'localStorage+cookie' })
  posthog.opt_in_capturing({ captureEventName: false })
  sendPageview()
}

// When the visitor rejects or withdraws consent:
function onReject() {
  posthog.opt_out_capturing()
  posthog.set_config({ persistence: 'memory' })
}

// On every page load, for visitors who accepted earlier:
if (accepted) sendPageview()

The saved landing goes in $set_once, as the third argument to capture. $set_once is PostHog's way of saying "set this only if it isn't set already." It's what keeps Reddit as the visitor's first source, whatever they click later. PostHog fills in first-touch values from the current page on its own, and values passed this way take their place.

Every campaign field gets a value, including null for tags the landing didn't have. That way a later page can't fill them in.

captureEventName: false skips the separate opt-in event, so this pageview is the first event PostHog receives for the visitor.

Step 2: Make the website and the app agree on who the visitor is

PostHog gives every visitor an anonymous ID. Because the app starts fresh, the visitor gets one ID on the website and a different one in the app. That looks like two people, and the one who signs up has no Reddit visit behind them.

The fix is to make the app use the ID the website already handed out.

On the website. There's nothing to add. Once the visitor accepts cookies, PostHog stores their ID in a cookie on your parent domain, where the app can read it. That setting is cross_subdomain_cookie, and it's on by default.

In the app. Start PostHog in memory only, so it doesn't read or write cookies until the visitor accepts. That means it won't pick up the website's ID on its own.

So read the cookie yourself and pass the ID in when PostHog starts, using the bootstrap option.

// Find the ID the website stored in the shared cookie.
function sharedDistinctId(posthogKey) {
  const prefix = `ph_${posthogKey}_posthog=`
  const cookie = document.cookie.split('; ').find((c) => c.startsWith(prefix))
  if (!cookie) return null
  try {
    return JSON.parse(decodeURIComponent(cookie.slice(prefix.length))).distinct_id || null
  } catch {
    return null
  }
}

const accepted = visitorHasAccepted()

// Once the visitor has accepted, PostHog reads the cookie itself.
const sharedId = accepted ? null : sharedDistinctId(POSTHOG_KEY)

// Start PostHog with that ID, if there is one.
posthog.init(POSTHOG_KEY, {
  api_host: POSTHOG_HOST,
  opt_out_capturing_by_default: !accepted,
  persistence: accepted ? 'localStorage+cookie' : 'memory',
  ...(sharedId ? { bootstrap: { distinctID: sharedId, isIdentifiedID: false } } : {}),
})

The app and the website now use the same ID for the visitor.

The last part happens in the app. Opt the visitor in when they accept cookies there, and tell PostHog who they are once they're signed in.

// When the visitor accepts cookies in the app. Run it then, not on every page:
posthog.set_config({ persistence: 'localStorage+cookie' })
posthog.opt_in_capturing({ captureEventName: false })

// When the visitor rejects or withdraws consent:
posthog.opt_out_capturing()
posthog.set_config({ persistence: 'memory' })

// Once they're signed in:
posthog.identify(userId)

PostHog attaches what the anonymous visitor did, including the landing from Reddit, to that user. isIdentifiedID: false in the code above is what allows that. It tells PostHog the shared ID doesn't belong to a known user yet.

Step 3: Put the source on the account

At this point PostHog knows the user came from Reddit. I wanted that on the account in my CRM, next to everything else I know about them.

A script asks PostHog for the first source of whoever created each account. This is my query. My accounts are organizations, so the first part finds the person who created each one. If your accounts are single users, you can skip that part and read the same properties off the person who signed up.

WITH creators AS (
  SELECT $group_0 AS org_key, argMax(person_id, timestamp) AS creator
  FROM events
  WHERE event = 'organization_created'
  GROUP BY org_key
)
SELECT c.org_key,
       p.properties.$initial_utm_source,
       p.properties.$initial_utm_medium,
       p.properties.$initial_utm_campaign,
       p.properties.$initial_referring_domain
FROM creators c
LEFT JOIN persons p ON p.id = c.creator
LIMIT 100000

Keep the LIMIT. Without one, the query API returns the first 100 rows.

The script then writes four fields to the account:

Channel: where they came from, like Reddit
Type: paid ad, organic, referral, or direct, worked out from the utm_medium tag and the referrer
Confidence: high when a campaign tag or referrer backs it up, lower when it's a guess
Basis: the raw values from PostHog, so I can check any account by hand

Step 4: Check that it works

Events showing up in PostHog doesn't mean the pieces are connected. I use three checks.

Walk the path yourself. Open a link to your site with campaign tags on it. Click to a second page and accept cookies there. Then click Sign up and create an account. Your new account should show that campaign.

Look at pageviews per person. If the number sits near 1, that often means every page load is being counted as a new person and the ID isn't being kept.

Count how many signups have a source. A signup has a source when the person has a campaign tag, or a referrer from somewhere other than your own site. Swap in your signup event and your domain.

SELECT
  uniqExact(person_id) AS signups,
  uniqExactIf(person_id,
    coalesce(person.properties.$initial_utm_source, '') != ''
    OR (coalesce(person.properties.$initial_referring_domain, '$direct') NOT IN ('$direct', '')
        AND person.properties.$initial_referring_domain NOT LIKE '%yoursite.com')
  ) AS signups_with_a_source
FROM events
WHERE event = 'user_signed_up'
  AND timestamp >= now() - INTERVAL 90 DAY

PostHog also gives each person a channel type on its own, like Direct, Referral, or Organic Search. If you count signups by that and get a much higher number than this query, check how many are labeled Unknown. PostHog uses Unknown for people it never saw on your website. They have no source, so this query leaves them out.

Where this leaves you

For every visitor who accepts cookies, the account they create shows the page they landed on and what sent them there. Visitors who never accept stay without a source. That's by design.

Before you trust the numbers, run the first check in step 4 once on your live site.

Redact PII before your model searches the web

Tessa Kriesel — Mon, 12 Oct 2026 14:00:00 +0000

Support chats, intake forms, benefits questions, patient messages. The messages people send your app are full of personal details: names, addresses, account numbers, IDs. And many of those questions need a current answer from the web, like a program’s income limits, a bank’s wire rules, or a state’s renewal process.

The answer depends on what they’re asking, not on who’s asking. This tutorial builds a loop that takes personal details out before the question leaves your machine: Rampart’s open model redacts personal details on your machine, Tabstack runs the search-to-answer loop on the redacted question, your own model in Ollama writes the reply, and the details go back in on your side.

Rampart, from National Design Studio

Rampart is an open PII redaction library from National Design Studio. It runs in the browser, in the moment between typing a message and sending it, and replaces personal details with placeholders like [GIVEN_NAME_1] before anything reaches a server. The mapping back to real values never leaves the device.

At its core is a 14.7 MB MiniLM model that recognizes names, phone numbers, bank account and routing numbers, tax and government IDs, passports, driver’s licenses, and every part of a street address. Rules cover card numbers, SSNs, emails, URLs, and IP addresses. Together they work across seven Latin-script languages and report 98.42% private-term recall on Rampart’s held-out test set. National Design Studio is clear about what it is for: harm reduction for people entering their own information.

Two choices make it a natural fit for web research. It keeps city, state, and ZIP by default, because location is what most “what applies to me” questions turn on. And the model is published openly on Hugging Face, so it runs from Python on your server as well as in the browser. That is how we use it here.

What Rampart’s model catches

Here is what the code in this tutorial sends onward for a few kinds of questions. Everything in brackets stays on the machine.

Intake	What leaves your machine
Benefits	My name is [GIVEN_NAME_1] [SURNAME_1], I live at [BUILDING_NUMBER_1] [STREET_NAME_1], Brookvale, CT 06482, and I make $1,950 a month. What housing assistance in Connecticut am I eligible for?
Banking	Hi, this is [GIVEN_NAME_1] [SURNAME_1] from Acme Corp, account [BANK_ACCOUNT_1], routing [ROUTING_NUMBER_1]. Why was my wire held?
Travel documents	Passport [PASSPORT_1], driver’s license [DRIVERS_LICENSE_1] issued in California. Can I renew my passport by mail from Fresno?
Health	I’m [GIVEN_NAME_1] [SURNAME_1], DOB 03/14/1988, member ID [GOVERNMENT_ID_1], diagnosed with type 1 diabetes. Does Anthem cover insulin pumps in Ohio?
Spanish	Mi nombre es [GIVEN_NAME_1] [SURNAME_1], vivo en [STREET_NAME_1] [BUILDING_NUMBER_1], Madrid. ¿Qué ayudas hay para alquiler?

The names and IDs are gone, and the state, program, amount, and topic are still there, so each question can still be answered. The health row also shows what the model leaves alone, like a date of birth, which we cover in tuning.

What you’ll build

View the diagram in the original post.

Four short files:

redact.py finds PII with Rampart’s model and swaps it for placeholders.
research.py sends the redacted question to Tabstack and gets back a cited report.
reply.py has your model write the answer from that report.
ask.py runs the loop and restores the details.

Set up

You need Python 3.9 or later, Ollama with a model pulled, and a Tabstack API key.

pip install onnxruntime tokenizers huggingface_hub numpy tabstack ollama
ollama pull llama3.1
export TABSTACK_API_KEY=...

The first run downloads Rampart’s model files from Hugging Face and caches them.

Step 1: Redact PII with Rampart’s model

The model labels each token: B-GIVEN_NAME starts a name, I-STREET_NAME continues a street, O is ordinary text. We join those labels into spans, skip the ones in KEEP, and run a short list of rules for the classes Rampart handles with rules instead of the model: card numbers, SSNs, IP addresses, emails, and URLs. Long messages run in overlapping 512-token windows, the model’s limit.

# redact.py
import json
import re
from typing import Callable, Optional

import numpy as np
import onnxruntime as ort # type: ignore[import-untyped]
from huggingface_hub import hf_hub_download
from tokenizers import Tokenizer # type: ignore[import-untyped]

REPO = "nationaldesignstudio/rampart"
KEEP = {"CITY", "STATE", "ZIP_CODE"} # the answer needs these; Rampart keeps them too
MIN_SCORE = 0.4 # start a span
EXTEND_SCORE = 0.15 # continue a span across punctuation, as Rampart does

tokenizer = Tokenizer.from_file(hf_hub_download(REPO, "tokenizer.json"))
tokenizer.enable_truncation(max_length=512, stride=64) # long messages run in overlapping windows
with open(hf_hub_download(REPO, "config.json")) as f:
    id2label = json.load(f)["id2label"]
label2id = {label: int(i) for i, label in id2label.items()}
model = ort.InferenceSession(hf_hub_download(REPO, "onnx/model_q4.onnx"))

GAP = re.compile(r"[\s\-.()/]*")

def luhn_ok(number: str) -> bool:
    digits = [int(d) for d in re.sub(r"\D", "", number)][::-1]
    total = sum(d if i % 2 == 0 else (d * 2 - 9 if d > 4 else d * 2) for i, d in enumerate(digits))
    return total % 10 == 0

# Rampart catches these with rules rather than the model, so we do too.
# Each rule is (pattern, label, optional check). Append your own for IDs the model doesn't know.
Rule = tuple[re.Pattern[str], str, Optional[Callable[[str], bool]]]
RULES: list[Rule] = [
    (re.compile(r"\b\d{3}-\d{2}-\d{4}\b"), "SSN", None),
    (re.compile(r"\b(?:\d{4}[-]?){3}\d{4}\b|\b\d{4}[-]?\d{6}[-]?\d{5}\b"), "CARD_NUMBER", luhn_ok),
    (re.compile(r"\b(?:\d{1,3}\.){3}\d{1,3}\b"), "IP_ADDRESS", None),
    (re.compile(r"\b[\w.+-]+@[\w-]+(?:\.[\w-]+)+\b"), "EMAIL", None),
    (re.compile(r"\bhttps?://[^\s<>\"')\]]*[^\s<>\"').,;:!?\]]"), "URL", None),
]

def find_pii(text: str) -> list[tuple[int, int, str]]:
    spans: list[tuple[int, int, str]] = []
    first = tokenizer.encode(text)
    for enc in [first, *first.overflowing]:
        feeds = {
            "input_ids": np.array([enc.ids], dtype=np.int64),
            "attention_mask": np.array([enc.attention_mask], dtype=np.int64),
            "token_type_ids": np.array([enc.type_ids], dtype=np.int64),
        }
        logits = model.run(None, feeds)[0][0]
        scores = np.exp(logits) / np.exp(logits).sum(axis=-1, keepdims=True)

        window: list[tuple[int, int, str]] = []
        for (start, end), row in zip(enc.offsets, scores):
            if start == end:
                continue
            label = id2label[str(int(row.argmax()))]
            kind = label[2:]
            last = window[-1] if window and GAP.fullmatch(text[window[-1][1]:start]) else None
            # Join word pieces and multi-part values ("Cedar Lane", "203-555-0142") into one span.
            if last and (kind == last[2] or (label == "O" and row[label2id["I-" + last[2]]] >= EXTEND_SCORE)):
                window[-1] = (last[0], end, last[2])
            elif label != "O" and row.max() >= MIN_SCORE:
                window.append((start, end, kind))
        spans += window

    rules = [
        (m.start(), m.end(), label)
        for pattern, label, check in RULES
        for m in pattern.finditer(text)
        if check is None or check(m.group())
    ]

    # Rule matches win over the model, longer spans win over shorter ones, and none overlap.
    chosen: list[tuple[int, int, str]] = []
    for span in sorted(rules, key=lambda s: s[0] - s[1]) + sorted(spans, key=lambda s: s[0] - s[1]):
        if span[2] not in KEEP and all(span[1] <= c[0] or span[0] >= c[1] for c in chosen):
            chosen.append(span)
    return sorted(chosen)

class Redactor:
    """Swaps PII for numbered placeholders and keeps the mapping in memory."""

    def __init__ (self) -> None:
        self.values: dict[str, str] = {} # placeholder -> original value

    def redact(self, text: str) -> str:
        out, last = [], 0
        for start, end, kind in find_pii(text):
            value = text[start:end]
            placeholder = next((p for p, v in self.values.items() if v == value), None)
            if placeholder is None:
                count = sum(p.startswith(f"[{kind}_") for p in self.values) + 1
                placeholder = f"[{kind}_{count}]"
                self.values[placeholder] = value
            out += [text[last:start], placeholder]
            last = end
        return "".join(out + [text[last:]])

    def restore(self, text: str) -> str:
        for placeholder, value in self.values.items():
            text = text.replace(placeholder, value)
        return text

The placeholder table lives in memory in your process. The same value always gets the same placeholder, so a name that comes up again in a follow-up message keeps its placeholder.

Step 2: Research the redacted question with Tabstack

/research plans searches, reads sources, and returns a Markdown report with the pages it cited, all inside one call. Your model never fetches a page or reads raw HTML, and Tabstack only receives the redacted question.

# research.py
from tabstack import Tabstack

client = Tabstack() # reads TABSTACK_API_KEY

def research(question: str) -> tuple[str, list[dict[str, str]]]:
    stream = client.agent.research(query=question, mode="fast")
    for event in stream:
        if event.event == "complete":
            pages = event.data.metadata.cited_pages or []
            sources = [{"title": page.title or "(untitled)", "url": page.url} for page in pages]
            return event.data.report, sources
        if event.event == "error":
            raise RuntimeError(f"Research failed: {event.data.error.message}")
    raise RuntimeError("Research stream ended without a result")

On the redacted benefits question, a fast call came back in about 17 seconds with a report on Connecticut income limits, the Section 8 Housing Choice Voucher Program, and the state Rental Assistance Program, citing five sources including the Connecticut Department of Housing. Use mode="balanced" when a question needs more sources.

Tabstack is built by Mozilla. Every request is covered by two commitments on our Data Handling page: never trained on, and private by default.

Step 3: Write the reply with your model

Your model gets the redacted question and the finished report. Tell it to keep placeholders exactly as written, so the details can go back in.

# reply.py
import ollama

MODEL = "llama3.1"

SYSTEM_PROMPT = (
    "Answer the user's question using only the research report. "
    "Keep every placeholder, such as [GIVEN_NAME_1], exactly as written."
)

def write_reply(question: str, report: str) -> str:
    response = ollama.chat(
        model=MODEL,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": f"Question:\n{question}\n\nResearch report:\n{report}"},
        ],
    )
    return response.message.content or ""

Any model you run in Ollama works here. Swap MODEL for Qwen, Mistral, or whatever you already use.

Step 4: Run the loop

# ask.py
import sys

from redact import Redactor
from reply import write_reply
from research import research

def ask(question: str) -> None:
    redactor = Redactor() # one per conversation; the mapping never leaves this process
    safe_question = redactor.redact(question)
    print(f"Sent: {safe_question}\n")

    report, sources = research(safe_question)
    reply = write_reply(safe_question, report)

    print(redactor.restore(reply))
    print("\nSources:")
    for source in sources:
        print(f"- {source['title']}: {source['url']}")

if __name__ == " __main__":
    if len(sys.argv) < 2:
        sys.exit('Usage: python ask.py "your question"')
    try:
        ask(" ".join(sys.argv[1:]))
    except Exception as err:
        sys.exit(f"Could not answer: {err}")

Run it with any question from the table above, or one of your own:

python ask.py "Hi, this is Tom Becker from Acme Corp, account 0042981173, routing 021000021. Why was my wire held?"

The first line shows exactly what left your machine. The answer that follows has the real details back in it, with the sources it came from.

Tune it for your data

Three adjustments cover most intakes.

Decide what stays. KEEP is the list of labels that pass through untouched. City, state, and ZIP are the default because local rules depend on them. A question about a federal program may not need the ZIP; a question about a city service needs the city. Set KEEP per use case, not once for the whole app.

Send structured fields by choice. When the intake is a form, you already know which field holds the name. There is no need to guess with a model: send the fields the answer needs, run the model only over the free text, and never send the rest.

# intake.py
from redact import Redactor

SEND = {"state", "zip_code", "household_size", "monthly_income"} # fields the answer needs
FREE_TEXT = ("question", "notes") # fields people type into

def build_question(form: dict[str, str], redactor: Redactor) -> str:
    lines = [f"{field.replace('_', ' ')}: {value}" for field, value in form.items() if field in SEND]
    lines += [redactor.redact(form[field]) for field in FREE_TEXT if field in form]
    return "\n".join(lines)

Research a form the same way as a question:

from intake import build_question
from redact import Redactor
from reply import write_reply
from research import research

form = {
    "full_name": "Maria Garcia",
    "email": "maria@example.com",
    "state": "CT",
    "zip_code": "06482",
    "household_size": "2",
    "monthly_income": "$1,950",
    "question": "My landlord Dave Kim raised the rent 30%. What help is there?",
}

redactor = Redactor()
question = build_question(form, redactor)
report, sources = research(question)
print(redactor.restore(write_reply(question, report)))

The question that goes to Tabstack is:

state: CT
zip code: 06482
household size: 2
monthly income: $1,950
My landlord [GIVEN_NAME_1] [SURNAME_1] raised the rent 30%. What help is there?

The name and email fields are never sent at all, and the one name inside the free text is redacted.

Add rules for your own identifiers. Case numbers, patient IDs, policy numbers, and dates of birth follow formats you know. Append them to RULES:

import re

from redact import RULES

RULES.append((re.compile(r"(?i)(?<=dob )\d{1,2}/\d{1,2}/\d{4}"), "DATE_OF_BIRTH", None))
RULES.append((re.compile(r"\bCASE-\d{6}\b"), "CASE_NUMBER", None))

With those added, “I’m Priya Nair, DOB 03/14/1988, case CASE-204871. Does Ohio Medicaid cover insulin pumps?” goes out as “I’m [GIVEN_NAME_1] [SURNAME_1], DOB [DATE_OF_BIRTH_1], case [CASE_NUMBER_1]. Does Ohio Medicaid cover insulin pumps?”

Send only the question. Redaction removes identifiers, but the rest of the message still goes to Tabstack: greetings, background, details the answer doesn’t need. Your model is already running locally, so have it rewrite the redacted message as one standalone research question first.

# focus.py
import ollama

from reply import MODEL

FOCUS_PROMPT = (
    "Rewrite the user's message as one standalone research question. "
    "Keep only the facts needed to answer it, such as location, program, amounts, dates, or conditions. "
    "Drop greetings, names, placeholders like [GIVEN_NAME_1], and any detail the answer does not depend on. "
    "Reply with the question only."
)

def focus(message: str) -> str:
    response = ollama.chat(
        model=MODEL,
        messages=[
            {"role": "system", "content": FOCUS_PROMPT},
            {"role": "user", "content": message},
        ],
    )
    return (response.message.content or message).strip()

In ask.py, import focus, research and print the focused question, and keep the full redacted message for the reply, so your model can still answer the person directly:

from focus import focus # add to the imports at the top of ask.py

safe_question = redactor.redact(question)
research_question = focus(safe_question)
print(f"Sent: {research_question}\n")

report, sources = research(research_question)
reply = write_reply(safe_question, report)

With the date-of-birth and case rules from above, one run sent Priya’s message to Tabstack as just “Does Ohio Medicaid cover insulin pumps?” and turned the redacted housing question into “What housing assistance programs in Connecticut are available to individuals with a monthly income of $1,950?” The wording changes from run to run.

Redact first and focus second. The rewrite is a model following instructions, so it can miss. In testing it often kept placeholders like [BANK_ACCOUNT_1], which is harmless because a placeholder contains no personal data. It also sometimes added details the person didn’t give, like “for individuals with diabetes”, so check the focused question in the Sent: line.

Know what Rampart leaves alone

Rampart redacts identifiers and leaves the rest of the message alone, which is usually what the answer needs: a diagnosis for a coverage question, an income for an eligibility question, an employer for a benefits question. Dates of birth, ages, usernames, and your own internal IDs also pass through until you add a rule.

Names in non-Latin scripts are out of scope for this release of Rampart, and so are indirect identifiers: details that only point to someone in combination. National Design Studio describes Rampart as harm reduction for people entering their own information, and that is the right way to use this loop too. This tutorial is a minimal Python port of Rampart’s approach, so Rampart’s published evaluation numbers apply to the library itself, not to this code.

Where to take it next

Redact in the browser. For a web app, use Rampart’s own JavaScript library so questions are redacted before they reach your server at all: npm install @nationaldesignstudio/rampart @huggingface/transformers.
Redact replies before you log them. Run the model’s output through Redactor().redact() before it goes into logs or analytics.
Keep one redactor per conversation. Placeholders stay consistent across turns, so follow-up questions about the same person keep working.

Start with your own question: get a Tabstack API key and make a research call.

Dev Tool GTM, Done Agentically

Tessa Kriesel — Thu, 08 Oct 2026 15:57:13 +0000

I'm Founding Technical GTM at Tabstack. I run GTM with a stack of agents and tools, and I stay in the loop for the calls that need a human.

This post is the whole system: the parts of dev tool GTM I run, how I do each one today, and what comes out of it.

One rule holds it together. Positioning, ICP, messaging, voice, and vocabulary are never hard-coded anywhere in the stack. Every system reads them from Quiver context at the moment it works. Change the context in Quiver once, and Tabstack OS, Hermes, and every skill follow on their next run.

The systems

Quiver : the system of record. It holds the product marketing context (positioning, ICP, messaging, competitive landscape, customer language, brand voice, words to use and avoid), plus research, campaigns, the content workflow and calendar, and every report the agents produce. It's versioned. Agents propose changes and I approve them.
Tabstack OS: where I work. A Claude Code workspace that loads Quiver context every session, drafts and reviews content and docs, queries analytics and the CRM, and runs the other systems.
Hermes: my always-on agent. It runs scheduled jobs for competitive intel, community listening, prospecting, sponsorships, social drafts, and event briefs. It reads Quiver context before every job and saves every report to Quiver. It drafts and never posts, publishes, emails, or contacts anyone.
Rival: my open-source competitive intel app. I built it on Tabstack to dogfood and learn the product.
Magister: weekly AI visibility tracking and AEO audits.
MentionDrop : social listening on the forums and social sites our own tools can't reach reliably.
PostHog and GA4 via Supermetrics: product and web analytics: signups, API usage, activation, and retention.
Airtable CRM: data-collection-approved signups have an account, with lifecycle stage, usage, acquisition source, and a PostHog event timeline.
Granola and Linear: call transcripts, and where product feedback lands for engineering.

At a glance

Marketing effort	How we do it today	Result
Customer and market research	Every signal, call, and thread lands in Quiver research, which extracts themes, sentiment, and quotes, scores it against our hypotheses, and proposes context updates.	Customer language and hypothesis evidence in one place. Approved proposals update what every agent reads.
Positioning	Human-led. I make the call from Quiver research, product data, and customer calls.	One positioning that Tabstack OS, Hermes, and every skill read from Quiver at runtime. Change it once, everything follows.
Messaging	Agents draft from Quiver context. I polish.	One vocabulary across site, docs, and posts, with the banned-word list enforced on every draft.
Competitive intel	Rival, my open-source app built on Tabstack, scans 15 competitors across 11 page types (homepage, pricing, changelog, docs, careers and hiring tech stack, blog, G2 and other reviews, social, GitHub, company profile, tech stack), keeps every diff, and writes intelligence briefs with threat tiers and 7-axis scores. Hermes reads it with Quiver's watchlist every morning, drops the noise, and scores each change against the positioning in Quiver context.	A daily brief of what actually changed and what to do about it. This week: check Parallel's new Task API tiers before writing Research comparison copy.
Community listening	Find the conversations where the ICP in Quiver context hits the problems we solve, so we show up where the pain is and learn their language. MentionDrop covers the sites our own tools can't reach reliably. Hermes turns the daily haul into exact threads with draft replies in my voice.	2,680 mentions a month filtered to a short daily list. I reply the same day; the best threads become research.
Content production workflow	Tabstack OS drafts, runs the editorial principles review, fact-checks against the live API, and renders covers. Quiver runs draft, review, approve, publish, plus the calendar.	13 posts in the 24 days after the pivot, every one technically verified.
Docs and API reference	Tabstack OS tech-writing, api-reviewer and docs-reviewer agents, sdk-sync to track SDK changes.	Docs keep pace with every SDK release.
Social	Hermes drafts LinkedIn and X posts every Thursday from the Quiver calendar.	Native drafts waiting for every published post. I edit and post.
Prospecting and CRM	Hermes prospects every Tuesday into Airtable. crm-import and trace enrich signups. My network: 10k people I've connected with over nearly 20 years in tech in one deduplicated database, which supplies prospects, warm intros, and contact emails.	5 to 10 qualified teams a week, each with a cited signal, deduped. 491 signups enriched. Contacts for outreach in minutes.
Sponsorships and creators	Hermes finds creators and newsletters whose audience matches the ICP in Quiver context.	A ranked sponsorship shortlist every week.
Events and CFPs	Hermes writes a Monday brief, scored attend, speak, or sponsor.	Events 30 days out and CFPs 90 days out, already scored.
Website	Copy and site architecture from Tabstack OS, Astro build, launch audit.	Rebrand live 21 days after the positioning decision was. made.
Analytics and attribution	PostHog and GA4 (via Supermetrics), queried in plain language from Tabstack OS.	Answers in minutes, no dashboard requests.
Activation and retention	PostHog tracks every account from signup to API key to API usage, plus retention at organization level. The Airtable CRM carries each account's milestone timeline, acquisition source, and use case.	This data, with Quiver research, is why we pivoted.
AI visibility (LLM citations)	Magister tracks 15 prompts weekly across Claude, ChatGPT, Gemini, and Google AI Overviews, plus AEO audits.	Visibility score 22.3 to 42.7 in two weeks. Homepage AEO 80 to 95.
Product feedback loop	Fully automated. When any of my agents hits a Tabstack error or DX problem, in any repo, it logs it, and a hook files it as a Linear ticket to engineering. No human in the loop.	Every bit of friction my agents hit becomes an engineering ticket, without anyone stopping to write it up.

.gtm-table-wrap { width: 100%; max-width: 100%; overflow-x: auto; -webkit-overflow-scrolling: touch; margin: 1.5em 0; } .gtm-table-wrap table.gtm-table { display: table !important; width: 100% !important; min-width: 640px; max-width: none; table-layout: fixed; border-collapse: collapse; border-spacing: 0; margin: 0; font-size: 0.85em; line-height: 1.45; white-space: normal !important; background: none !important; } .gtm-table-wrap .gtm-table th, .gtm-table-wrap .gtm-table td { padding: 10px 12px !important; text-align: left; vertical-align: top; border: 0; border-bottom: 1px solid rgba(128, 128, 128, 0.35); white-space: normal !important; overflow-wrap: break-word; background: none !important; } .gtm-table-wrap .gtm-table th { border-bottom-width: 2px; font-weight: 700; } .gtm-table-wrap .gtm-table th:nth-child(1) { width: 20%; } .gtm-table-wrap .gtm-table th:nth-child(2) { width: 46%; } .gtm-table-wrap .gtm-table th:nth-child(3) { width: 34%; } .gtm-table-wrap .gtm-table td:nth-child(1) { font-weight: 600; }

The eight cores of dev tool GTM, and how the stack covers them

1. Know the developer

ICP and jobs to be done from real interviews: Granola captures calls. Each call becomes a Quiver research entry, which is processed automatically for themes, sentiment, quotes, and whether it validates or challenges an active hypothesis, then turned into proposed context updates. I approve them.
Events and in-person community to validate the market: conversations at events and meetups go into Quiver research from my own notes, then get the same automatic processing: themes, quotes, hypothesis signals, and proposed context updates.
Watch developers try the product and log the friction: my agents use Tabstack every day. When one hits an error or DX problem, it logs it and a hook files it as a Linear ticket to engineering. Fully automated.
Track competitors and real alternatives, including build-it-yourself: Rival scans 15 competitors across 11 page types and writes threat-scored intelligence briefs. Hermes reads it with Quiver's watchlist every morning and scores each change against the positioning in Quiver context. Community listening surfaces the DIY builds people post.

2. Positioning and messaging

Category, alternatives, differentiated value: human-led. I make the call from Quiver research, product data, and customer calls. It lives in Quiver context.
Messaging hierarchy: messaging pillars live in Quiver context. Agents draft from them. I polish.
Homepage, docs landing, and pricing copy: Tabstack OS drafts from Quiver context and writes the site architecture. The rebrand went live 21 days after the positioning call.
Everyone saying the same thing: every agent and skill reads Quiver context at runtime and enforces its words-to-avoid list on every draft. Context changes are proposals I approve, so nothing drifts silently.

3. Developer experience

Time to first value: PostHog tracks every account from signup to API key to API usage. The Airtable CRM carries that milestone timeline per account, with usage days and span, so I can see how far each signup got and how long it took.
Docs, examples, SDKs, error messages: Tabstack OS writes docs, READMEs, changelogs, and API reference. Review agents check every claim against SDK source and the live API. sdk-sync tracks upstream SDK, API, and docs repos and updates what changed.
Agent-readiness: llms.txt and llms-full.txt on the site, plus AEO audits in Magister.
Route field feedback into the roadmap: the automated DX-notes-to-Linear loop, plus product signals flagged on Quiver research entries.
Pricing and packaging: competitor pricing changes arrive daily from Hermes. Pricing decisions are fed from research, made by a human.

4. Awareness and demand

Technical content: Tabstack OS drafts, reviews against the 10 editorial principles, fact-checks against the live API, and renders covers. Quiver runs draft, review, approve, publish, and the calendar. 13 posts shipped in the 24 days after the pivot.
Search visibility, including AI answer engines: Magister tracks AI visibility weekly across Claude, ChatGPT, Gemini, and Google AI Overviews, plus AEO audits. Search Console through Supermetrics.
Community presence: MentionDrop listens on the forums and social sites our own tools can't reach reliably, using keywords tuned to the ICP in Quiver context. Hermes turns the daily haul into exact threads with draft replies in my voice. I reply. The best threads become research.
Events, talks, CFPs: Hermes writes a Monday brief of events 30 days out and CFPs 90 days out, scored attend, speak, or sponsor.
Sponsorships: Hermes finds creators and newsletters whose audience matches the ICP in Quiver context.
Integrations, partnerships, listings: tracked as Quiver campaigns. Directory submissions and co-marketing skills in Tabstack OS.
Social: Hermes drafts LinkedIn and X posts every Thursday from the Quiver calendar. I edit and post.

5. Activation and conversion

Instrumentation: PostHog and GA4, queried in plain language from Tabstack OS, which also builds the dashboards.
Finding and fixing the biggest drop-off: the funnel from visit to signup to API key to usage is read in PostHog. Every account carries its acquisition source and use case, so drop-off splits by channel and by audience.
Retention: PostHog tracks retention at organization level. The CRM shows usage days and span per account.
Product-qualified accounts: the Airtable CRM holds every signup with lifecycle stage, usage, acquisition source, newsletter status, and a PostHog event timeline, kept current by crm-import. Accounts that created an API key are enriched through trace and classified by use case. Hermes adds qualified prospects weekly, each with a cited signal, deduplicated.
Network intelligence: my network: 10k people I've connected with over nearly 20 years in tech into one deduplicated database with append-only enrichment (Hunter, GitHub, Gravatar, Tabstack). Searchable in a local UI. It's where prospects, warm intros, and contact emails come from. Hermes keeps it up-to-date.
Lifecycle email: drafted in Tabstack OS and tracked as a Quiver campaign so performance can be logged.

6. Revenue

Sales enablement: one-pagers, objection handling, and demo material drafted in Tabstack OS from Quiver context.
Founder-led sales : Quiver holds the ICP shape. Hermes finds teams that fit it in my network and beyond, and adds them to the CRM with the signal. I reach out.
Customer stories (in progress): drawn from Quiver research, the same calls, quotes, and notes that feed everything else.

7. Measurement and ops

Decisions from the data: our most recent pivot came out of this system. PostHog activation and retention data, plus Quiver research from calls, interviews, and forum threads scored against active hypotheses, is why we repositioned.
Data that keeps getting better: I've improved the data the whole time. Client-side web analytics added in June. Accounts tagged by job (extract, research, automate) from endpoint usage, first-touch source backfilled, and PostHog events synced into the CRM. Analytics identity and signup-source tracking shipped for the pivot. Complete attribution tracking.
Weekly review: every Tabstack OS session opens with a brief pulled from Quiver: what's due, Hermes reports and research waiting on review, what's publishing, what shipped, and pending context proposals.
Experiments with stated hypotheses: active hypotheses live in Quiver context. Every research entry is scored against them.
Metrics: a north-star metric and the funnel behind it in PostHog, plus GA4, Search Console, and Magister AI visibility, all queried from Tabstack OS and synthesized into Quiver as performance logs and the weekly marketing report.
Tooling, budget, hiring: drafted in Tabstack OS. Decisions are human.

8. Internal alignment

One agreed ICP and priority set: Quiver context is the single source, versioned, with every change approved by me. Everyone works together in Quiver.
Report in business terms: exec updates, one-pagers, and decks drafted in Tabstack OS.
Weekly marketing updates : built and sent on Mondays by Hermes.

What stays human

Agents draft. People decide. Positioning calls, design, privacy, publishing, posting, and outreach all stay with a human.

If this stack is speaking your love language, check out Quiver—#3 on Product Hunt on Vercel Day. 🔥

You might also like How I Grew Signups 71% in 90 Days by Running GTM like an Engineer.

When should you use search, agentic search, a Research API, or build the loop yourself?

Tessa Kriesel — Mon, 05 Oct 2026 14:00:00 +0000

Use a search API when your application needs sources. Use agentic search when your model needs to choose what to investigate next. Use a Research API when your application needs a finished answer with citations. Build the loop yourself when control over source selection, evidence handling, or synthesis is part of your product.

The decision starts with the job. If a coding assistant needs a parameter definition from a known documentation page, fetch that page. If it needs to compare current capabilities across several projects, someone must find the sources, read them, check coverage, and reconcile the evidence before writing the answer.

Choose where that work belongs and which parts your team needs to control.

First, define what your application must receive

“Add web access” leaves the output undefined. Write the result your application actually needs:

Required output	Starting point
Relevant URLs and snippets	Search
Clean content from a known URL	Fetch or Markdown extraction
Specific fields from a known page	Structured extraction, followed by value checks
An answer supported by current sources	A research loop, operated by your application, agent, or managed service
A completed website task	Browser automation with explicit action boundaries

These jobs can exist inside the same application. A pasted link does not need source discovery. A request to find current deployment guidance might. Reading a pricing page and comparing several vendors’ pricing are different workflows even when both begin with a URL.

The search-versus-research guide explains the work between finding sources and producing a cited answer. This guide helps you decide who should perform it.

What each approach gives you

A search API returns candidates for your application to use

A search API fits a source picker, a documentation finder, or a system that already knows how to build an answer from retrieved material.

For example, Ollama’s Web Search API returns results with title, url, and a relevant content snippet. Its Web Fetch API returns a page’s main content and links. The two responses support different next steps: a snippet helps identify a source; fetched content gives your application material to inspect.

If your feature displays links, search can finish the job. If it displays a cited explanation, your application still owns source selection, reading, answer construction, and the connection between claims and sources. That can be a small, bounded implementation when one authoritative page contains the answer.

Choose search when finding sources is the required output, or your existing downstream system already meets the answer requirements.

Agentic search lets the model decide what to do next

In model-operated agentic search, the application gives the model search and fetch tools. The model chooses a query, reads the results, and decides whether to open a page, search again, or answer. The application executes those tool calls and enforces their limits.

Ollama’s search-agent example demonstrates this pattern. It adds tool results to the conversation and continues until the model stops requesting tools.

This approach fits a task whose next step depends on what the agent learns. A coding assistant can inspect a local dependency version, retrieve the corresponding documentation, and use it while proposing a change. The application decides which local context the model can use and which information it can send in a public-web query.

You still maintain tool definitions, context handling, loop limits, source mappings, and failure behavior. Treat web content as untrusted input; it must not grant the agent new permissions or override the application’s instructions.

Choose agentic search when the model’s adaptive investigation belongs inside the larger task.

A Research API returns the answer before your model continues

A managed Research API takes a question and performs the answer-building work inside the service. Your application receives the researched output instead of executing each search and fetch decision itself.

Tabstack Research plans queries, discovers and reads sources, checks gaps, follows up when needed, and synthesizes a Markdown report with citation metadata. It returns the report on the stream’s terminal complete event. Your application can give that result to your own model as an input to its next step.

That boundary suits a briefing feature or assistant that needs a current public-web answer but does not need to control every intermediate retrieval decision.

Your application still owns acceptance. Check whether the answer covers the question and whether its decision-driving claims have support. Handle task-level error events, and treat a stream that ends without a terminal event as incomplete. Research always streams; receiving progress is not the same as receiving the finished report.

Choose a Research API when you need the answer as an input and the service’s exposed controls satisfy the job.

A DIY loop gives your team control over the implementation

A custom loop lets you choose the retrieval components, source-selection rules, context limits, reconciliation method, synthesis prompts, and citation construction. You can combine deterministic steps with model decisions.

Build when those choices create product value or satisfy a requirement a managed service cannot meet. Examples include a prescribed source hierarchy, retrieval across private and public systems, or a requirement to preserve selected evidence passages in your own format.

Ownership includes maintenance. Your team handles changed pages, failed fetches, provider limits, model changes, incomplete evidence, and malformed output. It also maintains the evaluation that determines whether the loop still works.

Choose DIY when you can name the intermediate decisions you must control and assign the engineering capacity to operate them.

Agentic search and DIY overlap. “Agentic” describes how the system chooses its next action; “DIY” describes what your team builds and owns. A managed Research service can also run an agentic loop internally. Inspect the response contract and control boundary rather than choosing from the label alone.

The decision matrix

Approach	What your application receives	Who chooses follow-up work?	What your team maintains	Prefer another path when…
Search API	Candidate sources, commonly URLs and snippets	Your application or model	Retrieval integration and any downstream answer-building	You need a finished answer but have no dependable downstream loop
Model-operated agentic search	The configured agent’s answer or task result	Your model, within application limits	Tools, prompts, context, orchestration, and acceptance checks	The web question is a separable subtask and the model does not need intermediate control
Managed Research API	The service’s documented report and citation format	The service, within its exposed controls	Integration, output acceptance, and provider fit	Required sources, controls, or processing boundaries fall outside the service’s contract
DIY research loop	The output contract you implement	Your code, models, or both	Components, orchestration, evidence handling, evaluation, and operations	The custom loop adds no necessary control and your team cannot justify maintaining it

The architecture does not establish which implementation answers better, runs faster, or costs less. Compare the complete configurations on your questions. Include failed attempts and incomplete outputs rather than pricing only the successful demo.

Where does the work run?

Use this process model before selecting a provider:

View the diagram in the original post.

If a required processing boundary rules out a provider, eliminate it before testing answer quality. A good report cannot make an incompatible data path acceptable.

Apply the guide to a concrete job

Suppose a developer assistant receives this question:

What does Ollama’s Web Search API return, how does that differ from Web Fetch, and can I display the search response as a cited answer?

Before running anything, define the required answer elements:

Name the documented fields in each response.
Distinguish a relevant snippet from fetched page content.
Explain the application work required to turn those responses into an answer.
Cite the documentation for the API facts.

Here is a complete, documentation-derived answer example, not output from a new API run:

Ollama’s Web Search API returns a results array whose entries contain title, url, and a relevant content snippet. Web Fetch returns title, the page’s main content, and links. Neither response defines a finished cited-answer format. Your application can display search results directly, but it must construct the explanation and attach supporting sources if the feature promises a cited answer. See the Web Search and Web Fetch documentation.

For this job, start by reading the known documentation page. It contains the response definitions. If your application can write the explanation and preserve its source, it does not need an open-ended research loop.

Change the job and the choice can change:

Find official web-search documentation. Use search when you do not already know the destination.
Inspect this repository’s dependency version and work out the relevant integration steps. Use an agentic path when the model needs to move between local inspection and public documentation. Keep private context out of public queries unless your policy explicitly permits it.
Compare current retrieval approaches for a developer assistant using evidence across several projects. Evaluate a Research API if your application needs the comparison report before its model continues. Freeze the comparison criteria first.
Use only approved sources and preserve the exact passages selected at every stage. Build or select a loop that exposes those controls. Do not infer that a report and citation list satisfy an evidence-archive requirement.

The question determines the workflow. A product does not need one architecture for every web request.

When Tabstack is not the answer

We built Tabstack to return finished web output to applications. These requirements point elsewhere:

Your requirement	Better starting point	Why
You only need ranked links and snippets	A search API	Research performs answer-building work the feature does not need
Your existing assistant or search loop already meets the job	Keep the existing path	A new provider needs a concrete reason to enter the architecture
Your product must control intermediate source selection or synthesis beyond the API’s controls	A custom loop or service exposing those controls	The required behavior belongs in the implementation contract
Your workload must stay inside an air-gapped or exclusively controlled environment	An architecture compatible with that boundary	Hosted Tabstack Research processes requests outside your environment
Your workflow needs private sources the public-web service cannot access	An authorized private-data retrieval path, optionally combined with public research	Public-web research cannot supply inaccessible evidence
You need a hosted browser task behind a login	An authenticated automation system you control	Hosted Tabstack Automate handles public websites and has no credential store or session management

A known URL is a separate case: it can fit Tabstack, but usually through Extract, not Research. Ask for Markdown when the next step needs page content, or matching JSON when it needs fields. Validate the values as well as the shape.

Pilo provides a self-hosted automation path. It is not a self-hosted deployment of Tabstack Research.

Check the processing boundary

Running your own model does not make external web services local. Search queries and research requests follow the data paths of the providers your application calls.

Tabstack Research runs on hosted infrastructure. Contracted model providers process the query and web content to produce the answer. Our Data Handling documentation describes that path, and the Privacy Notice is the canonical policy. Confirm workload-specific retention and provider requirements before sending sensitive inputs. Do not interpret “never trained on” as a promise that no hosted processing or storage occurs.

Choose a path for one real job

Write down the output your feature promises, the intermediate decisions you must control, and the data it can send outside your environment. Then test the smallest approach that satisfies those requirements.

Use the citation review method to check claim support and answer coverage. The traced Python example shows how to preserve the report, cited pages, and request timeline. For a comparison between complete systems, use the search + fetch + model evaluation plan.

If your job needs a finished answer from public sources, bring that question to the Research playground, which requires sign-in. Write its required answer elements first, then check the returned report against them. If search or a known-page fetch already meets the job, use it.

How to Make Claude Code Stop Writing Output Essays

Tessa Kriesel — Fri, 31 Jul 2026 18:18:00 +0000

I can't stand reading long output.

Not "I mildly prefer brevity." I mean I hit a wall of text in my terminal and my brain refuses. I asked Claude to fix a nav dropdown, it fixed it, and then it wrote me eleven lines summarizing the fix I had just watched it make. Tool calls are visible. I saw the diff. I don't need the recap.

Sound familiar?

Telling it to be concise doesn't work. Not reliably. I had a memory saved that said "keep replies SHORT, she reacts strongly to long output," and it still handed me a status inventory with four headers.

So I went looking for what actually holds. Four options, ranked from weakest to strongest.

CLAUDE.md

The obvious one. Put your rule in ./CLAUDE.md for a project, or ~/.claude/CLAUDE.md for every project on the machine.

This is where most people stop, and it's the weakest option. Not because it doesn't load, it does, every session. It's that your rule sits in a file competing with project context, tool definitions, and a conversation that grows all afternoon. By hour two it's noise.

One thing that genuinely helps: put the rule at the very top of the file, not buried at the bottom. Position matters more than I expected.

Use this if you want something scoped, visible, and easy to change. Don't expect it to hold under a long session.

Skills

Skip this one. I went down this path first and it's a dead end for this problem.

Skills load when they're invoked. That's the whole design. A skill that says "write concisely" only helps if the model remembers to invoke it, which is the exact failure you're trying to fix. You'd be relying on the thing that's already unreliable.

Skills are great. They're just the wrong tool for always-on behavior.

Append a system prompt

This is what I landed on.

Claude Code takes a --append-system-prompt-file flag. Whatever is in that file gets appended to the system prompt every single turn. Not memory, not context, the actual system prompt.

Write your rules to a file:

~/.claude/concise.md

Then alias the command so you don't have to remember the flag:

# ~/.zshrc
alias claude='claude --append-system-prompt-file ~/.claude/concise.md'

Open a new terminal and you're done. Every session, every project, no flag to remember.

Two tradeoffs worth knowing. It applies everywhere, which I wanted but you might not. And it's invisible, so six months from now you may forget why your output looks different. Leave yourself a comment in the alias.

What goes in the file matters more than the mechanism. "Be concise" is useless. Ban specific behaviors:

No recapping work the user just watched
No status inventories unless asked
No preamble ("Great", "You're right", "Let me")
No closing summary paragraph
No unsolicited next-step menus
Answer first, context second

Specific prohibitions beat vague vibes. Every time.

A Stop hook

This is the only option that actually forces anything.

Hooks are different from everything above. The harness runs them, not the model. A Stop hook fires when Claude finishes a response, and it can block that response and send feedback, forcing a retry. You configure it in settings.json alongside whatever else you're running.

That's real enforcement. Nothing else on this list is.

I passed on it for two reasons. A blocked response means a retry, and a retry means more tokens, which is a strange way to solve "you're wasting my tokens yapping." And any length check crude enough to write is crude enough to block an answer that genuinely needed the room.

If you want determinism and you accept the cost, this is your option. I wanted most of the benefit without the retry loop.

What I'd tell you to do

Start with CLAUDE.md and append a system prompt. The alias is one line, it works everywhere, and you can delete it in five seconds.

Spend your effort on the rules file, not the mechanism. Ban behaviors by name. Vague instructions produce vague compliance.

And be honest with yourself about why you want this. Walls of text aren't a preference problem, they're a wall. Tuning your tools around how your brain actually works isn't precious. It's the whole point of having tools you can configure.

The model won't hold a rule you only mention once. Put it somewhere structural.

How I Grew Signups 71% in 90 Days by Running GTM Like an Engineer

Tessa Kriesel — Wed, 08 Jul 2026 02:19:55 +0000

In my first 90 days at Tabstack, signups went up 71%. I didn’t get there by running a pile of campaigns. I got there by building software.

I market developer tools, and I’ve been writing code since 2006. That combination changes how I work. Where a lot of marketers open a campaign calendar, I open my terminal. I treat go-to-market like an engineering problem: understand the system, build it, ship it, measure it, then make it better.

Here’s exactly what I did, week by week, and the play you can take from each one.

Before Day One: I Built on the Product First

The best decision I made happened before my first day.

Once I accepted the offer, I built a real product on Tabstack’s API. It’s called Rival, an open-source competitive-intelligence app that tracks everything a competitor does in public: pricing, hiring, product changes, positioning, reviews, GitHub activity, and it runs full cited research on demand, then tells you what actually changed and uses all five of Tabstack’s endpoints.

I do this kind of thing with every developer product I take on. I want to understand the product I'm marketing. So I built, and ran a full developer-experience audit while I built Rival, logging every rough edge I hit. Developers are the audience. I wanted the developer experience improved before I sent more developers there.

Rival actually serves multiple purposes. It’s a test surface for the product. It’s a competitive-intelligence tool I use every day. And it’s a distribution play: it’s open source, self-hosting it requires a Tabstack API key, and I had the campaign to get it in front of developers planned before I ever started.

The play: use the product yourself, at real depth, before you market it. Build something real with it, not a click-through demo, so you understand what you're actually marketing and you catch the gaps early, before your users hit them. If you're not a coder, try Claude code or other coding agents, they're quite good. Your developers will thank you.

Rival is open source. You can try the live demo or dig into the code.

Week One: I Stood Up My Systems Before I Ran Any Marketing

I came in already knowing the product and the direction I wanted to take, so I started with systems, not deliverables. I built our GTM Command Center, the hub the whole operation would run on, so a single change to our positioning reaches everything downstream. And I stood up our CRM under a real constraint: at Mozilla, consent is off by default and we collect almost nothing unless the user says yes, so I can’t lean on behavioral tracking to understand who signs up. My CRM works entirely from public data instead, respectful by design. The Mozilla way.

I'd done this before. I ran a developer GTM agency before Tabstack, so I came in with a playbook, not a blank page. These systems are that playbook, built for Tabstack.

The play: don’t start with campaigns. Start with the infrastructure that makes every campaign easier to run.

Week Two: I Turned the Docs Gap Into a System

When you actually build on a developer product, its problems announce themselves. A setup step fails silently. The docs skip the exact case you just hit. The messaging never quite says what the thing does. We had all of it, and I refuse to send a developer to a bad developer experience, so I fixed ours before I drove anyone to it. Since I’m a team of one, I didn’t rewrite the docs myself. I built a system, trained deeply on Tabstack: a skill for each kind of doc, a knowledge base of the product, review steps that check every claim against the live API, and a pipeline that drafts and opens a pull request for me to approve.

It wrote 45 docs in a single day, across half a dozen pull requests, that needed only light polish.

This is how I work: I build a system to solve the problem, then keep improving it as I use it. For a team of one, that's the best way to scale. The one I wrote for docs writes my marketing content now, too.

The play: never send a developer to a bad developer experience. Marketing only gets a developer to the door. The developer experience decides whether they stay, and developers judge it fast and rarely give it a second look. That makes DX a prerequisite for marketing, not a parallel track. So fix the docs, the messaging, and the setup that fails silently before you drive anyone to them. And fix them with a system, not a one-off, so they hold as the product keeps changing.

Week Three: I Did the Strategic Work, Then Placed the Bets

Leadership wanted to put money into marketing earlier than I’m normally ready to make bets on. Early or not, I wasn't going to spend it on a guess. So I started with the questions that actually matter.

What does this product need to succeed? How do we stand apart from everyone else fetching web data? What pays off this month, what compounds over multiple quarters, and how do I fund both at once?

The answer was five bets, each one tied to how developers actually find and adopt tools: showing up in the tools they already use, a public schema library where every use case becomes its own discoverable entry point, content built to be found by the AI models developers now ask, brand and marketing execution, and field and community distribution.

The play: do the strategic work before you spend a dollar. Understand the market, define what you are and how you differ, then make a short list of bets that balance quick wins against longer plays that compound. The best opportunities will make themselves known this way.

Week Four: I Launched by Day 24, With the Data Dashboard First

By day 24, we had our first Product Hunt launch live.

Product Hunt was deliberate, and not for the obvious reason that developers are there. It’s a source the AI models trust, so a launch doubles as training signal for the assistants developers now ask. I also have a track record there, which buys speed. I planned and launched this one in a few days. Early, trusted, and fast to run was exactly the profile I wanted for a first move.

Two days before it went live, I stood up our first metrics dashboard across the whole developer journey, API usage, activation funnel, time to value, playground to production, so the moment traffic hit I could see what it did, not just that it showed up. Instrument before you ship, or the launch is noise you can’t learn from.

The play: early on, ship one real growth effort, and choose it by impact. You’ll have a hundred things you could do. The discipline is picking the one that drives the most growth right now and getting it out, while the longer-term work keeps moving in parallel.

Weeks Five and Six: I Built an Agent to Scale Myself, Not to Replace Myself

There’s too much for one person to track. Competitors, community, prospects, sponsorships, content, events. So I built a go-to-market agent, on the open-source Hermes framework, to take the repetitive gathering off my plate.

The agent runs the recurring jobs on a schedule. Daily competitive-intelligence digests. Community listening across Hacker News, Reddit, GitHub, and more. Prospect signals from our CRM. Prospecting batches. Sponsorship research. A social pipeline with MentionDrop. Event-prep briefs. It commits its reports and pings me.

Here’s a concrete one. Rival already runs a cron every night, pulling competitor updates and pushing them into the GTM Command Center. The agent picks that up and synthesizes it, so instead of reading raw diffs every morning, I get the read that actually matters.

The agent gathers and drafts. I read every report, make the call, and do the work. The system makes me faster. It does not replace me. That distinction is the whole reason it works: I get leverage without giving up a single decision.

The play: use AI to scale yourself. There is always more GTM work than one person can do by hand, so hand the repetitive gathering to an agent and keep your hours for judgment and action. One person can run a full operation this way. The leverage is real, and every decision stays yours.

Week Seven: The Machine Started Pumping

This was the inflection point. The daily reports turned on, and the systems started feeding me fresh intelligence every morning instead of me hunting for it.

Two things made it click. First, I wired the systems together: the agent could now read and write the CRM directly, and the GTM Command Center held the single source of positioning that fed everything else. Second, I built the operating layer that gives all that daily output somewhere to go, a content calendar and backlog, a distribution system, and outreach lists pulled from my own network of thousands of contacts.

The play: this is where the work starts compounding. Once your systems connect and feed you output, the job flips from setting the machine up to running on what it produces, and every new problem is easier to solve because the system to handle it is already there.

Week Eight: I Turned Product Hunt Into an Engine

Our second launch, Web Research, went out about a month after the first. From there they came almost weekly, on purpose. A launch every week is a forcing function, and the amount of work behind a single one is the part people underestimate: the positioning and messaging, the website work itself (product pages designed and built, homepage and navigation updates, the SEO and discoverability layer), a working demo, images and launch cards, a video, the Product Hunt listing and maker comment, forum posts, social, a launch email, and answering every question in the thread on the day. The fixed deadline is what gets all of it out the door, week after week.

I already knew the whole site needed a rebuild. I didn’t wait for it. I designed and built a focused product page for this launch myself, so we had something strong to land on now, and left the full rebuild for later.

The play: don’t wait for perfect resourcing to move. Ship a focused page instead of blocking on a full redo, an agency, or a designer, and own your launch surface end to end. Then put yourself on a cadence and let it do the forcing. Nothing gets work out the door like a deadline you committed to.

Week Nine: I Rebuilt the Whole Website Presence in a Weekend

I knew from early on the site needed a full facelift. The old homepage was a single page, and it was weak. Developers couldn’t tell what Tabstack did or didn’t do.

But I didn’t rebuild the whole site first. I did higher-leverage work and waited. When our research page shipped as a one-off landing page with a new design, it set the standard the rest of the site had to meet. So the facelift happened between launches: the product pages, the homepage relaunch, and the search-visibility layer underneath it, llms.txt, per-page structured data, canonicals, a branded 404, a clean sitemap, so the models developers ask can actually find us. Each launch got a stronger surface to land on, and the presence kept improving without ever blocking a launch.

The play: improve your GTM presence, but sequence it. Recognize the big investment early, then hold off until it’s the highest-leverage move instead of the urgent one. Let one strong page set the design and messaging standard, then roll the rest out between launches, so your presence keeps getting better without ever blocking one.

Week Ten: I Reinvested in the Systems & Data

I pulled the whole operation into one terminal workspace with memory, so I start every session already in context instead of re-explaining myself. And I improved the data. With consent off by default and nothing collected unless you allow it, reliable attribution is genuinely hard, so I made it work within that limit, keeping referrer and UTM data through the consent step instead of losing it. A better workspace makes me faster. Fuller data makes every decision better. I do everything in here from ad copy, to email outreach drafts, to building agent skills. It empowers me to do everything within the defined system.

This is also the week I turned on our first paid spend, a TLDR newsletter placement, and not a day before. I waited until the site and the developer experience were good enough that the traffic would land on something ready.

The play: reinvest in your own systems and your data as you go. Efficiency and better information both compound. And don’t buy traffic until you’ve built something worth sending it to.

Week Eleven: The Systems Started Paying Me Back

By now, running go-to-market like an engineer had a feel to it. Every time I hit a wall, I solved it for good.

I’d made launch images by hand for three launches, hours each in Claude Design, so I built a helper that renders them on brand in minutes. My next launch’s imagery took me minutes. I planned a ten-category schema library and had the agent build it in a single overnight batch while I reviewed and shipped it in the morning, weeks of work done in hours. And I measured it hard: a four-launch traffic comparison, a growth report going back to April, attribution diagnostics, a case-study shortlist of our most-engaged orgs, and an exec narrative packaged into a deck.

I’m not doing less. The more my systems handle, the more I get to take on, well beyond what one person could carry alone.

The play: solve every hurdle once. Feel the pain a couple of times, then build the thing that removes it for good, a tool that turns hours of design work into minutes, an agent that runs an overnight build while you sleep. Each fix makes the next thing easier, and easier is what lets one person do the work of a team. That is what running go-to-market like an engineer actually buys you.

Week Twelve: Not Every Launch Is a Tentpole

Our fifth launch, Schema Source, was a quick one. It put eyes on the schema source tool our team built and the schema library of pre-defined schemas my agent built and kept the cadence alive without a heavy lift.

The rest of the week went to the work that doesn’t announce itself. I integrated a fresh positioning pass so our message stayed sharp as we grew, and I got the Browser Automation launch copy ready for the next week, keeping it strictly to what was actually shipping, because overselling a developer tool costs you the moment someone tries it.

The play: not every launch has to be a tentpole. A quick, light launch off something you already shipped keeps the cadence alive and puts eyes on an asset, without the heavy lift of a marquee release. The cadence is the engine. Sometimes you just feed it with what you already have.

Week Thirteen: Final Push before Day 90

Week thirteen ran hard. The sixth launch, Browser Automation, went out with its announcement email. I shipped a competitor-brief tool, and finished the CRM enrichment layer that turns an email into a cited public profile on Tabstack’s own research endpoint. Even then I kept tuning the systems, because at day 90 I’m still sharpening them.

Then I measured my own 90 days the same way I measure a launch. Here’s what the systems produced:

Signups up 71%
Activation up 29%
API requests up almost 6x
Website traffic up 129%
Around 1,096 commits
29,938 Tabstack API calls of my own, the heaviest usage on the platform

Tabstack is one tool in my stack, but it earns its spot. Rival is built entirely on it. Here’s a small example of why. I wanted to pull a recipe off a page and get clean, structured JSON back for a blog. I asked Claude to just do it, and it failed. Then I had it call Tabstack, and the data came back in seconds, structured exactly the way I needed and exactly what the model couldn’t produce on its own.

The Bottom Line

I grew signups 71% in 90 days by building the systems that made me faster. Work smarter, not harder.

If you market a developer tool, or you’re a developer thinking about marketing your own, here’s the whole thing in one breath. Learn the product by building on it. Fix the developer experience, and build the system that keeps it fixed. Turn launches into a cadence, and never launch without your data ready. Use AI to scale yourself without handing over the judgment. And solve every hurdle once.

You don’t have to be an engineer to run go-to-market this way. It just helps to think like one. I’m a marketer for developers because I’m a developer, and after 90 days I’m more sure than ever that’s the advantage.

If you’re building agents that need to read the live web, Tabstack is worth a look. Grab an API key.

Everything I Built and Shipped

If you want the action behind the playbook, here’s the full body of work from the quarter.

The systems (the machine everything runs on)

Rival : an open-source competitive-intelligence app on all five Tabstack endpoints: multi-surface scanning (homepage, pricing, blog, changelog, reviews, profile), AI-generated diffs, on-demand cited research, a positioning matrix, threat scoring, a public streaming demo, nightly pushes into the GTM Command Center, and its own MCP server. Open-sourced.
GTM Command Center : the hub the whole operation runs on: versioned positioning and product context, AI work sessions (strategy, creation, feedback, analysis, optimization), a campaigns module with an AI brief generator, a versioned content and artifact library with SEO/OG and distribution, customer-research ingestion (themes, sentiment, quotes), performance logging that proposes updates back to the context, live metric connectors (PostHog, GA4, EthicalAds), a campaign-performance dashboard, team management, notifications, magic-link auth with domain gating, and an MCP server every other system reads and writes to.
Tabstack CRM : public-data-only enrichment and ICP scoring, an identity pass (Gravatar, GitHub, npm, PyPI, Slack), a source-linked signal feed, a prospect and account pipeline, funnel-stage imports, and a hosted MCP server. Later moved to Airtable with a trace-based enrichment pipeline.
GTM agent : built on the open-source Hermes framework: recurring competitive-intel, community-listening, prospecting, sponsorship, and social jobs, network intelligence, publishing integrations, and a Telegram interface. It gathers and drafts; I decide and act.
Tabstack OS : one terminal workspace with memory: a state-of-work brain, session briefings, reference-repo sync, and a portable knowledge mirror.
The tech-writing and blog-authoring system : a skill per doc type, a product knowledge base, accuracy-checking review agents, and a draft-to-pull-request pipeline.

Skills, subagents, and com mands I built

Custom skills: api-docs, tech-writing, changelog, readme-writer, tabstack-sdk, blog-author, external-publication-editor, sdk-sync, quiver-publish, product-marketing, vbf-messaging, competitor-profiling, state-of-work, crm-import, writing-clearly-and-concisely, plus a mirrored marketing-skills pack
Review subagents: api-reviewer, docs-reviewer, readme-writer
Slash commands for the doc and publishing workflows

Marketing website (built and rebuilt)

Four product pages: structured extraction, browser automation, web research, and competitive intelligence, with hero demos and real token-cost visuals
A full homepage relaunch (dark nav and footer, Stack mega-menu) and a blog redesign
A Developer Hub / resources overhaul and a Use Cases section
Attribution plumbing: referral/UTM passthrough into signup CTAs, entry referrer/UTMs retained across the cookie-consent delay, and internal links de-tagged
Consent-gated web analytics
AI-search visibility: llms.txt and llms-full.txt, per-page JSON-LD, canonicals, keyword-targeted meta titles, a branded 404, a clean sitemap, and legacy redirects
A consistent UTM + GA taxonomy across every CTA, an accessibility pass, and a font migration to the Mozilla brand faces

Docs

45 docs in a single day
The Interactive Mode set (quickstart, guides, launch post) and an accuracy pass
Per-endpoint API references (extract/json, extract/markdown, generate/json, automate, research)
Quickstarts (first extraction, interactive mode, research)
Seven how-to guides (effort levels, geotargeting, schema design, streaming, production reliability, integrations, LangChain)
A 14-page comparisons section and SDK-first rewrites
A changelog page covering SDK 2.0.0–2.6.1
Ongoing docs work from DevRel: error reference, research guide, a generate-vs-extract page, production rate limits, a full docs audit, and single-source changelog data

Developer tooling

The Tabstack CLI (built by DevRel, hardened to launch-ready, with a schema-pull command and install script)
A public vertical schema library, ten categories built in a single overnight batch, plus a competitor-profile schema
A competitor-brief demo app spanning all five endpoints (an 11-section streaming brief with a two-URL positioning comparison)

Product Hunt launches (six)

Six launches: the first Tabstack launch, then Web Research, Structured Extraction, Dev Tools, Schema Source, and Browser Automation
For each one: positioning, messaging, taglines, the listing and maker comment, ten forum posts, a launch email, and answering questions in the thread on the day
A reusable Product Hunt image helper (on-brand launch cards in minutes), launch-day sentiment analysis, and launch-history + launch-comparison research
Launch videos by DevRel

GTM intelligence and reporting

Daily competitive-intel, community-signal, and CRM-signal reports; weekly competitive summaries, account intelligence, prospecting, sponsorship, social, and events reports
A four-launch traffic comparison, a growth report since April, attribution diagnostics, an engaged-orgs case-study shortlist with pre-filled briefs, and an exec growth narrative with a slide deck

Content

A dozen technical blog drafts, personal-voice and brand voice profiles, a monthly content calendar, a distribution system, and a 30-post backlog

Positioning and messaging

A five-bet growth investment plan, a VBF messaging framework, six audience micro-segments, a jobs-to-be-done framework, exec and company one-pagers, and a full positioning integration that leans on Mozilla’s trust and privacy stance (ephemeral processing, no training on your data, privacy by default)

Specs and plans

Design specs and implementation plans for the blog-author system, the Tabstack OS workspace, the using-Tabstack skill, the competitive-intelligence sales page, and the competitor-brief tool

Analytics and dashboards

Three PostHog dashboards (GTM Metrics, live Launch Day, and Growth/Attribution/Signups) with 36 insights
The GTM Command Center’s own campaign-performance dashboard, pulling spend and metrics from PostHog, GA4, and EthicalAds to show cost-per-outcome across every campaign

Paid and community distribution

A TLDR newsletter placement (first paid spend), a bytes.dev newsletter sponsorship, an EthicalAds campaign (per-segment copy and creative), Reddit Ads, and a paid-channel experiment plan
Creator and sponsorship lists mined from a network of thousands
An agent-ecosystem play: registered the product as an agent, plus an agent-injection campaign plan

Team and ops

Hired and directed our DevRel engineer, Steve McDougall, brought on Flo Merian as our Product Hunt hunter, and onboarded our brand-and-marketing agency, Conversion Factory

Research and evaluation

A pricing evaluation, competitor ad-intelligence research, and a developer-marketing playbook synthesis

And the dogfooding receipt

27 product issues filed to engineering from real use, and the #1 power user of the Tabstack API at 29,938 calls

I Stopped Checking My Mentions Dashboard. Now an Agent Drafts the Replies.

Tessa Kriesel — Tue, 16 Jun 2026 23:06:44 +0000

Every monitoring tool sells you the same thing. A dashboard. You open it, you scroll, you promise yourself you'll keep up. Two weeks later you stop opening it.

Sound familiar? The dashboard was never the point. The point is the handful of conversations happening right now where you could be useful, and those rot in a browser tab you forgot to check.

I run GTM for Tabstack, so this as an actual problem I needed to solve. I need to know when someone mentions us, when someone's frustrated with a tool in our space, when there's a question I can help with. But those few conversations sit buried under hundreds that look relevant and aren't. That's the work: cutting the noise down to what matters, then replying before it goes cold. Neither half scales by adding another tab to your morning.

So I'd already taken a swing at it. My Hermes GTM agent runs a community signals report every morning before I'm even at my desk. It works. But on its own, it leaned toward housekeeping tasks: which PR merged, which repo pushed overnight. The conversations I actually wanted stayed the thinnest slice of the report, because raw keywords drown them in noise.

Then MentionDrop shipped an MCP. I wired it into Hermes this week, and the report I re-ran was completely different.

What the MCP actually changes

MentionDrop has always done the hard part, filtering social noise into real signal. The MCP changes how you use it. Instead of a dashboard I visit, it becomes a set of tools my agent calls directly: the same filtered data, exposed so Hermes can pull it without me clicking anything.

That's the whole shift. A tool with an MCP doesn't have to be a place you go. It can be a capability your agent has.

Connecting it took about two minutes. The tuning came after. A broad keyword pulls in everything at first; one of mine was matching a soccer player, an obituary, and a makeup app. What sold me is how much control MentionDrop hands you to fix that. Every keyword takes a role, a context sentence, a relevance floor, and exclusion terms. A few minutes turned a messy feed into a high-signal one, and because the filter learns from what I mark reviewed, tomorrow's run will be sharper than today's.

Here's the division of labor I landed on: MentionDrop does the filtering, Hermes pulls what's left into one morning list and drafts a reply for each in my voice. But Hermes can't touch my account or post anything in public. That part is mine. I read the draft, make my edits, and hit post. Every draft also clears my editorial principles before it reaches me, so what I'm reviewing already sounds like me.

What I actually get now

A short list each morning. Five or six conversations, not five hundred. Each one comes with a line on why it matters, a direct link, and a reply already drafted the way I'd say it.

Here's one from today's run. Hermes flagged a developer on r/ClaudeAI stuck trying to install Apify, the kind of friction moment where an alternative is genuinely welcome. It pulled the thread, noted why it mattered, and wrote the draft:

Apify's MCP setup can be pretty finicky, the actor runtime adds a bunch of overhead that doesn't play nicely with Claude's tool calling in every setup.

If you're trying to pull structured data or do browser-based research, you might find Tabstack's API simpler to wire up. It's a plain REST API with no runtime to configure. Happy to share a quick example if you tell me what you're trying to extract.

I read it, removed spaces around the em dashes, and hit post. Done in 3o seconds.

Compare that to what Hermes gave me on its own a few days ago: a thorough rundown of GitHub activity. Which issues got acknowledged, which PR was blocking a launch, which repo pushed overnight. All true, all useful for ops. But not one conversation I could go be useful in. That report was a status update. This one's a revenue-driving to-do list.

"MentionDrop's MCP turns my Hermes agent's morning report from a status update into a revenue-driving to-do list."

The bottom line

I only set this up this week, so I won't pretend it has remade everything. But the first real run handed me a list worth acting on, and that's the part that has me hooked. The tool didn't change. MentionDrop was already good at the filtering. What changed is that it shipped an MCP, and that turned it from something I check into something my agent runs for me.

That's what I keep coming back to. My agent was already doing the work. It just needed better inputs. An MCP is how a good tool stops being a destination and becomes a capability.

How I Deployed Hermes Agent on Railway with Telegram (And Every Gotcha I Hit Along the Way)

Tessa Kriesel — Tue, 12 May 2026 21:14:06 +0000

TL;DR for the skimmers

Hermes is NousResearch's self-improving AI agent framework—persistent memory, skills system, multi-platform messaging, multiple LLM providers
You can run it as a persistent Docker service on Railway with Telegram as your interface
The docs don't cover Railway/Docker deployment, so I had to figure most of this out through trial, error, and log-reading
There are 10 specific gotchas that will wreck your deploy if you don't know about them
Full working files at the end of this post

Why I built this

When I found Hermes—NousResearch's self-improving agent framework—I knew I wanted it in the cloud. I didn't want to think about whether my laptop was on. I wanted to message it from my phone at any hour and have it just work.

The problem? There is essentially zero documentation for deploying Hermes to a persistent cloud service like Railway. The install docs assume you're running it locally. The Docker support exists but isn't documented in any deployment context. And when you're dealing with a gateway-based agent that uses volumes, environment variables, MCP subprocess servers, and Telegram webhooks, the failure surface is enormous.

This post is the full deployment story—every gotcha, every fix, every file—so you don't have to spend three days in Railway logs the way I did.

What Hermes actually is

Before I get into the deployment mechanics, let me quickly explain what Hermes is and why it's worth the deployment friction.

Hermes is NousResearch's AI agent framework. What makes it different from a simple chatbot or wrapper:

Self-improving skills system : Hermes creates skills from experience and improves them over time. You can also load external skill directories.
Persistent memory : MEMORY.md and USER.md build up across sessions. The agent actually remembers you.
Multi-platform messaging : Telegram, Discord, Slack, WhatsApp, Signal, email—you pick your interface.
Multi-LLM support : Anthropic, OpenRouter, Nous Portal, and more. You can configure fallback providers.
MCP support : Connect external tools via Model Context Protocol servers.
Cron jobs, kanban, session search, hooks : Full agentic infrastructure baked in.
Config lives in ~/.hermes/ : Main files are config.yaml, .env, and SOUL.md.

Getting it running on Railway

This section covers only what's Railway-specific. For general Hermes setup—skills, SOUL.md content, memory configuration—refer to the Hermes repo. What follows is the deployment layer on top of that.

Prerequisites

Before you start:

Railway account with the CLI installed (npm install -g @railway/cli)
Railway CLI logged in: railway login
A Telegram bot token (create one via @BotFather)
Your Telegram user ID (message @userinfobot)
An Anthropic API key (or whichever LLM provider you're using)
Docker installed locally if you want to test the build before deploying

Step 1: Create your repo structure

your-agent/
├── Dockerfile
├── entrypoint.sh
├── railway.json
├── .env.example
├── .gitignore
├── AGENTS.md # Optional — auto-injected context every session
└── hermes/
    ├── cli-config.yaml
    └── SOUL.md

The hermes/ directory is your agent's config source. The entrypoint copies these files into the Railway volume on every startup, so changes you commit and deploy take effect immediately.

Step 2: Create the core files

Create Dockerfile, entrypoint.sh, railway.json, and hermes/cli-config.yaml using the full working files later in this post. Come back to this step once you've read the gotchas—they'll explain some choices in those files that would otherwise seem arbitrary.

The one file you'll want to create now is hermes/SOUL.md. This is your agent's identity—who it is, what it knows, how it behaves. The Hermes repo covers what goes in SOUL.md. Create it before your first deploy so the agent starts with a defined persona rather than a blank slate.

Step 3: Set up the Telegram bot

Message @BotFather on Telegram
Send /newbot and follow the prompts
Copy the token it gives you—that's TELEGRAM_BOT_TOKEN
Message @userinfobot to get your numeric user ID—that's TELEGRAM_ALLOWED_USERS and TELEGRAM_HOME_CHANNEL (they're the same value for a personal DM setup)

Step 4: Initialize the Railway project

From your repo root:

# Initialize and link to Railway
railway init

# Create the service with an initial deploy
railway up --detach

# Link to the service by name
railway service "your-service-name"

Step 5: Set environment variables

railway variables set ANTHROPIC_API_KEY=sk-ant-...
railway variables set TELEGRAM_BOT_TOKEN=...
railway variables set TELEGRAM_ALLOWED_USERS=123456789
railway variables set TELEGRAM_HOME_CHANNEL=123456789
railway variables set TELEGRAM_HOME_CHANNEL_NAME="Your Name DM"
railway variables set HERMES_HUMAN_DELAY_MODE=natural
railway variables set HERMES_ACCEPT_HOOKS=1

Add any additional secrets for MCP servers you're connecting:

railway variables set NOTION_TOKEN=...
railway variables set NOUS_API_KEY=... # if using Nous Portal fallback

Step 6: Add the persistent volume

This is the most important step and the order matters—add the volume before your first real deploy.

railway volume add --mount-path /root/.hermes

The volume must be mounted at /root/.hermes—the entire Hermes home directory, not a subdirectory. See Gotcha 2 for why this matters.

Step 7: Redeploy and verify

railway up --detach

Watch the build logs in the Railway dashboard. A successful startup looks like:

Hermes config ready. Starting gateway...
⚕ Hermes Gateway Starting...
Messaging platforms + cron scheduler

Once you see the gateway banner, send a message to your Telegram bot. It should respond.

Step 8: Set up auto-deploy on push

Railway doesn't automatically redeploy when you push to GitHub unless you connect the repo through their UI. The easiest workaround is a git post-push hook:

# Create the hook
cat > .git/hooks/post-push << 'EOF'
#!/bin/bash
echo "Deploying to Railway..."
railway up --detach
EOF

chmod +x .git/hooks/post-push

Now git push triggers a Railway deploy automatically.

The 10 gotchas

I want to share the gotchas because they are the entire reason this post exists. None of these are documented anywhere. The first six will affect everyone doing this setup. The rest depend on your configuration choices—but if you're adding MCP servers or fallback providers, you'll hit those too.

Gotcha 1: `gateway start` vs `gateway run`

The Hermes docs (and some examples floating around) reference hermes gateway start. Don't use that in Docker. It's for host machine services.

Inside a Docker container, you will get this error:

Service start is not applicable inside a Docker container.
Or run the gateway directly: hermes gateway run

Fix: In your entrypoint.sh, use exec hermes gateway run—not start.

exec hermes gateway run

The exec matters too—it replaces the shell process so tini can properly manage the process tree.

Gotcha 2: Volume mount path was wrong

This one cost me the most time. I initially mounted the Railway Volume at /root/.hermes/memories—just the memories subdirectory—thinking that was the only thing I needed to persist.

Wrong. Hermes stores a lot more than memories outside that subdirectory:

Gateway state
Session history (state.db)
Cron jobs
The home channel setup state

Every deploy wiped all of that, so the bot would come back up and immediately ask me to "set your home channel" as if it had never been configured. Completely stateless on every restart.

Fix: Mount the volume at /root/.hermes—the entire Hermes home directory.

railway volume add --mount-path /root/.hermes

Gotcha 3: Wrong MCP config key

I burned an embarrassing amount of time on this one. My cli-config.yaml had:

mcp:
  servers:
    my-tool:
      url: "https://..."

Hermes does not recognize this structure. The correct top-level key is mcp_servers:—flat, not nested under mcp:.

Fix:

mcp_servers:
  my-tool:
    url: "https://..."

No error is thrown for the wrong key—Hermes just silently ignores the MCP servers entirely. You only notice because your tools don't show up.

Gotcha 4: `config.yaml` vs `cli-config.yaml`

The sample file in the Hermes repo is named cli-config.yaml.example. Naturally, I named my file cli-config.yaml. That works for CLI usage.

But the gateway reads config.yaml at runtime. The CLI and the gateway use different config loaders, and they look for different filenames.

Fix: Write your config to both files in entrypoint.sh:

cp /app/hermes/cli-config.yaml "$HERMES_HOME/cli-config.yaml"
cp /app/hermes/cli-config.yaml "$HERMES_HOME/config.yaml"

Maintain one source file in your repo and copy it to both locations on startup. Config changes propagate correctly regardless of which loader is running.

Gotcha 5: MCP secrets not reaching Hermes

I had all my MCP secrets set as Railway environment variables. They were there in the Railway dashboard, visibly set, verified. Hermes couldn't see them.

The issue: Hermes reads secrets from ~/.hermes/.env, not directly from the system environment. My entrypoint.sh was only writing a handful of variables to that file—not the MCP secrets.

Fix: Write ALL secrets to ~/.hermes/.env in entrypoint.sh:

cat > "$HERMES_HOME/.env" << EOF
ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY:-}
TELEGRAM_BOT_TOKEN=${TELEGRAM_BOT_TOKEN:-}
TELEGRAM_ALLOWED_USERS=${TELEGRAM_ALLOWED_USERS:-}
TELEGRAM_HOME_CHANNEL=${TELEGRAM_HOME_CHANNEL:-}
TELEGRAM_HOME_CHANNEL_NAME=${TELEGRAM_HOME_CHANNEL_NAME:-}
MY_MCP_SECRET=${MY_MCP_SECRET:-}
NOTION_TOKEN=${NOTION_TOKEN:-}
HERMES_HUMAN_DELAY_MODE=${HERMES_HUMAN_DELAY_MODE:-natural}
HERMES_ACCEPT_HOOKS=${HERMES_ACCEPT_HOOKS:-1}
NOUS_API_KEY=${NOUS_API_KEY:-}
EOF

The :- pattern uses the environment variable if set, empty string if not. Keeps the file clean even if some optional secrets aren't configured yet.

Gotcha 6: Agent is completely silent after deploy

After fixing the gateway run command, the bot would start, logs looked clean, no errors—and then complete silence. Send a Telegram message, nothing comes back. No errors, no timeouts, no indication anything was wrong.

The cause: TELEGRAM_HOME_CHANNEL was not set. When Hermes doesn't know its home channel, it enters a waiting-for-setup state and ignores incoming messages entirely. It's not an error state—it's just waiting.

Fix: Set TELEGRAM_HOME_CHANNEL to your Telegram user ID as a Railway environment variable. For a personal DM bot, this is the same as your user ID. Find it by messaging @userinfobot on Telegram.

railway variables set TELEGRAM_HOME_CHANNEL=123456789

Gotcha 7: No tini = zombie MCP processes

Hermes runs MCP stdio servers as subprocesses—for example, the Notion MCP runs as npx @notionhq/notion-mcp-server. Without a proper init system as PID 1, these subprocesses become zombies when they exit. They accumulate over time and eventually cause problems.

The upstream Hermes Dockerfile uses tini explicitly for exactly this reason. I missed that detail initially.

Fix: Install tini in your Dockerfile and use it as the entrypoint. Critically, do NOT set a startCommand in railway.json—Railway's startCommand overrides the Docker ENTRYPOINT, which means tini never runs. Leave startCommand out and let Railway use the ENTRYPOINT directly.

RUN apt-get install -y tini
ENTRYPOINT ["/usr/bin/tini", "-g", "--", "/entrypoint.sh"]


{
  "build": { "builder": "DOCKERFILE" },
  "deploy": {
    "restartPolicyType": "ON_FAILURE",
    "restartPolicyMaxRetries": 10
  }
}

The -g flag means tini will send signals to the entire process group, which matters for subprocess cleanup.

Gotcha 8: stdio MCP servers fail to start in the container

If you're using stdio-based MCP servers (like the Notion MCP via npx), the package has to be downloaded on first invocation. In a Railway container, this download can fail entirely—leaving the MCP server in an infinite retry loop with no useful error message. You won't see a crash. You'll just see your MCP tools never appear and Notion never responds.

Fix: Pre-install the package in the Dockerfile so it's already there when the container starts:

RUN npm install -g @notionhq/notion-mcp-server

After this, npx finds the package locally and starts immediately. Do this for every stdio MCP server you configure.

Gotcha 9: `api_max_retries` too high with fallback providers

I had api_max_retries: 3 in my config. What this means in practice: if Anthropic hits an error or rate limit, Hermes retries the same provider 3 times before trying the fallback. That's potentially 3 failed requests with exponential backoff before you failover.

Fix: Set api_max_retries: 1 when using fallback providers:

agent:
  api_max_retries: 1

One failure on the primary, immediately tries the fallback.

Gotcha 10: Will the volume hide the Hermes install?

When running as root on Linux (Docker default), the Hermes install script puts code at /usr/local/lib/hermes-agent—not at ~/.hermes. So mounting a volume at /root/.hermes does NOT hide or overwrite the installed Hermes code. It only covers user config and data.

I mention this because I spent time wondering whether my volume mount was somehow replacing the install. You don't need to reinstall Hermes on every container start. The install is baked into the image. The volume handles your data only.

The full working setup

Dockerfile

FROM python:3.13-slim

RUN apt-get update && apt-get install -y --no-install-recommends \
    curl bash git nodejs npm build-essential tini \
    && rm -rf /var/lib/apt/lists/*

# Pre-install any stdio MCP servers to avoid cold-start downloads
RUN npm install -g @notionhq/notion-mcp-server

# Install Hermes — pinned to a specific commit for reproducibility
# Update this SHA when you want to upgrade Hermes
ARG HERMES_COMMIT=c23a87bc163b188abc7e40fbdccf07a9739231c3
RUN curl -fsSL "https://raw.githubusercontent.com/NousResearch/hermes-agent/${HERMES_COMMIT}/scripts/install.sh" \
    | bash -s -- --skip-setup

ENV PATH="/root/.local/bin:${PATH}"

WORKDIR /app
COPY . .

# Sanity check — verify install succeeded
RUN hermes --version

COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh

EXPOSE 8443

# tini as PID 1 to handle zombie subprocess cleanup
ENTRYPOINT ["/usr/bin/tini", "-g", "--", "/entrypoint.sh"]

entrypoint.sh

#!/bin/bash
set -euo pipefail

HERMES_HOME="${HERMES_HOME:-$HOME/.hermes}"

# Create all directories Hermes expects (volume may be empty on first deploy)
mkdir -p "$HERMES_HOME/memories" \
         "$HERMES_HOME/skills" \
         "$HERMES_HOME/sessions" \
         "$HERMES_HOME/cron" \
         "$HERMES_HOME/cron/output" \
         "$HERMES_HOME/hooks" \
         "$HERMES_HOME/logs"

# Write config to BOTH filenames — CLI uses cli-config.yaml, gateway uses config.yaml
cp /app/hermes/cli-config.yaml "$HERMES_HOME/cli-config.yaml"
cp /app/hermes/cli-config.yaml "$HERMES_HOME/config.yaml"

# Write SOUL.md — agent identity/persona
cp /app/hermes/SOUL.md "$HERMES_HOME/SOUL.md"

# Write ALL secrets to ~/.hermes/.env — Hermes reads from here, not system env
cat > "$HERMES_HOME/.env" << EOF
ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY:-}
TELEGRAM_BOT_TOKEN=${TELEGRAM_BOT_TOKEN:-}
TELEGRAM_ALLOWED_USERS=${TELEGRAM_ALLOWED_USERS:-}
TELEGRAM_HOME_CHANNEL=${TELEGRAM_HOME_CHANNEL:-}
TELEGRAM_HOME_CHANNEL_NAME=${TELEGRAM_HOME_CHANNEL_NAME:-}
NOTION_TOKEN=${NOTION_TOKEN:-}
HERMES_HUMAN_DELAY_MODE=${HERMES_HUMAN_DELAY_MODE:-natural}
HERMES_ACCEPT_HOOKS=${HERMES_ACCEPT_HOOKS:-1}
NOUS_API_KEY=${NOUS_API_KEY:-}
EOF
chmod 600 "$HERMES_HOME/.env"

echo "Hermes config ready. Starting gateway..."
exec hermes gateway run

hermes/cli-config.yaml

model:
  default: "anthropic/claude-sonnet-4-6"
  provider: "anthropic"

fallback_providers:
  - provider: "nous-api"
    model: "hermes-4-405B"
  - provider: "nous-api"
    model: "hermes-4-70B"

# Note: model names vary by Nous Portal plan. Verify yours at portal.nousresearch.com
# before deploying — a wrong model name causes silent fallback failures.

terminal:
  backend: "local"
  timeout: 180

memory:
  memory_enabled: true
  user_profile_enabled: true
  memory_char_limit: 4000
  user_char_limit: 2000
  nudge_interval: 10
  flush_min_turns: 6

session_reset:
  mode: both
  idle_minutes: 1440
  at_hour: 4

group_sessions_per_user: true

agent:
  max_turns: 60
  reasoning_effort: "medium"
  api_max_retries: 1

tool_loop_guardrails:
  warnings_enabled: true
  hard_stop_enabled: true

platform_toolsets:
  cli: [hermes-cli]
  telegram: [hermes-telegram, session_search]

# IMPORTANT: top-level key is mcp_servers, NOT mcp.servers
mcp_servers:
  notion:
    command: "npx"
    args: ["-y", "@notionhq/notion-mcp-server"]
    env:
      NOTION_TOKEN: "${NOTION_TOKEN}"
  # Add HTTP MCP servers the same way — secrets go in Railway vars and .env
  # my-http-tool:
  # url: "https://your-mcp-endpoint.com/mcp"
  # headers:
  # Authorization: "Bearer ${MY_MCP_SECRET}"

railway.json

{
  "$schema": "https://railway.app/railway.schema.json",
  "build": {
    "builder": "DOCKERFILE"
  },
  "deploy": {
    "restartPolicyType": "ON_FAILURE",
    "restartPolicyMaxRetries": 10
  }
}

No startCommand here intentionally. Railway's startCommand overrides Docker's ENTRYPOINT—which means tini never runs if you add one. Leave it out and Railway uses the ENTRYPOINT from the Dockerfile.

Railway setup walkthrough

# 1. Initialize Railway project
railway init

# 2. Initial deploy to create the service
railway up --detach

# 3. Link to the service
railway service "your-service-name"

# 4. Set environment variables
railway variables set ANTHROPIC_API_KEY=sk-ant-...
railway variables set TELEGRAM_BOT_TOKEN=...
railway variables set TELEGRAM_ALLOWED_USERS=123456789
railway variables set TELEGRAM_HOME_CHANNEL=123456789
railway variables set TELEGRAM_HOME_CHANNEL_NAME="Your Name DM"
railway variables set HERMES_HUMAN_DELAY_MODE=natural
railway variables set HERMES_ACCEPT_HOOKS=1
railway variables set NOUS_API_KEY=...
railway variables set NOTION_TOKEN=...

# Optional: webhook mode is more efficient than polling for hosted deployments
# railway variables set TELEGRAM_WEBHOOK_URL=https://your-service.up.railway.app/telegram
# railway variables set TELEGRAM_WEBHOOK_SECRET=$(openssl rand -hex 32)

# 5. Add persistent volume — MUST be /root/.hermes, not a subdirectory
railway volume add --mount-path /root/.hermes

# 6. Redeploy to pick up the volume
railway up --detach

Getting your Telegram IDs:

Create a bot via @BotFather → TELEGRAM_BOT_TOKEN
Message @userinfobot → your user ID for TELEGRAM_ALLOWED_USERS and TELEGRAM_HOME_CHANNEL
For a personal DM setup, both values are the same

Auto-deploy on push:

# .git/hooks/post-push
#!/bin/bash
railway up --detach


chmod +x .git/hooks/post-push

What I'd do differently

Start with the volume at the right path from day one. The /root/.hermes vs /root/.hermes/memories mistake wasted the most time because the symptom—bot asking to set home channel—looked like a configuration problem, not a persistence problem.

Verify MCP tools are loading before assuming they work. Ask Hermes directly: "What MCP tools do you have available?" If it lists nothing, you've got a config key problem or a secrets problem. Don't assume silence means they're loading quietly.

Set TELEGRAM_HOME_CHANNEL before your first deploy. The silent-bot symptom with no logs is deeply confusing. Just have this set from the start.

Quick reference: the 10 gotchas

#	Problem	Fix
1	`gateway start` fails in Docker	Use `hermes gateway run`
2	Bot resets on every deploy	Mount volume at `/root/.hermes`, not a subdirectory
3	MCP servers silently ignored	Config key is `mcp_servers:`, not `mcp.servers:`
4	Gateway ignores your config	Copy config to both `cli-config.yaml` AND `config.yaml`
5	MCP secrets not available	Write all secrets to `~/.hermes/.env` in entrypoint
6	Bot starts but never responds	Set `TELEGRAM_HOME_CHANNEL` env var
7	Zombie subprocess accumulation	Install and use `tini` as PID 1
8	MCP stdio server cold-start delay	Pre-install packages in Dockerfile
9	Slow LLM failover	Set `api_max_retries: 1` when using fallback providers
10	Will volume hide the Hermes install?	No—install is at system path, volume is for data only

How to Measure Time-to-Value for Your API (Without a Data Team or Custom Analytics Build)

Tessa Kriesel — Wed, 11 Mar 2026 19:37:03 +0000

You don't know your TTV. I'm willing to bet on it.

Not because you're not tracking things. You probably have a dashboard full of signups, API calls, session lengths, and something labeled "activation rate" that nobody has ever cleanly defined. You're tracking plenty. You're just not tracking the right thing.

Here's why that matters: developers who don't reach value fast leave fast. And you won't know they were ever close until they're already gone.

The Problem With Activity Metrics

Most API founders measure activity and call it progress. Sound familiar?

You end up tracking things like time to first API call, number of calls in the first session, or completion of an onboarding checklist. These metrics tell you a developer showed up and touched your product. They don't tell you whether that developer got what they came for.

Here's where it breaks down. The developer who made 3 API calls and churned looks identical in your data to the developer who made 3 API calls and became a power user. Until you look at whether they hit your Value Moment.

You're measuring motion, not progress. That's the real problem.

What Is a Value Moment?

Your Value Moment is the specific interaction where a developer gets undeniable proof that your API does what they need it to do.

Not "they called the endpoint." Not "they saw a response." The moment they got a result that made them think: okay, this actually works for my use case.

It looks different for every product:

For a payments API, it's the first successful test transaction in their dev environment
For a speech-to-text API, it's the first transcript that returns with over 95% accuracy on their test audio
For an AI completions API, it's the first response that matches their expected output format

The Value Moment is not generic. It's specific to what your product actually does. And it's almost always one concrete event, not a sequence.

Your TTV is the gap between signup and that moment. That's it.

The Value Moment Method: Define It, Track It, Fix It

You can define your Value Moment and start measuring TTV in an afternoon. Here's exactly how.

Step 1: Define Your Value Moment

Pull up data on your best-retained developers, the ones who've been active 60+ days. What did they do in the first 24 hours that churned developers almost never did?

You're looking for one identifiable event. If you can't access the data yet, interview three to five developers who successfully integrated your API. Ask them: "What was the moment you knew this was going to work?" Their answers will cluster around one or two specific interactions.

That cluster is your Value Moment. Write it down in plain language: "A developer successfully [specific action] with our API." That sentence is the foundation of everything else.

Step 2: Identify the Signal

Your Value Moment needs to map to something your system actually logs: an event, a response code, a specific API call.

For most APIs this is straightforward. If your Value Moment is "first successful test transaction," your signal is probably transaction.success with a test-mode flag. If it's "first accurate transcript returned," your signal might be a response with a confidence score above your threshold.

If your system doesn't log the signal you need, that's a one-line tracking addition, not an analytics overhaul. Add the event, tag it as your Value Moment signal, move on.

Step 3: Measure the Gap

TTV = time from signup to first Value Moment signal.

That's the whole formula. You're not averaging session lengths or weighting engagement scores. You're measuring one gap: how long does it take a developer to get their first undeniable proof your API works?

Track it for every new developer who signs up. Over time you'll have a distribution. What's the median? What's the 75th percentile? And the number that really matters: what percentage of developers hit the Value Moment at all within their first 7 days?

That last number explains your retention curve better than anything else in your dashboard.

What to Do With Your TTV Number

Your TTV number tells you two things immediately.

First, whether your onboarding is actually working. If the median TTV for developers who stay long-term is 18 minutes and your current median is 4 hours, that gap is your onboarding problem. Not your product. Your onboarding.

Second, where your intervention window is. Developers who don't hit the Value Moment in their first session almost never come back. That's not a judgment call, it's a pattern. Your job is to get them there before that window closes.

From here you optimize by working backward from the Value Moment. What's the friction between signup and that first signal? Where do developers drop off before they get there? Those are the specific things to fix. Not "improve the docs" or "add more examples" in the abstract, but the precise steps between where developers start and where they need to get.

See Your TTV Before It Shows Up in Your Churn Report

Here's the bottom line: defining your Value Moment takes an afternoon. Watching TTV data accumulate manually, correlating it against retention, turning it into something you can actually act on? That takes a lot longer.

That's what Built for Devs handles for you. Sign up, define your Value Moment, and as new developers flow through your onboarding, you'll see your TTV in your dashboard automatically. No custom analytics pipeline required.

You don't need a data team to start. You need to define the right moment, then let the tool do the tracking.

Start measuring your developer TTV →

View of Built for Devs Dashboard

Stop asking developers what they think. Watch what they do.

Tessa Kriesel — Tue, 10 Mar 2026 17:55:12 +0000

Ask a developer how your onboarding went and they'll probably say it was fine. Maybe a little long. Could use better docs. You take notes, you nod, you move on.

What they won't tell you: they almost quit twice, they couldn't find the API reference without Googling, and the moment they hit your pricing page they had to go back and re-read the docs before they trusted you enough to continue.

That version of events lives in their behavior. Not in their answers.

The problem isn't your feedback. It's what feedback can't capture.

You're getting feedback. You're talking to users. You're doing the things the startup playbook says to do. But the signal you're collecting is optimized for your comfort, not your understanding.

When a developer answers "was that confusing?", they don't replay the experience. They construct a memory of it—filtered through what they think you want to hear, how competent they want to appear, and what they've already forgotten. They won't tell you your onboarding took 47 minutes and they almost quit three times. They'll say it was "a little long, but fine." They self-censor because they've moved on and don't remember the exact sticking points.

That's not dishonesty. That's just how memory works.

The enemy isn't your users. It's the method you're using to understand them.

Behavior tells you what answers can't

Your existing users have context. They've learned your product, forgiven its rough edges, built workarounds. They're not starting from scratch—they're showing you what a loyalist does, not what a new developer does. Your network is worse. They want to be helpful. They'll push through friction that would make a stranger quit.

What you need is a developer who actually matches the profile of someone you're trying to win—not a friend, not a power user, not someone already in your orbit. The friction that matters is what happens before they care about you. Before they've decided to give you the benefit of the doubt.

What you see when that developer tries your product for the first time will not match what your feedback sessions told you.

You'll see where they actually get stuck—not where they said they got stuck. You'll see the moment they almost quit. You'll see the exact point where your product either earns their trust or loses it. And you'll see things you never thought to ask about, because you didn't know they were problems.

One founder we worked with discovered this the hard way. Their onboarding requested full Google Drive permissions when all they actually needed was access to a single spreadsheet. Feedback sessions didn't surface it. But when they started watching real developers go through the flow—and listening to what those developers said out loud—they found something no form had captured: developers were genuinely annoyed. Many said they would never grant those permissions. Nobody had written that in a feedback session because nobody asks "how do you feel about our OAuth scope?"

Most founders who go from asking to watching describe the same experience: they thought they knew their onboarding flow. They'd walked through it dozens of times. Then they watched a real developer do it and saw three failure points they'd never once noticed—because they already knew where everything was.

You know your product too well to use it naively. A developer who's never seen it before doesn't. That's exactly why their experience is the one worth watching.

The cost of collecting the wrong signal

With limited runway, every product bet has a price. The features you ship in the next two months either move retention or they don't. If those decisions are grounded in self-reported feedback, you're making expensive guesses dressed up as research.

There's a specific kind of founder regret that comes from shipping a big feature, watching it go unused, and then finding out through a casual conversation that the feature developers actually needed was something much simpler. Something they never mentioned in a feedback session because they didn't think you'd want to hear it, or they'd already found a workaround and stopped thinking about it.

That regret is avoidable. Not by doing more research. By doing a different kind.

The signal you can't collect any other way

The only way to get honest signal is to watch a developer who's never seen your product try to use it—without context, without guidance, without anyone from your team in the room. No prompts. No hand-holding. Just a real developer and your product.

The problem is finding them. Not just any developer—one who actually looks like the developer you're trying to win. Your network is limited. Your existing users aren't starting from zero. There's no obvious place to find the right people with zero prior exposure—so most founders never get there at all.

That's the problem Built for Devs solves. We match real developers who fit the profile of your target user, bring them to your product for the first time, and capture everything—every click, every hesitation, every moment they say "this makes no sense" out loud. That data powers a findings report that tells you precisely where the experience breaks and what to fix first. Not a list of opinions. A prioritized action plan grounded in real behavior.

That discomfort of watching someone struggle through something you built—that's the data you've been missing.

Why Developers Aren't Coming Back After Their First Session

Tessa Kriesel — Mon, 09 Mar 2026 14:50:57 +0000

You've fixed the tutorial. You've cleaned up the docs. You've killed the bugs early users reported. Developers still aren't coming back.

If this is your situation, you're probably running the same troubleshooting loop most founders run: something about the product isn't clicking. So you improve the product. You add a better onboarding flow. You write a getting-started guide. You watch the metrics. Nothing changes.

Here's what I keep seeing across dev tool companies stuck in this loop: the diagnosis is wrong. First-session churn usually isn't a product problem. It's a stage problem. And if you're measuring product metrics to fix a stage problem, you're going to keep shipping features that don't move retention.

The Onboarding Loop That Goes Nowhere

Most founders respond to first-session churn by improving onboarding. This makes sense on the surface—developers aren't coming back, so something about their first experience isn't working, so you fix the first experience.

The problem is that "first experience" and "first session" aren't the same thing. The onboarding flow is one slice of the first session. But a developer's first session spans a journey that started before they ever opened your product—and continues through stages that have nothing to do with your tutorial.

When you only measure what happens inside your product during session one, you're measuring a fraction of what actually determines whether someone comes back. You can optimize that fraction indefinitely and never touch the part that matters.

I call this stage blindness: diagnosing developer churn with product metrics when the drop-off is happening at the journey level.

Where Developers Actually Exit

There's a framework I use to think about how developers move through any tool's ecosystem. Five stages: Discover, Evaluate, Learn, Build, Scale.

Most first-session exits happen in Evaluate and Learn. Not Build.

Discover is how they found you—search, a recommendation, a conference talk. If they're in your product, they've cleared Discover.

Evaluate is the decision stage. Before a developer commits to learning anything, they're asking a question you probably can't see on your dashboard: is this actually for me? They're scanning docs, reading changelog entries, checking the GitHub repo, looking at the community, reviewing pricing. They're trying to answer the question fast, before they invest real time. If your product doesn't give them a fast, clear answer, they're gone—and they look like a session-one exit, but the real exit happened during Evaluate.

Learn is where developers start building skills, not building product. They're running the quickstart. They're working through examples. They're hitting friction and deciding whether the friction is temporary (learning curve) or permanent (bad product fit). If the gap between your quickstart and real-world usage is too big, or if the docs assume knowledge they don't have, they exit here. Again, looks like session-one churn. Actually a stage failure.

Build is where most onboarding optimization is aimed. More tutorials, better error messages, cleaner UI. The problem: if developers are exiting in Evaluate and Learn, they never make it to Build. Improving Build doesn't help.

Scale is retention at depth—developers who've integrated your tool and are now spreading it to their team or expanding usage. You don't have a Scale problem if you're seeing first-session exits. But you won't get there without clearing the earlier stages.

What Stage Blindness Costs You

When you can't see which stage developers are exiting, you default to shipping. More features, more onboarding improvements, more docs. I've watched companies spend a quarter iterating on their Build experience when their real problem was an Evaluate stage that left developers unsure whether the tool was even meant for them.

The compounding cost: every iteration cycle you spend in the wrong stage is a cycle you're not spending on the right one. And with 4-6 months of runway, you don't have many cycles left.

The other cost is signal distortion. When you're measuring product engagement during session one and attributing poor retention to product quality, you're training yourself to look in the wrong place. The feedback loop reinforces the wrong diagnosis.

How to Diagnose Which Stage Is Failing

You can't improve what you can't see. The starting point is mapping your developer journey with enough specificity to identify where the drop-off actually happens.

Start with Evaluate. What does a developer experience between finding your product and deciding to invest time in it? What questions are they trying to answer? How fast can they answer them? If you don't know, your Evaluate stage might be invisible to you—which is exactly the problem.

Then look at Learn. What's the gap between your quickstart and a real use case? Where does that first "wait, this doesn't work the way I expected" moment happen? That's usually where Learn exits occur.

If you want a fast read on where your developer journey is failing, take the Developer Adoption Score at builtfor.dev/score. Free, about 5 minutes, and it gives you an actual score on your journey—not a generic 'improve your onboarding' recommendation. Want to go deeper? Our product offering tracks real developer behavior across every stage so you can see exactly where the drop-off happens.

Developer Journey Tracking in the Pro Plan

The Real Question to Ask

The next time a developer doesn't come back after session one, the question isn't "what was wrong with their first experience?" The question is "which stage failed them?"

Those are different questions with different answers. And until you can ask the right one, the onboarding loop keeps going.

You've already tried fixing the product. That's not what's broken.

The 3 Signals That Tell You Whether to Rebuild or Improve Your Dev Tool Onboarding

Tessa Kriesel — Mon, 09 Mar 2026 04:57:13 +0000

Most teams are having the wrong argument. The debate isn't "should we rebuild"—it's whether you've looked at the right data first.

I've pulled more than a few product teams out of this exact trap—one that burned a quarter rebuilding their onboarding to see activation barely move, another that iterated for six months on a flow with a structural problem from the start, adding tooltips and reorganizing steps while the actual issue sat untouched two layers deeper.

Here's what both teams had in common: they made the rebuild vs. improve call based on instinct and internal pressure, not signal.

That's fixable.

The real problem with this decision

When activation flatlines, the conversation usually goes one of two ways.

The first: "We've been incrementally improving this for months. It's time to rebuild." This feels decisive. But it's often a reaction to frustration, not evidence. Rebuilding satisfies the internal need to feel like you're taking bold action. It doesn't automatically address the root cause.

The second: "A full rebuild is too expensive right now. Let's keep iterating." This feels prudent. But if the current flow has a structural problem—a wrong assumption about what developers need to experience first, or a sequence that doesn't match how they actually think—iterating around it will only produce marginal gains. You're polishing something that needs to be torn down.

Neither instinct is wrong, exactly. They just aren't grounded in what the data can actually tell you.

There are three signals that cut through the argument. I've seen them hold up across enough developer products to trust them. And the time it takes to check all three is probably less than your next sprint planning meeting.

Signal 1: Where in the journey is the drop-off happening?

This one requires you to be honest about what "activation" actually means for your product. Not sign-up. Not email confirmation. The moment a developer has experienced enough of your product to understand whether it solves their problem.

Now look at your drop-off data. Where are developers leaving?

If developers are dropping off before they've had the chance to reach that value moment—before the first API call, before the first integration runs, before they've gotten any output at all—that's usually a rebuild signal. The problem isn't that your steps are confusing or your copy is unclear. The problem is that the path to value is too long, or the framing at the front end is creating the wrong expectations, or developers aren't self-selecting correctly before they start.

The instinct in this situation is to improve the steps right before the drop-off. But drop-off location is telling you something different: it's showing you where the gap between developer expectation and product reality became too wide to cross. If that gap opens before developers have experienced core value at all, improving the surrounding steps won't close it. The architecture is wrong—the flow is asking developers to commit before they understand what they're committing to.

That's not an iteration problem. That's a structural one.

If your drop-off is concentrated in the middle or later stages—after developers have had initial contact with the core value, but before they've completed the full flow—that's more often an improvement opportunity. The structure is right. There's friction you haven't removed yet.

The signal: Drop-off concentrated before value exposure means rebuild. Drop-off concentrated after value exposure means improve.

Signal 2: Does completing onboarding predict activation?

This one surprises teams every time they run it. Pull two cohorts: developers who completed your onboarding flow, and developers who skipped it and went straight to your docs. Compare activation rates.

If developers who completed onboarding activate at significantly higher rates than those who skipped it, your flow is doing something useful. The signal-to-noise ratio is there. You have a structure worth improving.

If the difference is small—or if developers who skipped onboarding activate at similar or better rates—that's a different story. It means your onboarding isn't providing something developers couldn't find themselves. They're working around it. That's a structural indictment, not an execution one.

When developers route around your designed path and still succeed, it usually means the flow is optimized around internal assumptions about what developers need—not around what they actually do to reach value. The flow reflects how your team thinks about the product, not how developers experience it. You can improve that flow indefinitely. You're improving the wrong thing.

The variation worth watching: if high-ICP developers skip onboarding and succeed, but lower-ICP developers complete it and still don't activate, your onboarding might be serving the wrong segment. That's a rebuild with a different diagnosis—you're building onboarding for developers who don't need it, instead of for the ones who do.

The signal: Completion predicts activation means improve. Completion and activation are uncorrelated means rebuild.

Signal 3: Has friction migrated without completion rates improving?

This is the one that tells you whether you've been iterating in circles.

Pull the last three iterations of your onboarding. For each one: where was the primary drop-off point, and what was the overall completion rate?

If the friction point has moved between iterations—step 4 became step 5 became step 7—but the completion rate hasn't meaningfully improved, you've been relocating friction, not removing it. Developers are hitting the same resistance; it's just showing up in a different place now.

This is the clearest rebuild signal in the data. It tells you there's a load-bearing structural issue that iteration hasn't touched. The friction isn't in the steps. It's in the assumption the steps are built on.

When the problem location moves but completion doesn't improve, you haven't solved anything—you've relocated the resistance. Eventually teams stop noticing that the friction point keeps shifting because each iteration feels like progress. It's not. You're rebuilding the same wall in a different spot.

If friction is migrating but completion is improving—say, from 30% to 45% to 55%—that's a healthy iteration signal. Keep going. The structure is sound and the changes are working.

The signal: Friction migrates, completion stays flat means rebuild. Friction migrates, completion improves means improve.

How the three signals combine

You rarely need all three to point the same direction to get clarity. Here's how I use them:

Two or more signals point to rebuild: Stop iterating. You're spending engineering time on the wrong problem. Rebuild means rethinking the journey architecture—what the flow is trying to accomplish, in what order, for which developer. Not just a new UI.

One signal points to rebuild, others are inconclusive: Dig deeper before committing. One anomaly is worth investigating; it might be segment-specific or explained by another variable. But don't dismiss it.

All signals point to improve: Good news—you have a structural foundation worth building on. The work ahead is meaningful friction removal, clearer copy, better error handling, tighter time-to-value within the existing architecture.

No signals point to rebuild and activation is still flat: Look upstream. The problem might not be in your onboarding at all. It might be in how developers arrive—the expectations your marketing and content are creating before they sign up. Fixing the flow won't fix a positioning mismatch.

One more thing before you start

The instinct call usually costs you a quarter. Occasionally it costs two. The three signals cost an afternoon with your analytics.

That's worth doing before you walk into the sprint planning meeting and announce a rebuild that might be the wrong call—or commit to another round of improvements on a flow that needs to be torn down.

If you want to know where your developer journey is actually breaking, the Developer Adoption Score is a free tool that evaluates your current experience across all five stages. It takes about ten minutes and gives you a starting point for this kind of diagnosis.

The debate teams have about rebuild vs. improve is usually the right debate. They're just having it without the data that would make it productive.