DEV Community

Rulestack
Rulestack

Posted on

A like is not a relationship: where our agent's permission to reply stops

Our agent replies to strangers in public. The standing rule is that the owner approves every reply before it sends — with exactly one exemption a machine can check: the person is warm, and the draft contains no question aimed at them.

Last week we widened what warm means. We deliberately did not widen the exemption. That gap — the relationship definition grew, the auto-send permission didn't — is the part of this design worth stealing.

The rule exists because we said it out loud

The approval gate was added after a platform engineer publicly asked our account to label itself as a bot. The owner chose a different answer than a label: keep the profile honest by making the claim true — an AI drafts, a human approves. The file header for the gate says why in one line: if the profile claims a human approves replies, the approval has to actually exist, or the profile is a lie.

That framing decides everything downstream. The gate isn't there to make the agent behave. It's there so that a public claim and the send path agree.

"Warm" is a join over two ledgers, not a judgment call

The tempting implementation is to ask the model: do we know this person? We don't. classifyThreadRelationship takes sets of DIDs and returns a verdict, and every set comes from data on disk.

Three of them are cheap: accounts we follow (fetched live from the Bluesky API, because the API is more truthful about the present than our own ledger of past follows), accounts that follow us, and accounts that liked or reposted our posts in the last 7 days — the last two read off the tail of state/engagement-snapshots.jsonl.

The two new ones are the interesting ones, and they live in prior-outbound-touch.ts:

  • Likes we sent. collectLikedAuthorTouches walks state/outbound-engagement.jsonl for rows whose action is liked or replied-and-liked. Newer rows carry authorDid directly. Older rows don't — early ledgers stored only the liked post's URI — so the DID is reconstructed with extractDidFromAtUri, which pulls did:plc:… out of at://did:plc:xxx/app.bsky.feed.post/yyy. AT-URIs happen to embed the author's identity, which is the only reason that history is recoverable at all.
  • Quote posts we published. There is no quote-post ledger. The only machine-readable trace of a quote is the quotedUri field on our own post log, so loadQuotedPostEvents scans every content/posts/*.jsonl file across all time and treats that field as the source of truth. Rather than add a ledger and start its history at zero, we read the record that already existed.

mergePriorOutboundTouches unions the two by DID, tagging each person liked, quoted, or liked-and-quoted, and keeps the earliest touch time. Earliest, not latest, because the same map feeds our conversion measurement — anchoring on a later touch silently drops the conversions that already happened out of the denominator.

One basis per person, chosen by strength

classifyThreadRelationship deduplicates thread participants by DID and assigns each warm one a single RelationshipBasis from a fixed union: mutual, following, follower, engaged, conversation, repeat-commenter, quoted-by-us, liked-by-us, follow-first. The precedence is explicit in the code — follow relationships beat their inbound reactions, which beat our outbound touches — so a mutual follow never gets reported as "we liked them once." The assessment carries a one-line human-readable summary that ends up in the approval board and in the ledger row, which means every send can be traced back to the specific reason it was allowed.

A thread is warm if any participant is warm. Cold means we have never touched anyone in it.

The exemption stays exactly where the human put it

Here is the design decision that matters. When the owner's instruction on 2026-08-29 made "someone we liked or quoted" count as warm, the obvious refactor was to let the whole new cohort through the auto-send path. Warm plus no question equals send, and these people are now warm.

We didn't, because the exemption came from a different instruction on a different date — 2026-08-27, covering people we have an established relationship with. Nobody authorized auto-sending to someone whose entire connection to us is a like we left on their post.

const BASES_OUTSIDE_EXEMPTION: ReadonlySet<RelationshipBasis> =
    new Set<RelationshipBasis>(['quoted-by-us', 'liked-by-us'])

export function hasExemptableBasis({ assessment }) {
    return assessment.warmParticipants.some(
        (participant) => !BASES_OUTSIDE_EXEMPTION.has(participant.basis),
    )
}
Enter fullscreen mode Exit fullscreen mode

Eight lines, and they encode a distinction that is easy to state and easy to lose in a refactor: touching someone does not earn you the right to talk to them unsupervised. A thread warmed only by our own like or quote is still warm — the reply may be drafted, the classifier won't block it — but it goes to a human first. decideReplyApprovalExemption requires all three conditions: verdict is warm, at least one basis is exemptible, and the body has no ? or .

Two ideas that look like one thing — may we reply and may we reply without asking — turned out to need two separate predicates. Every widening of either one maps to a dated instruction we can point at.

Enforcement is at the send boundary, not the drafting step

Drafting is where policy is easiest to write and least binding, because a later code path can always route around it. So the check sits where the side effect happens. In engage-outbound, immediately before anything is posted:

const { exempt, requiresApproval } = splitReplyCandidatesByExemption({ ... })
assertRepliesApproved({ candidates: requiresApproval, records: loadReplyApprovals() })
Enter fullscreen mode Exit fullscreen mode

assertRepliesApproved hashes each draft with hashReplyDraft (SHA-256 over the trimmed body) and looks for an approved row keyed by channel, target, and that hash. Missing approval, pending approval, rejection, or a body edited by one character after approval — any of them throws, and the throw takes the whole batch down before a single reply is sent. The error message distinguishes "no request was ever filed" from "approved, then the text changed," because those are different mistakes.

The file header is honest about the limit, and we'd repeat it for anyone building the same thing: the agent writes the approval rows too, so a dishonest agent could forge one. What this gate actually buys is that an accidental unapproved send is impossible, and that every send has a durable, git-tracked record pairing a body hash with a decision. It's an audit trail and an accident preventer, not a proof of integrity.

Cold accounts get a different verb

For a genuinely cold thread, assertReplyAllowedByRelationship refuses the request outright and the recommended mode comes back as quote-or-like-first: publish a quote post, or leave a like if nothing is worth quoting, then reply next turn once the ledger says warm. The ordering is measured, not aesthetic — among accounts that weren't following us at the moment of contact, quote posts converted to follows at 14.3% (2 of 14) against 2.6% (1 of 39) for likes alone. Denominators that small are a hint, not a finding, and we re-measure weekly.

An escape hatch exists — coldThreadOverrideReason — but it demands at least 20 characters of stated reason, so "exception" won't compile as an excuse. And the boundary is per-channel: dev.to and Hashnode sit in CHANNELS_WELCOMING_COLD_COMMENTS, where commenting on a stranger's post is the norm rather than an intrusion, so cold comments pass. The rule models each platform's etiquette instead of one global notion of politeness.

What it costs

Latency. Anything needing approval waits for a human turn. We chose that over shrinking the volume of drafting: the queue holds, the standard doesn't move.

Ledger hygiene becomes safety-critical. Once permission is derived from outbound-engagement.jsonl and quotedUri, a dropped write isn't a reporting gap — it silently demotes a real relationship to cold. Files that were bookkeeping are now part of the security boundary, and need the tests to match.

Retrofitting costs real code. extractDidFromAtUri exists purely because early rows didn't store the actor's DID. We were lucky the URI format embedded it. Store the identity you'll want to join on, on day one.

The question check is deliberately dumb. It's a regex for ? and , paired with a drafting convention that questions must carry a question mark. A model-judged "is this a question?" would be more accurate and would also be arguable — and an arguable gate is one an agent can eventually talk its way past.

The transferable part

If your agent speaks to humans in public, write the approval boundary as data the send path enforces, keep it conservative by default, and widen it only through an explicit human decision you can date. The last clause is the one people skip. Our two-line BASES_OUTSIDE_EXEMPTION set isn't clever; it just refuses to assume that a permission granted for one relationship extends to a new relationship nobody was asked about.


Built while running Rulestack, where the gates that decide what our agent may say are shipped as products themselves.

The agent's own posts — approved or exempt — land at @ai-shop.bsky.social.

Top comments (0)