Giving an AI agent memory of our Twist threads: building a Twist connector for cognee

Twist is built for long, slow conversations. Someone opens a thread with a proposal, people reply over two or three days, and somewhere around the fifth comment the actual decision gets made. That is great for a team, but it is terrible if you want to ask "what did we decide about X?" later. Search finds the thread. It does not find the answer.

So for Mergetober I built a Twist connector for cognee, an open source memory layer for AI agents. You point it at a Twist workspace, it pulls threads and channels into cognee, and then you can ask questions in plain language. It is PR #364 in cognee-community, for issue #4736.

This post is how I use it, how it works inside, and the bugs a reviewer found that I would never have found on my own.

How I actually use it

To try this properly I set up a small product launch the way a lot of small teams actually run one: the discussion in Twist, the tasks in Basecamp, and the announcement email in Mailchimp. I wrote connectors for all three during Mergetober (the Basecamp and Mailchimp posts are separate), and the point is to sync them into one cognee setup and ask about the launch without opening three apps.

In Twist I have a #launch channel with a thread called "Checkout redesign". The opening post says "Final call: option B ships first, on 15 March." and the comments confirm it. After a sync, this is the document cognee receives for that thread. This is real output from my workspace:

Twist thread: Checkout redesign
Channel: #launch in Anand Test
Started by Anand Prakash Srivastava on 2026-10-10

Final call: option B ships first, on 15 March.

Anand Prakash Srivastava on 2026-10-10:
_Final call: option B ships first. Confirmed by QA and design._

Anand Prakash Srivastava on 2026-10-11:
_Ship date confirmed._

Then I ask:

import json, os
import cognee
from cognee_community_connector_twist import twist_source

token = json.load(open(os.path.expanduser("~/.twist_tokens.json")))["access_token"]

await cognee.remember(
    twist_source("254427", access_token=token, channel_ids=[852087]),
    dataset_name="twist",
    write_disposition="merge",
)
print(await cognee.recall("What did the team decide about the checkout redesign?", datasets=["twist"]))

These are the results I got on my workspace. I ran this one without an LLM key, in cognee's keyless mode, so recall returns the best matching passage instead of a written answer:

Step	Question	Top result
Sync #launch and #Twist	What did the team decide about the checkout redesign?	"Checkout redesign": "Final call: option B ships first, on 15 March."
	What to do next in Twist?	the "What to do next" thread
Delete "What to do next" in Twist, sync again	What to do next in Twist?	"Get started in 4 steps". The deleted thread is not returned any more

That last row matters to me more than the first two. If memory keeps things you deleted, you stop trusting it.

The design in one picture

A few decisions shaped everything else.

One document per thread, not per comment. My first idea was to send each comment as its own document. But the decision is usually in comment five, and comment five only makes sense next to the opening post. So a thread becomes one document: title, channel, opening post, then every comment in order with author and date. Each channel also gets a small document of its own. Every row is tagged with a node set like twist:<workspace_id>:<channel_id>, so you can scope recall to one channel later.

It goes through dlt, with merge. cognee accepts dlt sources directly. The connector is a dlt source that yields rows keyed by thread:<id>. When a thread disappears, it yields {"id": "thread:123", "_deleted": True} and dlt removes the row. cognee's orphan cleanup then removes the document and its part of the graph. One thing caught me early: remember() defaults to write_disposition="replace", which quietly breaks both incremental sync and deletes. You have to pass "merge".

Figuring out what "changed" means in Twist

This took longer than writing the code. Twist's docs give you last_updated_ts and comment_count on a thread, and my plan on the issue used those two. Then I tested it against a real workspace, one change at a time, and wrote down what moved:

What I did in Twist	version	last_updated_ts	Anything on the thread?
New comment	moves	moves	yes
Edit the opening post	moves	no	yes
Delete a comment	moves	no	yes
Edit the latest comment	moves	no	yes
Edit an earlier comment	no	no	nothing at all
Bookmark the thread	moves	no	yes, but the text did not change

So last_updated_ts misses most edits. There is a version field that catches nearly everything, but it is not in the documented thread object. It is in the API response though, so the connector compares four values per thread:

signal = [
    thread.get("version"),
    thread.get("last_updated_ts"),
    thread.get("comment_count"),
    thread.get("channel_id"),
]
if not sweep and old and old[:4] == signal:
    stats["unchanged"] += 1
    continue

Bookmarking moves version too, even though nothing in the text changed. To avoid sending that to cognee, each rendered row gets a SHA-256 hash, and if the hash matches last run, the row is skipped.

The earlier comment edit is the annoying one. Nothing on the thread moves, so no cheap check can see it. The connector re-reads every thread's comments on the first run and every 10th run after that (full_sync_every), or when you pass full_sync=True. It is not instant, and the README says so.

Here is what two runs back to back look like on my workspace today. First run sends everything, second run sends nothing:

When I was testing, I went through the cases one by one on the live workspace:

Run	Change in Twist	Result
1	First sync	15 threads + 4 channels, the archived channel skipped
2	Nothing	0 changed, 15 unchanged
3	New comment on one thread, another thread deleted	1 changed, 1 deleted, 13 unchanged
4	Opening post edited	1 changed (caught by version)
5	Thread bookmarked	0 changed (content hash)
6	Latest comment edited	1 changed
7	Earlier comment edited	normal run: 0 changed. full_sync=True: 1 changed

Some other things the live API showed me that are not obvious from the docs:

• A deleted comment does not disappear. It stays in comments/get with deleted: true and the content "-". If you do not filter it, your memory gets a line that says "-".
• Renaming, moving or closing a thread adds a system message as a comment. Those are skipped too.
• A thread's is_archived is not an archive. It is your own Inbox "done" state. I almost used it to drop threads.
• comments/get without a range returns the last N comments, not the first. So comments are read in explicit from_obj_index / to_obj_index windows of 500.
• A rejected token comes back as HTTP 403 with error code 200, not 401.

The part I underestimated: not deleting things by mistake

Forget-on-delete sounds simple. A thread missing from the listing gets a _deleted row. But "missing from the listing" is also exactly what a wrong channel id, a token that lost access, or a half-broken API response looks like. One bad run and the whole dataset is gone.

So there are a few rules:

• A 404 is never a delete on its own. A deleted thread answers 404, but Twist uses the same error for something the token can no longer see.
• Any API error stops the run before a single tombstone is written. dlt only saves state when the load succeeds, so the next run just retries.
• If a run finds more than half of what it synced newly gone, and at least 10 items, it holds those back and reports them as held_back. The next run forgets the ones that are still gone and keeps the ones that came back.

The core of it is short:

gone = sorted(set(previous) - set(present))
pending = set(state.get("pending_delete") or [])
confirmed = [key for key in gone if key in pending]
fresh = [key for key in gone if key not in pending]
large = len(fresh) >= MASS_DELETE_MIN_ITEMS and len(fresh) > max_delete_ratio * len(previous)
if large and not allow_mass_delete:
    present.update({key: previous[key] for key in fresh})
    state["pending_delete"] = fresh
    stats["held_back"] = len(fresh)
    gone = confirmed

My first version raised an error instead of holding. The reviewer pointed out that a scheduled sync would then stop forever until someone noticed. Holding for one run fixes that without a manual step.

What code review taught me

goran-radonic reviewed this PR in three rounds. Every round found something real, and most of it was stuff my tests could not have caught because I did not think of the case. A few I remember:

• Rolling deletes were held forever. My hold-back only confirmed a batch if the next run found exactly the same items gone. If you deleted a few more threads before the next run, the set changed and the whole batch was held again. Forever. The fix is the confirmed / fresh split above.
• An archived channel could be lost. If you ran once with a wrong channel list, the mark that said "this archived channel was synced while active" got dropped, and the next normal run forgot all its threads. He wrote out the scenario run by run, and I turned it into a test that went through real dlt and SQLite. Before the fix it deleted 13 rows. After, 0.
• Retry-After edge cases. I already stopped the run on any wait over 60 seconds, because cognee holds its staging lock while a source runs. But Retry-After: 0 made the client retry back to back with no wait, nan slipped through, and inf was not treated like a long wait. Now 0, negative and nan fall back to exponential backoff, and inf stops the run with TwistRateLimitError like any other long wait.
• Stats from the previous run leaked. The counters reset in the wrong place, so a run that failed early reported the last run's numbers.

For the last round I did something I will keep doing: I reverted each of the 9 changes on its own and checked that at least one test went red. If a test still passes with the fix removed, it is not testing the fix.

Testing

The tests run against an in-memory fake of the Twist API that copies the behaviour I saw live: version bumps, deleted comments with "-", system messages, archived channels, 404s, comment windows. No token needed.

There is also one test that goes through cognee itself (add + cognify with the LLM mocked) and checks that a deleted thread actually leaves the knowledge graph while the others keep their ids. It goes red if you turn off either the tombstone or dlt's hard_delete hint.

Limits I am honest about

• Every run lists all threads of the selected channels. That is one request per 500 threads, plus one per changed thread.
• An edit to an earlier comment shows up only on the next sweep.
• On Twist's free plan, the API hides comments older than one month. When an old thread is re-rendered, those comments drop out of memory too.
• Images are dropped. Attachments are listed by name and link but not downloaded.
• Direct messages are never read. That one is on purpose.

Try it

The connector lives in packages/connector/twist/ in cognee-community. The README walks through creating a Twist integration and getting a token with read-only scopes (workspaces:read,channels:read,threads:read,comments:read), and examples/example.py runs the whole thing.

If you use Twist and try it, I would like to know which questions you end up asking it. That tells me more than any test.