After my BambooHR connector, I picked up a second Mergetober issue: a Customer.io connector for cognee. Last time the lesson was "read the API before writing code". This time it was "test your plan before writing code", because my plan was wrong in public, twice.
The issue
cognee turns your data into a knowledge graph that AI agents can search. Issue #4786 asked for a connector for Customer.io, a marketing messaging platform. The idea: if your team's emails, push notifications and in-app messages live in Customer.io, your agent should be able to answer "what does our French welcome email say?"
I posted a plan on the issue: merge + tombstones like BambooHR, with a doc id of "action id + language" so translations wouldn't overwrite each other.
Honestly, I didn't expect to get it. I'd asked a maintainer on Discord whether we could comment on several issues at once, since a lot of people were doing it. The answer was yes, but issues go to the best proposal, and people who already have an issue assigned are lower priority so everyone gets a chance. I already had BambooHR. A few other people had posted plans on this issue too.
So I didn't comment on every issue I could find. I only commented on two in total, and I took my time on both plans.
Yesterday I checked my mail as soon as I woke up: "Issue assigned to @abhishiv17". That felt really good. 😄
A trial workspace before any code
I signed up for a free Customer.io trial and built a small test automation: one email in Default, English and French, plus a newsletter. Then I wrote a throwaway script that:
- saved every API response as raw JSON (
snapshot NAME), - printed what appeared, disappeared or changed between two snapshots (
diff A B).
Keeping the raw responses turned out to matter. Fields I wasn't even looking at showed up in the diffs, and that's how most of the surprises below got caught.
A small tip if you try this: my trial account was "on hold" until verified. That only blocks sending. The App API worked fine.
Wrong #1: translations
My plan assumed one message comes back once per language. In reality, each language is its own action with its own id, linked by a multi_language_branch_action_id. The default language is "", not en. Adding "English" created a brand new action.
So the action id alone was already unique, and my "id + language" key wasn't needed. But then I found a collision I hadn't planned for: a newsletter's content and a campaign message were both id 5. They use separate counters. Doc ids got a type prefix: action:5, newsletter_content:5.
Wrong #2: deduplicate_id
My plan said deduplicate_id "looks like id + timestamp" and proposed using it to detect changes. Then action 5 came back as 4:1792011300. The prefix was the branch id, not its own id.
It got worse:
- Publishing an edit to one language bumped
updatedon every language, even ones I didn't touch. - Adding a push step to the campaign bumped
updatedon all the existing emails, with nothing else changed.
Neither field could tell me "this content changed".
The open question, and changing one thing at a time
The issue suggested using the campaign's updated timestamp as the sync cursor. I wasn't sure editing a message actually moves it, so I'd said I'd check.
My first test was useless: in the same publish I edited a body and added a language. The campaign's updated moved, but I couldn't tell which change did it. I reran with a body-only edit. Result: the campaign's updated doesn't move when you edit a message. It only tracks structure (steps added or removed).
Lesson: change one thing per test, or the test can't answer anything.
Changing the design (again)
I first posted a fix on the issue: hash the message fields and re-ingest only when the hash changes. Then, while coding, it clicked.
I had copied my own BambooHR design (merge + tombstones), but the reason it fit BambooHR was its change feed. Customer.io doesn't have one. And since editing a message doesn't move the campaign timestamp, I have to list every message on every run anyway.
So the connector became a full snapshot, like cognee's Notion connector:
- Every run lists everything, with
replace. Anything that disappeared upstream is forgotten by cognee's cleanup. - Rows hold only the message text, no
updatedand nodeduplicate_id. cognee already hashes each row, so an untouched message keeps the same hash and isn't processed again, even when Customer.io bumps its timestamp. - It works with cognee's default settings. No
write_disposition="merge"for users to forget.
The one rule this brings: if a listing fails, the run must abort. A partial snapshot would make cognee forget messages that still exist.
The half-built SMS that broke everything
Then that rule bit me. I added a push step and an SMS step but left them unfinished, since both needed a provider set up.
Suddenly the campaign's actions list returned 500, every time. But each message still loaded fine on its own by id, except the empty SMS. Deleting the SMS step made the list work again.
With "abort on a failed listing", one half-finished draft would have blocked syncing the whole workspace. So now, on a 5xx, the connector fetches that campaign's messages one by one, and skips only the one that keeps failing. If nothing at all loads, the run aborts.
Cleaning up the text
body_plain comes back empty, so the connector converts the HTML body to text and strips the head, style and script tags, plus the unsubscribe footer Customer.io adds to every email. Otherwise every document in the graph would end with "Unsubscribe" (or "Se désabonner").
A few more things the trial showed: unpublished edits never reach the API (good, only published content goes in), a deleted newsletter just disappears and returns 404, and webhooks are refused because their bodies can carry tokens.
Testing
- 36 unit tests with a mocked API that copies the real response shapes, including a real load into SQLite.
- I broke the code on purpose: the footer stripping, the 5xx fallback and the abort guard, one at a time. The tests caught each one. A test suite that passes on the first try proves nothing until you've seen it fail.
- I also ran my own code over my saved snapshots. That caught a wrong claim in my own notes about which text had been published.
- A live run against the trial made 3 API calls and staged exactly what I'd typed in the UI. The empty push was skipped, and there was no "Unsubscribe" anywhere.
| id | channel | language | subject | text |
|---|---|---|---|---|
action:2 |
default | hello | English version four | |
action:3 |
fr | bonjour | Version française une | |
action:5 |
en | hello | English version one | |
action:9 |
in-app | default | In-app body one |
Then end to end on local Ollama (llama3.1:8b + nomic-embed-text, CPU only): the first sync took about 141s, and the second sync with nothing changed took 7s, because nothing was re-processed.
Windows gotchas (again)
-
Git Bash rewrote my API path. Passing
/v1/...as an argument turned it intoC:/Program Files/Git/v1/.... PowerShell didn't do that. -
"Version française" in my output looked like an encoding bug in cognee. Printing the stored text with
ascii()showed\xe7, a correctç. It was just PowerShell decoding the output with the wrong code page. Check the bytes at each step before blaming the code. - And yes, I ran
git initin the wrong folder again. 😅
Using it
await cognee.remember(customerio_source(), dataset_name="customerio")
You need an App API key (not the Track API keys). Setup steps are in the package README.
What I learned
- Your plan is a guess until you test it. Correcting it in public with evidence is better than quietly changing the design in the PR.
- Don't copy a design without its reason. Merge fit BambooHR because of the change feed. No feed here meant a snapshot was simpler.
- Change one thing per test.
- One broken item shouldn't stop everything.
- Only claim what you've run. I removed two claims from the README that I hadn't actually tested.
What's next
The PR is up and waiting for review: cognee-community#388. Hoping it gets merged! 🤞
Thank you @dexters1 for assigning me the issue, and to the WeMakeDevs community for Mergetober. On to the next one.
Links
Top comments (0)