My housing society has a WhatsApp group with 164 members. It's where the notices go out: water supply timings, maintenance work, decisions from meetings. It's also where those notices get buried. The export I took had 3,320 messages spread over 239 days, and when I wanted to know something simple, like when the water was going to be off, my only option was to scroll and hope.
So for Mergetober (WeMakeDevs x Cognee), I built a WhatsApp connector for Cognee, the open-source memory layer for AI agents. It turns exported chats into memory you can ask questions of, and the answers come back with who said what and on which day.
- Issue: topoteretes/cognee#5836
- Pull request: topoteretes/cognee-community#471
Why an export and not an API
WhatsApp has no API for reading your personal or group chat history. The WhatsApp Business Platform is for businesses messaging their customers, not for reading a residents' group you happen to be in. The one official way to get your history out is the Export chat option in the app:
-
Android: open the chat, tap the three-dot menu, then More, then Export chat. You get a
.txtfile. -
iPhone: open the chat, tap the chat's name, then Export Chat. You get a
.zipwith_chat.txtinside.
That file is what the connector reads. Nothing else touches your account, and nothing is read until you point the connector at a file.
Using it
Cognee works in three steps: you add data, cognify turns it into a knowledge graph, and search asks it questions. A connector is one more thing you can hand to cognee.remember, which does the first two for you:
import cognee
from cognee_community_connector_whatsapp import whatsapp_source
await cognee.remember(
whatsapp_source(["exports/WhatsApp Chat with Residents Block 5.txt"]),
dataset_name="whatsapp",
primary_key="id",
write_disposition="merge", # the default "replace" would drop days that didn't change
max_rows_per_table=0, # no row cap, so deletes are checked against everything
)
Each chat gets its own node set, named whatsapp:<chat name>, so a question can stay inside one group:
from cognee.modules.engine.models.node_set import NodeSet
answer = await cognee.search(
query_text="When is the water supply off?",
query_type=cognee.SearchType.GRAPH_COMPLETION,
datasets=["whatsapp"],
node_type=NodeSet,
node_name=["whatsapp:Residents Block 5"],
)
On my society's export, questions like this came back with dated answers, for example "water supply for one hour on August 8", instead of a vague summary of the whole chat.
What the connector actually stores
A chat isn't a document, so the first decision was what to cut it into. One document for the whole chat means every new export re-processes everything, and a question about one date has to dig through months of text. One document per message gives you thousands of tiny pieces where a reply like "yes, Friday" means nothing on its own.
I went with one document per chat per day. A day is how people remember things in a group like this ("they said it on Saturday"), and it keeps replies next to the messages they reply to. Each day starts with plain Chat:, Date: and People: lines, so the date and the people are spelled out for Cognee instead of being implied. Here's a made-up example of what one looks like:
# Suppliers, 2026-10-14
Chat: Suppliers (WhatsApp)
Date: Wednesday, 14 October 2026
People: Asha, Ravi
09:16 Asha: Supplier quoted 420 per kg
09:17 Ravi: Can they deliver by Friday?
Also ask about the GST invoice
13:05 Asha: [media] Delivery note
08:00 · Asha added Meera
Media becomes a short marker ([media], [image], [attachment: Society rules.pdf]) and the caption is kept. System events, like someone joining, are kept with a ·. Deleted messages are dropped, and the "edited" marker is removed.
Re-importing without re-processing
You don't export a chat once. You export it again next month, and the new file contains everything from before plus whatever is new. If the connector sent all of that back through Cognee every time, each re-import would cost as much as the first one.
So every day is rendered to text and fingerprinted with a hash, and the fingerprints are kept in dlt's resource state. Re-importing the same export emits nothing. A newer export emits only the days that are new or whose text changed. On my real export, the first sync took about 14 minutes and the second took 15 seconds, because there was nothing to do.
The tricky part: knowing when a day was deleted
This is the part I spent the most time on. If someone deletes a message "for everyone", it should disappear from memory too. The obvious approach is: if a day was in the old export and isn't in the new one, delete it.
That approach would quietly destroy your oldest memories. WhatsApp caps exports at the newest 40,000 messages without media and 10,000 with media. In a busy group, the oldest days simply fall off the end of a new export. They weren't deleted, they just weren't included.
So the connector only forgets inside the window the new export actually covers:
- A day older than the new export's first message is kept, because the cap may have cut it off.
- A day newer than the new export's last message is kept, because it came from a newer export.
- If an export has 10,000 messages or more, its oldest day may have been cut partway through, so that day is never treated as changed or deleted.
- A file that can't be read is skipped and counted. It never causes a delete.
When a day inside the window really is gone, the connector emits a _deleted row for it, and Cognee's orphan cleanup removes it from the graph. I tested this on a copy of my real export where every message on one day was deleted: one _deleted row came out, and the dataset went from 239 documents to 238.
Phone numbers are masked by default
In my society's group, 161 of the 164 senders show up as raw phone numbers, because they aren't in my contacts. That's normal for a big residents' group, and it's not something I wanted to feed into a knowledge graph by default.
So international numbers are masked to the last four digits (+…3640), whether they appear as a sender, in the text, or as an @ mention. Plain digit runs like amounts or order numbers are left alone. You can turn masking off with mask_phone_numbers=False, but it's on unless you ask.
What my real export taught me
I first wrote the parser against WhatsApp's documented formats and synthetic files. Then I ran it on my society's export, and it found shapes the synthetic files didn't have:
- On Android,
<Media omitted>is followed by the caption on the same line. - With media included, attachments are written as
<file name> (file attached) <caption>, and file names can contain spaces and brackets, likeSociety rules 2026 (1).pdf. - A message that starts with a line break is written as
Name:alone on the first line. - View-once media shows up as
<View once voice message omitted>. - My zip had been renamed to
WhatsApp.zip, but the.txtinside still had the real chat name, so the connector now takes the name from there. - The file is full of invisible characters: narrow no-break spaces, no-break spaces and left-to-right marks. All of them are cleaned before parsing.
Each one now has its own test. Dates were another trap. Their order follows the phone's locale, so 03/04/26 could be March or April. The parser reads every date in the file and uses any value above 12 to settle which part is the day. My export used month-first dates and a 24-hour clock, and it was detected without any ambiguity.
The live run, in numbers
Everything below ran through cognee.remember with Cognee's default OpenAI models, on my society's Android export (3,320 messages over 239 days):
| Check | Result |
|---|---|
| First sync | 239 days in, 239 documents, about 14 minutes |
| Same export again | Nothing emitted, 239 days unchanged, 15 seconds |
| A copy with one day deleted for everyone | One _deleted row, 239 to 238 documents |
| Questions scoped to the group | Dated answers, with senders masked as +…3640
|
None of my real chat content is in the repo. The 34 tests use synthetic exports only: the names, numbers and messages are made up, but the line shapes, invisible characters and placeholders match real exports. One of them, test_whatsapp_forget.py, runs the real Cognee pipeline with a mocked LLM and mocked embeddings, and checks that a day emptied inside the window leaves the graph while a day before the window stays. CI runs the tests on Python 3.11 to 3.13, plus ruff and a wheel build.
What it doesn't do yet
- It reads text only. Media files aren't read, but their placeholders and captions are.
- The iPhone format is covered by synthetic tests, not a real iPhone export yet. If you have an iPhone, I'd love for you to try it.
- Days follow the exporting phone's clock, so exports from phones in different time zones won't line up by day.
- A chat's identity is its name. A renamed group is a new chat unless you map both names to one id with
chat_ids={"Old name": "my-group", "New name": "my-group"}. - System messages and placeholders are matched in English. Exports from phones in other languages still parse, but those markers stay as written.
A note on privacy
An export holds other people's messages, not just yours. Only ingest chats whose members are fine with it, and keep them in their own dataset, so one cognee.forget removes them. Anyone with read access to that dataset can see what went in.
Try it
The PR is open and waiting for review. Once it's merged and published, it will be pip install cognee-community-connector-whatsapp. Until then, you can run it from the PR branch:
git clone -b feature/whatsapp-connector https://github.com/N-45div/cognee-community-mergetober
cd cognee-community-mergetober/packages/connector/whatsapp
uv sync
export WHATSAPP_EXPORTS="/path/to/WhatsApp Chat with Your Group.txt"
export LLM_API_KEY="sk-..."
uv run python examples/example.py
The example syncs your export, prints the sync stats, and asks what was decided or promised in the chat and by whom. Export the chat again later and run it a second time: only the new and changed days get processed.
If you try it on your own groups, especially an iPhone export, tell me what broke on the PR: topoteretes/cognee-community#471.
Top comments (0)