Removed doesn't mean deleted: syncing Outlook mail and meetings into AI memory

Microsoft Graph tells you a calendar event was removed in two very different situations: someone deleted it, or it just slid out of the date window you asked about. Treat those the same and your AI memory forgets a meeting that still exists.

Almost everything hard about this connector looks like that. Outlook says one thing, and the safe move is to check what it really means before you delete anything.

cognee is an open source memory layer for AI agents. For Mergetober I built an Outlook connector that syncs mail folders and your calendar through Microsoft Graph into one memory.

Status: the PR is still open as I write this. It has been through three review rounds, and I pushed the fixes for the latest one on October 30.

What I want to use it for

My college actually uses Outlook, so this is the connector that would really help me out: work, events, dates, all in one place I can ask.

My live test was on my own Outlook.com account, but with test emails and meetings, so I haven't run it on real mail yet. One question I want to ask needs both sides: "what did this person email me before our last meeting?" Mail search finds the email and the calendar finds the meeting, but neither connects the two. To run it regularly I'll need an app registration, because a Graph Explorer token only lasts about an hour, and on a college account an admin may have to allow it.

What happened on a real account

I tested on a personal Outlook.com account with 4 test emails and 3 test events about a made-up drone project, one of them a daily recurring meeting. Gemini was the LLM and local fastembed did the embeddings.

I asked who leads the project, how long the battery lasts, and how much money was spent:

Project lead: Meera Joshi. Battery duration: 55 minutes. Money spent: 12 lakh rupees on batteries, plus 8 lakh rupees approved for solar panels

Then, in Outlook, I moved the kickoff email to Archive (one of the synced folders), and deleted the Budget update email and the Budget meeting. Same question after the next sync:

Project Falcon is led by Meera Joshi, the battery lasts 55 minutes, and the provided context does not mention how much money was spent.

Run	What happened	Sync stats
1	First sync	mail 4, events 3
2	Move kept as an update, deleted email and meeting forgotten	mail 1, mail deleted 1, events deleted 1
3	No changes	all 0
4	Asked only about the deleted money: not in the context	all 0

The live test also caught a bug before I recorded it. The daily meeting came back as the series plus 3 bare occurrences with no title. Now a series is one document and the bare entries are skipped. The PR has a video of all four runs.

Try it

Until the PR merges, install it from my branch:

git clone -b feature/outlook-connector https://github.com/laypatel13/cognee-community
cd cognee-community/packages/connector/outlook
uv sync

The quick way, no app registration. This is how I ran my live test. Sign in to Graph Explorer, consent to Mail.Read and Calendars.Read, and copy the token from its Access token tab. It lasts about an hour and is never refreshed.

export OUTLOOK_ACCESS_TOKEN=<token>
export OUTLOOK_MAIL_FOLDERS=inbox,sentitems,archive
export LLM_API_KEY=<your key>
uv run python examples/example.py "What are my recent emails and meetings about?"

The long-term way. Register an app in the Microsoft Entra admin center for any directory plus personal accounts, turn on public client flows, and add the delegated Graph permissions User.Read, Mail.Read and Calendars.Read. Then sign in once with a device code:

export OUTLOOK_CLIENT_ID=<application id>
export OUTLOOK_TOKEN_CACHE_PATH=~/.cognee_outlook_token_cache.json
uv run python examples/login.py

To be upfront: I haven't run this path live. Registering an app on a personal account needs an Azure subscription, which I don't have as a student, so the device code login is only covered by unit tests.

Once you're signed in, syncing looks like this:

import asyncio
import os

import cognee
from cognee_community_connector_outlook import outlook_source


async def main():
    await cognee.remember(
        outlook_source(
            mail_folders=["inbox", "sentitems"],
            client_id=os.environ["OUTLOOK_CLIENT_ID"],
            token_cache_path=os.environ["OUTLOOK_TOKEN_CACHE_PATH"],
        ),
        dataset_name="my_outlook",
        primary_key="id",
        write_disposition="merge",  # makes re-runs incremental and lets deletes reach the graph
        max_rows_per_table=0,  # removes the default cap of 50 rows
    )

    answer = await cognee.recall("What are my emails about?", datasets=["my_outlook"])
    print(answer)


asyncio.run(main())

If you only want headers, pass include_mail_bodies=False and Mail.ReadBasic is enough.

The interesting parts

Removed events are checked first. When Graph reports an event as removed, the connector looks it up by id. A 404 forgets it. Anything else means it only left the window, which is 180 days back and ahead by default, so it stays.

Moving mail is not deleting it. Every request asks for immutable ids, so a message keeps its id when it moves, and the connector tracks which synced folders each message is in. Inbox to Archive, with both synced, is an update. Deleting moves mail to Deleted Items, which is never synced, so the message is in no synced folder and gets forgotten. The same goes for mail moved to any folder you don't sync.

Later runs only fetch changes. Each mail folder and the calendar use Graph delta queries. The delta link only arrives on the last page, so it's saved once the whole round has finished.

People connect across mail and meetings. Sender, recipients, organizer and attendees are written into every message and event, so cognee's entity extraction can link someone's emails to their meetings. That comes from the LLM reading the text, not from a hard identity mapping.

Errors never delete. Only a 404, a cancellation, or mail leaving every synced folder forgets anything. A Graph outage, throttling or one unreadable event removes nothing, and one failing folder doesn't stop the others.

It only reads. It never sends, moves or changes anything. Attachments and Bcc are never read, and the logs hold counts only. Your mail text does go to your LLM provider during processing, like any document you add to cognee, so use a local model if that matters to you.

What three review rounds taught me

goran-radonic reviewed this PR the same way as my Dropbox one: real dlt, real cognee, failures injected on purpose. The test count went from 65 on October 18 to 164 on October 30.

Round one was 20 points. Three of them could lose data or fail without a clear reason. Deleting a series reported every occurrence, and each one was looked up. A failing lookup could wait through Retry-After with no limit: 20 events answering 503 with a 60 second wait, while cognee holds a staging lock. Now all the waits in a run share a 2 minute budget. For this round I reverted each of the 38 guards in the commit, one at a time, to check that a test went red.

Round two found mail and meetings that could never be forgotten. I stream rows page by page, but I only saved state at the end. If page 2 failed, the rows from page 1 still loaded, and the state didn't know about them. Delete one of those later and it stayed in memory for good, with no error anywhere. Yielded ids are now kept in the state even when a part fails.

That round also caught a test that couldn't fail. Graph names all-day time zones the Windows way, like Tonga Standard Time, and my test used Pacific/Tongatapu, which Graph doesn't send. So an all-day event on November 8 could be written as November 7, and my test stayed green. The names are now mapped with tzlocal.

It also showed that a short outage could forget live events. I counted a 5xx as a failed attempt, and after 3 attempts I forgot the event. So a 5xx stopped counting, and giving up now keeps the event.

Round three was caused by that fix. If a 5xx doesn't count, it has no limit. A series master that kept answering 500 stayed in the retry list forever and held up the lookups behind it. Over six runs in a row, nothing moved. Now outages are counted per id, separately from real failures, an id in an outage goes after the others, masters and removals are looked up separately, and after 3 runs in a row the id is given up on and kept.

What I took from this one: a fix is new code with its own failure modes. "This doesn't count as a failure" sounds like a safe rule until you ask what stops it. Every retry, wait and skip needs a limit, and every limit needs a decision about what happens when you reach it.

Limits so far

• Only the primary calendar, since Graph v1.0 event delta covers that one.
• Attachments are flagged, not ingested.
• Subfolders of a selected folder are not synced.

Links

• PR: https://github.com/topoteretes/cognee-community/pull/365
• cognee: https://github.com/topoteretes/cognee
• Built during Mergetober by WeMakeDevs x cognee