How I Built an AI Meeting Agent With Hindsight Memory
Meetings create a simple problem that is easy to underestimate: important information disappears as soon as the meeting ends.
A decision might be made, a deadline might change, or someone might join the meeting late and have no idea what has already been discussed.
I built MeetingHindsight to solve this problem by combining meeting intelligence with persistent memory using Hindsight.
What MeetingHindsight Does
MeetingHindsight is an AI meeting intelligence agent designed to help teams understand and remember their meetings.
It focuses on five things:
- 🎙️ Meeting summary
- ⚡ Late-join catch-up
- ✅ Decision detection
- 🧠 Persistent memory using Hindsight
- 🔄 Meeting change detection
The goal is not simply to summarize a conversation. The important part is allowing the agent to remember information from previous meetings and use that information later.
Turning a Meeting Conversation Into Useful Information
The first step is converting a meeting conversation into text.
For my prototype, I used speech-to-text so that spoken conversation can be processed by the agent.
Once the conversation becomes a transcript, the meeting agent sends it to an LLM and asks it to identify the important information.
For example, a meeting might contain a conversation like:
text
Ravi: Let's discuss our project demo.
Sam: I think we should have the demo on Friday.
Priya: Friday works for me.
Ravi: Okay, let's agree on Friday for the project demo.
Sam: Agreed. The demo will be on Friday, October 2nd.
Priya: We also need to finish the presentation before the demo.
Instead of forcing someone to read the entire conversation again, the agent produces a concise summary containing the important points and decisions.
Helping People Who Join Late
One of the situations I wanted to handle was the late-join problem.
Imagine joining a meeting ten minutes after it started. The conversation may already have moved through several topics and decisions.
Instead of asking everyone to repeat the discussion, MeetingHindsight can analyze the part of the transcript that happened before the participant joined.
The result is organized into:
What You Missed
Decisions Already Made
Important Points
What You Need to Know Now
This makes the catch-up much shorter than reading the entire transcript.
Separating Decisions From Discussions
Meetings contain many statements that sound important but are not actually decisions.
Someone might say:
"I think we should move the demo to Monday."
That is a suggestion.
But when someone says:
"Agreed. Let's move the project demo to Monday."
it represents an actual decision.
I therefore made decision detection a separate part of the system.
The agent is instructed to ignore suggestions, opinions, questions and possibilities and identify only decisions that were actually agreed upon.
This distinction is important because storing every statement as a decision would make meeting memory much less useful.
The Important Part: Persistent Memory
The main idea behind MeetingHindsight is memory.
A normal meeting assistant can summarize the current conversation, but that information can become difficult to use once the meeting is over.
I integrated Hindsight so that important meeting information can be retained and recalled later.
The basic flow is:
Meeting transcript
↓
AI analysis
↓
Important meeting information
↓
Hindsight memory
↓
Recall when needed
The application stores meeting information in a Hindsight memory bank.
Later, a user can ask a question such as:
What decisions were made about the demo?
Instead of analyzing the original conversation again, the application can query the stored memory.
This is where the idea becomes more useful than a simple summarization tool.
The Hindsight documentation describes Hindsight as a memory system designed for agents, and that persistent memory is the part I wanted to make central to this project.
Comparing Meetings Over Time
Meetings rarely happen only once.
Plans change.
A deadline that was Friday might become Monday. A responsibility might move from one person to another. A decision might be changed during a later discussion.
MeetingHindsight includes a change detection component that compares information from two meetings.
For example:
Previous meeting
The project demo will be on Friday, October 2nd.
The presentation should be completed before Thursday.
Current meeting
The project demo will be on Monday, October 5th.
The presentation deadline remains before Thursday.
The agent can identify that the demo date changed while the presentation deadline stayed the same.
This is useful because people usually care more about what changed than about rereading two complete meeting transcripts.
How the System Is Structured
The application is built as several focused components rather than one large agent.
Meeting Conversation
|
↓
Speech-to-Text
|
↓
Meeting Intelligence
/ | \
/ | \
Summary Decisions Late Join
\ | /
\ | /
Hindsight
Memory
|
↓
Recall / Compare
|
↓
Useful Context
The project uses Python, Streamlit, Groq for language-model processing, speech-to-text, and Hindsight for persistent memory.
The Hindsight GitHub repository was particularly useful for understanding how the memory layer could fit into the application.
What I Learned
1. Memory is more useful when it has a purpose
Simply storing information is not enough.
The useful part is being able to retrieve information when a new question or meeting makes that information relevant.
2. Not every important sentence is a decision
A meeting contains suggestions, questions, opinions and actual agreements.
Treating all of them equally would create noisy memory, so decision detection needs its own logic.
3. Context from previous meetings can be more valuable than another summary
A summary tells you what happened.
Memory can help answer questions about what happened previously and how that information relates to the current situation.
4. Small focused components are easier to reason about
Instead of making one large prompt handle everything, I separated summary generation, late-join catch-up, decision detection, memory and change detection.
This made it easier to test each part independently.
What's Next
The current system demonstrates the core workflow of meeting understanding and persistent memory.
The next step would be to make the memory layer increasingly useful as more meetings are retained, so that the agent can build a longer-term understanding of discussions, decisions and changes over time.
The broader idea is simple:
A meeting assistant should not forget everything when the meeting ends.
That is the problem MeetingHindsight is designed to address.
Top comments (0)