Built for the Flip the Switch Hackathon 2026.
GitHub: https://github.com/engineerkrish/joe-ai-email-assistant
Demo video: https://youtu.be/Q9CcvPKpbho?si=n_RmI6IHP4l8NZOn
Every morning, the same thing happens.
You open Gmail, see a bunch of unread emails, and start figuring out which ones actually matter.
Some are important. Some are promotional. Some need a response. Others can wait.
The problem isn't receiving emails.
The problem is figuring out what deserves your attention.
I wanted an assistant that could handle the first part of that work: check my inbox, identify important messages, search for specific emails, and help me draft replies.
But there was one rule.
The assistant should help me manage my inbox, not take control of it.
That's how I started building Joe AI Email Assistant.
And what I thought would be a straightforward AI integration turned into a series of authentication problems, missing integrations, local server debugging, Docker issues, and a surprising amount of learning about how AI agents actually work.
Here's how it went.
Meet Joe: My AI Email Assistant
Joe is an AI-powered email assistant built using Python, Gmail API, Google Gemini, Switch Console, and Mattermost.
Instead of manually navigating Gmail, I can open my Mattermost room and ask Joe to help.
For example:
Me: @joe Check my emails.
Joe retrieves the latest five emails and presents their sender, subject, and date.
It can also suggest which messages deserve attention first.
Me: @joe Search my inbox for hackathon emails.
Joe searches Gmail and returns matching messages.
Me: @joe Draft a reply to the hackathon registration email.
Joe prepares a proposed response that I can review and edit.
It doesn't send the email.
That distinction matters.
Joe is designed to assist with reading, organizing, and drafting—not to make irreversible decisions on my behalf.
Caption: Joe retrieves recent Gmail messages and helps identify which ones need attention.
GitHub: https://github.com/engineerkrish/joe-ai-email-assistant
Demo video: https://youtu.be/Q9CcvPKpbho?si=n_RmI6IHP4l8NZOn
Why I Built It
Most people don't need another email application.
They need something that reduces the effort required to use the one they already have.
Traditional email filters can organize messages according to predefined rules.
But they don't always answer questions like:
- Which emails should I focus on today?
- Did I receive anything important about my hackathon?
- Which messages might require a response?
- Can you help me write a professional reply?
I wanted to interact with my inbox conversationally.
Not through complicated commands.
Just by asking.
And I wanted that experience inside an agent workspace rather than building an entirely separate email dashboard.
That's where Switch Console came in.
What Joe Actually Does
Joe currently supports four main workflows.
1. Check Recent Emails
When I ask Joe to check my inbox, it retrieves up to five recent messages.
For each message, the Gmail bridge returns:
- Sender
- Subject
- Date
Joe can then organize those results into a readable response and suggest priorities.
The important detail is that the bridge currently retrieves email metadata, not full message bodies.
That means Joe's prioritization is based on limited information.
It can identify potentially important messages, but it cannot reliably determine everything an email contains.
2. Search Gmail Using Natural Language
I can ask:
@joe Search my inbox for hackathon emails.
Joe can use a Gmail search query such as:
in:inbox hackathon
The bridge forwards that query to Gmail and returns matching messages.
Caption: Joe searches Gmail for relevant messages directly from the Mattermost conversation.
3. Draft Email Replies
After finding a relevant message, I can ask Joe to draft a response.
For example, during testing, I used a hackathon email from WeMakeDevs.
Joe generated a proposed reply based on the information available.
Because it only had email metadata, the draft had to avoid inventing details from the unseen email body.
Caption: Joe prepares a proposed email reply without sending it.
4. Refine Drafts Through Conversation
I can continue the conversation:
@joe Make the reply shorter and friendlier.
Joe updates the draft based on my instructions.
Caption: Joe adjusts its proposed response based on user feedback.
This is one of my favorite parts of the project.
It feels less like operating a tool and more like collaborating with an assistant.
How It Works
The architecture is relatively simple.
But getting every component to communicate was the hardest part.
USER
|
v
Mattermost Room
|
v
Switch Console
|
v
Joe AI Agent
|
v
gmail_helper.py
|
v
Authenticated HTTP Request
|
v
Python Bridge (127.0.0.1:8765)
|
v
Gmail API
|
v
Email Metadata / Search
|
v
Joe AI Agent
|
v
Mattermost Response
There are two important pieces of Python code.
bridge.py
This runs a local HTTP server.
It communicates with Gmail using OAuth credentials and returns email metadata in JSON format.
The bridge listens on:
127.0.0.1:8765
It also checks an authentication token before processing requests.
gmail_helper.py
This is the interface Joe uses to communicate with the bridge.
Instead of manually constructing authenticated HTTP requests, Joe can run the helper script.
The helper loads the bridge token from the local environment and sends the request.
The result is a small but functional connection between an AI agent and a real Gmail inbox.
The standalone main.py also provides a Python-based Gmail and Gemini workflow.
The Part Nobody Sees: What Went Wrong
This is where the project became interesting.
The final demo looks simple.
Ask Joe a question.
Joe checks Gmail.
Joe responds.
But making that happen required solving several problems.
Problem 1: Gmail Authentication Wasn't Just an API Key
Initially, I thought connecting Gmail would be similar to calling a regular API.
It wasn't.
Gmail requires OAuth authorization to access a user's mailbox.
That meant configuring Google API credentials, handling the authorization process, and storing the resulting tokens locally.
There were multiple moving parts.
A mistake in the credentials, permissions, or token handling could prevent the entire integration from working.
How I Solved It
I configured the Gmail API authentication flow and used the gmail.readonly scope.
This was an intentional decision.
Joe only needed to retrieve information.
It didn't need permission to send, delete, or modify emails.
Once the OAuth flow was working, the Python application could access Gmail successfully.
Lesson: Give an AI application only the permissions it actually needs.
Problem 2: Switch Couldn't Directly Access Gmail
This was one of the biggest challenges.
I had a working Python Gmail integration.
I also had Switch Console running.
But connecting those two systems wasn't automatic.
The available Google Workspace connection didn't provide the Gmail access required for our workflow.
So even though Gmail worked in Python, Joe couldn't simply ask the connector to retrieve my messages.
For a while, I had two working components that couldn't communicate.
How I Solved It
I built a local HTTP bridge.
The bridge runs in Python and exposes a small authenticated endpoint.
It connects to Gmail, retrieves a limited set of messages, and returns structured JSON.
Then I connected Joe to that bridge through a Python helper script.
Instead of waiting for a direct integration, I created a small interface between the systems.
That was the breakthrough.
Lesson: When two tools don't integrate directly, a narrow, well-defined interface can connect them.
Problem 3: The Bridge Needed Authentication
Once the bridge was working, another question appeared.
What stops an unauthorized local request from accessing the email data?
Even a development prototype needs to think about security.
How I Solved It
I added token-based authentication.
The bridge checks the X-Joe-Token request header against a locally configured token.
Requests without the correct token are rejected.
I also bound the bridge to 127.0.0.1, keeping it on the local machine rather than exposing it as a public web service.
The token is stored in the local .env file, which is excluded from Git.
This doesn't make the entire system immune to attacks, but it reduces unnecessary exposure.
Lesson: Security should be part of the integration design, not something added after deployment.
Problem 4: Joe Needed a Reliable Way to Call the Bridge
The bridge was working.
The authentication token was working.
But I still needed a simple way for Joe to make requests.
Manually handling authentication headers and query parameters during every interaction was unnecessarily complicated.
How I Solved It
I created gmail_helper.py.
The helper handles the HTTP request, authentication header, and optional Gmail search query.
For example:
python gmail_helper.py
And for searching:
python gmail_helper.py --query "in:inbox hackathon"
This made the interface easier for Joe to use and easier for me to test independently.
Lesson: A small helper tool can make an agent integration significantly more reliable.
Problem 5: Mattermost Suddenly Couldn't Be Reached
This was one of the most frustrating moments.
The Python integration was ready.
Switch Console was configured.
But the Mattermost room wasn't responding correctly.
The error indicated that the Mattermost channel couldn't be reached.
The problem wasn't Gmail.
It wasn't Gemini.
It wasn't even the bridge.
The messaging environment itself wasn't fully available.
How I Solved It
I checked Docker Desktop and the local Switch services.
After starting Docker Desktop, enabling Mattermost, and restarting the relevant Switch server components, the room became accessible again.
Joe could finally receive and respond to messages.
Lesson: When an integration fails, check the infrastructure underneath it before rewriting working application code.
Problem 6: Checking Emails Wasn't Enough
At first, Joe could retrieve recent inbox messages.
That was useful.
But I wanted to ask more specific questions.
For example:
Did I receive anything about the hackathon?
The original bridge workflow focused on retrieving recent emails.
It needed additional support for search.
How I Solved It
I extended the bridge to accept a Gmail query parameter.
I also updated the helper script to pass search queries correctly.
That allowed Joe to use Gmail's existing search capabilities.
During testing, I searched for:
in:inbox hackathon
And Joe returned relevant messages.
This made the assistant much more practical.
Lesson: Build a small working version first, then extend it based on real interactions.
Problem 7: Publishing the Project Without Publishing My Secrets
After the application worked, I wanted to upload it to GitHub.
But the project used sensitive configuration files.
These included Gmail OAuth credentials, authorization tokens, and API keys.
Uploading those files would have been a serious mistake.
How I Solved It
I updated .gitignore to exclude sensitive files and directories.
I also created .env.example with placeholder values so other developers could understand the configuration without seeing real credentials.
Before publishing, I checked the repository contents.
The committed project included the Python source files, dependency list, README, and safe configuration template.
It did not include the local credential files.
Finally, I pushed the repository to GitHub.
Seeing the successful push was a satisfying moment because the project had gone from a local experiment to something other developers could explore.
Lesson: A successful GitHub push is only a success if the repository is safe to publish.
Why I Chose Switch Console and Mattermost
I didn't want Joe to be just another Python script that prints email information in a terminal.
I wanted a conversational assistant.
Switch Console provides the agent environment.
Mattermost provides the chat interface where I interact with Joe.
The Python bridge connects that environment to Gmail.
This separation makes the system easier to understand.
Each component has a specific responsibility.
The Gmail API retrieves information.
The bridge exposes a limited interface.
The helper communicates with the bridge.
Joe interprets requests and generates responses.
Mattermost displays the conversation.
Instead of trying to make one component do everything, I connected several smaller components.
The Security Decision I'm Most Glad I Made
Joe can read email metadata.
Joe can search Gmail.
Joe can draft replies.
Joe cannot send emails through the current integration.
This is not a hidden feature waiting for an approval button.
Sending is simply not implemented.
The Gmail OAuth scope is read-only, and the bridge exposes read operations.
That creates a useful boundary for the prototype.
The assistant can help me prepare a response, but I remain responsible for reviewing and sending it.
There is an important distinction between telling an agent not to send emails and giving it no email-sending capability.
For this version, I chose the latter.
What I Tested
I tested Joe through the Mattermost room using real requests.
| Test | Result |
|---|---|
| Check recent Gmail messages | Successful |
| Retrieve sender, subject, and date | Successful |
| Suggest which emails need attention | Successful |
| Search for hackathon emails | Successful |
| Draft a proposed reply | Successful |
| Make the reply shorter and friendlier | Successful |
| Connect through the authenticated Python bridge | Successful |
| Publish source code to GitHub | Successful |
These tests demonstrate the core workflow.
They don't establish that Joe's priority suggestions are always correct or that drafts always reflect the full meaning of an email.
The current bridge doesn't provide email bodies, so those limitations matter.
What I Learned About Building AI Agents
Three lessons stood out.
1. Integration Is Often Harder Than the AI
The most difficult parts weren't writing prompts.
They were getting OAuth, Python, local HTTP requests, Switch Console, Docker, and Mattermost to work together.
The AI experience depended on the reliability of everything underneath it.
2. Smaller Tools Make Better Building Blocks
Creating gmail_helper.py was a small change, but it made the workflow easier to operate.
Instead of giving Joe a complicated sequence of HTTP instructions, I gave it a focused tool.
That reduced unnecessary complexity.
3. An AI Assistant Doesn't Need Full Control to Be Useful
Joe doesn't send emails.
It doesn't delete messages.
It doesn't modify my inbox.
And it still helps with real tasks.
Sometimes the most useful automation is the one that prepares the work and leaves the final decision to the human.
What Joe Doesn't Do Yet
I want to be clear about the limitations.
No automatic email sending. Joe drafts responses but cannot send them through the current Gmail integration.
No full email-body retrieval through the bridge. The current bridge returns sender, subject, and date, so summaries and drafts are limited by that information.
No public cloud deployment. The bridge runs locally and must remain active for Joe to access Gmail.
No autonomous background monitoring. Joe responds to requests; it doesn't continuously monitor the inbox and independently notify users.
No guaranteed priority accuracy. Prioritization is an AI-generated suggestion based on available metadata.
Local setup is required. Running the complete experience requires the Python environment, Gmail authorization, Switch Console, and Mattermost services.
These aren't features I'm claiming to have completed.
They're opportunities for future development.
What's Next?
If I continue building Joe, I'd like to explore:
- Secure access to full email bodies for more accurate summaries.
- A reviewed, explicit approval workflow for sending drafted replies.
- Scheduled inbox summaries.
- Better email categorization and priority signals.
- A deployment option that doesn't require manually running a local bridge.
- More robust automated tests and monitoring.
But for the hackathon, my goal was to build a working end-to-end AI email assistant.
And I achieved that core workflow.
Try Joe AI Email Assistant
The source code is available on GitHub:
https://github.com/engineerkrish/joe-ai-email-assistant
The repository contains the Python application, local Gmail bridge, helper script, dependencies, configuration example, and setup documentation.
You'll need to configure your own Gmail OAuth credentials and local environment.
The README explains the project structure and setup process.
Final Thoughts
When I started, I thought the hardest part would be making an AI understand emails.
I was wrong.
The hardest part was getting all the systems around the AI to communicate reliably.
Gmail authentication.
Missing connector functionality.
A custom Python bridge.
Token authentication.
Docker and Mattermost connectivity.
Gmail search support.
Protecting secrets before publishing.
Each issue forced me to understand a different part of the system.
And each solution made the project more complete.
Joe AI Email Assistant started as an idea to make email management easier. It became a practical lesson in building secure, connected AI agents.
The best part?
I can now open Mattermost, type:
@joe Check my emails.
And Joe actually does.
That's a small interaction built on a lot of engineering.
And that's what made this project worth building.
Built for the Flip the Switch Hackathon 2026.
GitHub: https://github.com/engineerkrish/joe-ai-email-assistant
Demo: https://youtu.be/Q9CcvPKpbho?si=n_RmI6IHP4l8NZOn
Thanks for reading!
Top comments (0)