DEV Community: Tanvi P

VaultMind: Building an AI That Remembers Why an Organization Made Its Decisions

Tanvi P — Tue, 29 Sep 2026 17:54:14 +0000

An exploration of persistent organizational memory using Hindsight, MongoDB, and an AI agent.

What happens when the one employee who knows why something works leaves the company?

Imagine a senior backend engineer leaves an organization after four years.

Six months later, a new engineer is asked to modify a critical service.

They find the documentation.

“The service uses Redis.”

But the documentation doesn't answer the questions that actually matter:

Why was Redis chosen?
What alternatives were considered?
What happened when they were tested?
Was there an incident that influenced the decision?
Are there workarounds the new engineer should know?
What mistakes should they avoid?

The organization hasn't necessarily lost the information.

It has lost the context.

That is the problem we explored with VaultMind, an AI-powered Institutional Memory Agent built using Hindsight as its persistent memory layer.

The goal wasn't to build another chatbot that searches company documents.

It was to explore whether an AI system could preserve organizational experiences and recover the reasoning behind decisions long after the people who made them were gone.

The Problem: Organizations Don't Just Lose People, They Lose Context

Organizations generate enormous amounts of knowledge every day.

Architecture decisions are discussed in meetings. Incidents are documented in tickets. Customer requirements appear in conversations. Workarounds are discovered during production failures.

And a lot of expertise never gets formally documented at all.

Eventually, organizational knowledge becomes scattered across documents, incident reports, architecture decisions, conversations, handover notes, and individual employees.

The result is a deceptively difficult problem:

The answer exists somewhere, but finding it — and understanding why it is true — is difficult.

Consider a simple example.

A document says:

“The payment service retries failed requests three times.”

That's useful.

But suppose an engineer is about to change the retry logic. What they really need to know might be:

“The retry limit was reduced from five to three after repeated retries caused duplicate payment attempts during a provider outage. The change was introduced after incident INC-201 and was kept because the payment provider's idempotency guarantees were inconsistent.”

Now the engineer has more than a configuration value.

They have the reasoning, history, and constraints behind it.

That is the distinction we wanted VaultMind to explore.

Why Standard RAG Isn't the Whole Answer

Retrieval-Augmented Generation is extremely useful when the answer lives in a document.

For example:

“Where is the authentication configuration documented?”

A conventional RAG system can retrieve the relevant document and provide the answer.

But institutional memory creates a different class of questions whose answers may not exist in a single document.

It might require connecting an architecture decision with an incident, an engineering discussion, a workaround, and a later change.

The challenge therefore isn't simply retrieving text.

It's recovering organizational experience from information accumulated over time.

That is where persistent memory becomes useful.

VaultMind uses Hindsight as the persistent memory layer so important organizational experiences can be retained and recalled when they become relevant to a future question.

Introducing VaultMind

VaultMind is built around a simple idea:

An organization should be able to remember what it has learned.

Instead of treating every question as an isolated interaction, VaultMind gives organizational knowledge a persistent memory layer.

The intended lifecycle is:

Organizational Experience
          ↓
        Retain
          ↓
       Hindsight
          ↓
    Future Question
          ↓
        Recall
          ↓
Historical Context + Evidence
          ↓
       AI Response

The memories can represent different kinds of organizational knowledge:

Decisions
Incidents
Workarounds
Lessons learned
Risks
Customer requirements
Constraints
Processes
System knowledge

The important part is that these aren't treated as interchangeable pieces of text.

A system configuration, an incident, and a decision all carry different meanings.

Architecture

VaultMind uses a full-stack architecture consisting of:

React + Vite — frontend
Django REST Framework — backend API
MongoDB — application data and metadata
Hindsight — persistent organizational memory
LLM — reasoning and response generation
JWT — authentication
RBAC — authorization

High-level architecture

[ADD ARCHITECTURE DIAGRAM HERE]

MongoDB handles application-level information such as users, roles, documents, metadata, and audit records.

Hindsight serves a different purpose: persistent organizational memory.

Keeping these responsibilities separate means the application's operational data and the agent's memory don't have to be treated as the same thing.

Hindsight: Giving the Agent Persistent Memory

Hindsight is at the center of VaultMind's memory layer.

There are two important parts to the interaction: retaining organizational experience and recalling it later.

When meaningful information enters the system, it can be retained as persistent memory.

For example, an incident might establish:

“The payment provider occasionally returned duplicate success responses during timeouts, so the retry policy was changed to avoid repeated payment attempts.”

Later, an engineer might ask:

“Why is the payment service so conservative about retries?”

Instead of treating the question as completely new, VaultMind can recall the relevant organizational experience before generating the answer.

The important distinction is that the LLM isn't expected to remember the organization on its own.

Hindsight provides the historical context; the LLM reasons over that context.

[ADD HINDSIGHT RETAIN/RECALL SCREENSHOT HERE]

The "Why" Layer

The most interesting capability we explored with VaultMind is what we call the Why Engine.

Most knowledge systems are designed around questions like:

“What is this?”

Institutional memory often needs to answer:

“Why is this the way it is?”

Consider a customer requirement.

A customer may have a seemingly unusual requirement that affects the way a system handles data.

Months later, a developer sees the implementation and thinks:

“Why don't we just simplify this?”

A normal documentation search might find the implementation.

Institutional memory should ideally surface the history:

The customer required this behavior because of a contractual reporting requirement. A previous implementation removed the extra processing and caused a reporting discrepancy. The current behavior was introduced after that incident.

Suddenly, the “weird” implementation makes sense.

That's the value of the Why Engine.

It isn't about giving the model more general technical knowledge.

It's about helping it recover organization-specific reasoning.

Evidence Matters More Than Confidence

This is one of the most important design principles behind VaultMind.

A system that remembers organizational history should not invent that history.

If the available memories only show that the architecture changed, but don't explain why, the system should not manufacture a plausible explanation.

This is particularly important for institutional memory because a confident but incorrect historical explanation can become a new source of misinformation.

Memory should provide evidence, not just confidence.

Security: Memory Needs Boundaries

Persistent organizational memory can contain sensitive information.

Consider an employee asking:

“Show me the contract value for Customer X.”

If that information is restricted, it shouldn't be retrieved for the LLM and filtered out afterward.

VaultMind places authorization around the memory retrieval process.

The intended flow is:

Question
   ↓
Authentication
   ↓
Authorization
   ↓
Allowed Memories
   ↓
Hindsight Recall
   ↓
LLM
   ↓
Response

VaultMind includes:

JWT authentication
Role-Based Access Control
Multiple access levels
PII and secret detection
Audit logging

[ADD SECURITY/AUDIT SCREENSHOT HERE]

This matters because persistent memory changes the security problem.

A piece of information that is temporarily visible in a conversation is one thing.

A piece of information that becomes part of an organization's long-term memory is much more consequential.

That leads to a simple principle:

Not everything an organization knows should become persistent memory.

When the Memory Itself Becomes the Problem

Once organizational knowledge becomes persistent, new problems appear.

Knowledge Concentration

What happens when most of the knowledge about a critical system exists with one employee?

VaultMind can identify this kind of concentration and surface it as a continuity risk.

For example:

Payment Service

47 related memories

Most associated with one engineer

Limited secondary knowledge

High continuity risk

The goal is to identify knowledge-loss risks before someone leaves.

Knowledge Gaps

Sometimes the most useful result is discovering that there isn't enough information.

VaultMind can surface gaps such as:

No documented owner
An incident with no recorded root cause
An outdated or unverified policy
No secondary knowledge holder

In other words, the system can help answer not only:

“What do we know?”

but also:

“What don't we know?”

Conflicting Memories

Organizational knowledge can also contradict itself.

For example, one record might say:

“The payment service retries failed requests three times.”

while a later record says:

“The payment service retries failed requests five times.”

Rather than silently choosing one, VaultMind can surface the conflict along with its sources and dates, allowing someone to verify the current state.

This matters because old information can still explain why a system works the way it does, even when it is no longer current.

Keeping the Timeline

That leads to another important part of institutional memory: history matters.

A system might evolve through:

Initial architecture
       ↓
Production issue
       ↓
Workaround
       ↓
Architecture decision
       ↓
New implementation

The current system tells you where the organization ended up.

The history tells you why.

VaultMind therefore shouldn't simply replace old memories with new ones. Older information can still be valuable — as long as it's clear that it is historical rather than current.

The NovaPay Demo

To demonstrate VaultMind, we created a fictional company called NovaPay using synthetic organizational data.

The dataset includes:

Employees
Architecture decisions
Incidents
Workarounds
Customer requirements
System knowledge
Handover information

The important part is that these aren't isolated examples.

They are connected.

For example, an incident might lead to an architecture decision, which affects a system, which is associated with an engineer's knowledge and eventually appears in a handover.

That gives the agent something more interesting to work with than a collection of unrelated documents.

[ADD NOVAPAY / UI SCREENSHOT HERE]

What Can VaultMind Actually Answer?

Instead of demonstrating the same Redis/Kafka scenario repeatedly, we designed the demo around different kinds of institutional questions.

“Why does the payment service only retry three times?”

The system can connect the current behavior to the incident and reasoning that led to the policy.

“What should I know before changing the customer reporting pipeline?”

The system can combine customer requirements, previous incidents, decisions, and known workarounds.

“Who has historically worked on this system?”

The answer can be based on the relationship between people and organizational memories rather than simply an employee directory.

“Is there anything incomplete or conflicting in this system's documentation?”

The system can surface knowledge gaps, conflicting information, or stale memories.

“Why was this architecture decision made?”

This is where the Why Engine becomes particularly useful: the goal is to retrieve the historical evidence behind the decision rather than provide a generic technical explanation.

The common thread is that these questions aren't asking only for facts.

They're asking for organizational context.

Testing the System

Memory systems have failure cases that ordinary question-answering systems don't necessarily have to deal with.

VaultMind includes automated tests covering areas such as:

Authentication
RBAC
Hindsight integration
PII detection
Memory extraction

We also considered cases such as:

No relevant memory exists
Two memories conflict
A user isn't authorized to access a memory
A memory contains sensitive information
Historical information is outdated
There isn't enough evidence to answer a question

For an institutional memory system, these aren't edge cases to ignore.

They're fundamental to whether the system can be trusted.

What We Learned

The biggest lesson from building VaultMind was that memory is not just storage.

Once information becomes persistent, entirely new problems appear.

Historical information can be valuable even when it isn't current.

An old decision may no longer describe the current system, but it can explain why the current system exists.

Missing information is itself information.

A system that can identify knowledge gaps is more useful than one that confidently fills every gap.

Knowledge concentration creates organizational risk.

If critical knowledge exists almost entirely with one person, the organization has a continuity problem even if all of its systems are currently working.

Security has to happen before generation.

If sensitive information reaches the LLM first, filtering the final response is already too late.

AI memory needs humility.

For institutional knowledge, an honest:

“There isn't enough evidence to determine that.”

can be much more valuable than a convincing guess.

Limitations

VaultMind is currently a prototype.

The NovaPay organization and its employee information are synthetic.

The quality of the system depends heavily on the quality of the memories it receives.

PII and secret detection cannot guarantee that every possible sensitive value will be identified.

Historical memories can conflict, and automated systems cannot always determine which information represents the current truth.

Important organizational decisions should therefore still involve human verification.

VaultMind is intended as an institutional-memory assistant, not an autonomous authority on organizational truth.

What's Next

There are several directions we'd like to explore further.

The first is connecting VaultMind directly to the systems where organizational knowledge is created:

Slack
Jira
Confluence
GitHub
Incident-management systems
Internal documentation platforms

Another direction is memory verification.

Instead of simply identifying stale or conflicting memories, the system could route them to the appropriate team or owner for verification.

There is also an opportunity to build a richer organizational knowledge graph connecting:

people → systems → incidents → decisions → customers → outcomes

The long-term idea is to move toward organizational memory that continuously evolves instead of requiring someone to manually maintain another knowledge base.

Final Thoughts

The idea behind VaultMind started with a simple observation:

When an employee leaves, an organization doesn't just lose their labor. It can lose years of context.

Traditional documentation can tell you what a system does.

Institutional memory should also help explain:

Why it does it that way
What was tried before
What failed
What decisions shaped it
What constraints still matter
What mistakes should be avoided
What information may be outdated
What the organization still doesn't know

That's what we wanted VaultMind to explore.

The goal isn't to build an AI that knows everything.

It's to build an AI that remembers what an organization has learned.

Because sometimes the most valuable piece of information isn't the answer.

It's why the organization arrived at that answer in the first place.

Project: VaultMind

GitHub: https://github.com/samyukthaareddy/vaultmind

Tech: React · Vite · Django REST Framework · MongoDB · Hindsight · LLM · JWT · RBAC

Building AceExam: An AI-Powered Accessible Examination Platform with MongoDB

Tanvi P — Mon, 27 Apr 2026 15:17:09 +0000

By Tanvi, Ranadeep, Nachiketh, Jeathraditya

Developed under the guidance of Professor Chanda Raj Kumar,
and we are thankful for his valuable support throughout this project.

Check out AceExam here

Online exams aren't built for everyone.

Most digital examination platforms assume every student sees the screen clearly, types comfortably, and processes text without difficulty. The reality for students with visual impairments, dyslexia, or motor disabilities is entirely different, they are left to patch together other methods like browser plugins, screen readers bolted on , or human invigilators typing on their behalf.
The gap isn't just inconvenient. It's a systemic barrier to fair assessment.
AceExam is our attempt to close it. We aim to implement accessibility at the architecture level, training custom AI models for exam interaction, and using MongoDB as the data backbone for a platform that serves genuinely different kinds of users from a single codebase.

"Accessibility is not a feature. It is a starting point. AceExam treats accommodation types as first-class citizens, not optional add-ons."

How AceExam Is Structured

AceExam follows a clean three-tier hierarchy with clear permissions at each level:

Organisation (super admin) → Teacher / Admin → Student

The system has three roles Organisation, Teacher, and Student each with their own dashboard. A teacher creates an exam, adds questions with difficulty ratings, and assigns it to students. When adding a student, they specify the student's disability type. That single field disability_type is the key that unlocks everything.

When a student logs in and starts their exam, the system reads their disability_type from MongoDB, activates the right accessibility feature profile automatically, extends the exam timer, and transforms the UI — before the first question even loads. No manual setup. No settings menus.

The Tech Stack

Backend
Python (Flask) with MongoDB — scoped queries and mutations per role, with MongoDB as the primary data store for flexible document structures across exam types.

Frontend
HTML, CSS, Vanilla JavaScript (Server-side rendering via Jinja2 templates) with a consistent light theme and a compact five-button accessibility toolbar that persists across all exam pages.

AI / ML
Custom PyTorch models for speech-to-text and eye tracking, plus Python NLP libraries for text simplification and TTS via the Web Speech API.

Deployment
Built as a web-based platform — no installation required for students. Accessibility features run client-side using browser APIs where possible.

Why MongoDB?

Exam data is not flat. A single student attempt bundles together their profile, accommodation type, question set, per-question response, timestamps, and accessibility state and these fields vary significantly between accommodation types. A blind student's record carries voice command logs; a dyslexic student carries font and overlay settings.

Forcing that into a rigid relational schema means either sparse columns everywhere or a patchwork of JOIN tables that make every query more complex than it needs to be.
MongoDB's document model handles this naturally. Each student attempt is one document. Loading an attempt for grading or review is one read — no assembling rows from multiple tables, no N+1 queries. The flexible schema also means adding a new accommodation type or a new accessibility field doesn't require a migration that touches every existing record.
We use MongoDB as the primary data store across three collections:
users — organisation, teacher, and student accounts with roles and accommodation types
exams — question banks, assignment metadata, and per-exam settings created by teachers
attempts — one document per student attempt, with responses, scores, and accessibility state embedded
The once-only constraint on attempts is enforced at the data layer. Once a student submits, the attempt document is locked — subsequent submission calls check the document status before writing anything.

Accessibility by Accommodation Type

When a student logs in, AceExam reads their registered accommodation type and automatically enables the relevant feature set. No manual configuration. No settings menu to navigate.
🔵 Blind
Text-to-speech reads every question aloud. Voice commands handle navigation and answer selection.
🟣 Dyslexic
Dyslexia-friendly font, increased letter spacing, and pastel background overlays to reduce visual stress.
🟢 Motor Impairment
Full keyboard navigation with single-key shortcuts. Eye tracking allows gaze-based option selection without any physical input device.
⚪ Standard
The default interface, with the five-button accessibility toolbar available if needed.

The AI We Built: Custom Speech-to-Text

Rather than dropping in a general-purpose speech API, we trained a custom keyword classification model — exam_listener_v1.pt — specifically for exam interaction using PyTorch.
The model classifies audio input into exactly 11 command categories:
cancel · confirm · next · option_a · option_b · option_c · option_d · previous · read · silence · submit

Why a narrow vocabulary?

A general ASR model that understands free-form speech introduces significant surface area for misclassification during an exam. If a student says "I think it's option B" while thinking aloud, a general model might select option B unintentionally. Our model knows only the valid exam commands — and classifies only those.

Accuracy Comparison

Our model trades raw accuracy for domain specificity. The accuracy gap reflects limited training data and noise sensitivity — known drawbacks we're addressing through data augmentation and noise-robust training in the next iteration.

Eye Tracking for Motor Accessibility

For students with motor impairments, AceExam includes a webcam-based eye tracking module built in Python using OpenCV. It maps pupil position to on-screen MCQ options — sustained gaze on an option triggers selection with no mouse or keyboard required.

Accuracy Comparison

The current implementation is computationally intensive and struggles under variable lighting — an acknowledged limitation. It demonstrates the feasibility of gaze-based exam interaction; production-grade accuracy would require a calibrated backbone like MediaPipe. That replacement is already on the roadmap.

The Exam Attempt Flow

Every student attempt follows the same path:

Login
  → Accommodation type read from MongoDB
    → Accessibility features auto-enabled
      → Exam loads
        → Attempt (once only)
          → Submit confirmation dialog
            → Score written to MongoDB (attempt locked)

During the attempt, three accessibility channels run in parallel: the five-button toolbar, single-key keyboard shortcuts, and voice commands. Students can use whichever channel suits them without any mode-switching.

What MongoDB Enables in Practice

Flexible document structure across accommodation types. A blind student's attempt document carries different fields from a motor-impaired student's — MongoDB handles both without schema pressure.
Single-read exam loading. The full context needed to render an exam attempt — questions, student profile, accessibility settings, prior responses — lives in one document. One read to load the exam. No assembly required.
Atomic attempt locking. Once a student submits, a single $set on the attempt document marks it as locked. No race conditions, no partial writes, no separate status table to consult.
Scalable across teacher and student volumes. MongoDB's horizontal scaling means adding new schools, new exam sets, and new student cohorts doesn't require redesigning the data layer.

What's Next

AceExam is a research prototype — a proof of concept that accessible digital examination is achievable with a focused stack and well-scoped AI models. The next directions:
Automated question generation via NLP
Semantic answer grading (not just keyword match)
Text simplification for dyslexic users using transformer-based models
MediaPipe-backed eye tracking to replace the current custom implementation
Noise-robust STT training with data augmentation to close the accuracy gap

Key Takeaways

The decision to train domain-specific AI models rather than use off-the-shelf APIs was deliberate. A narrow vocabulary STT model makes fewer exam-context errors even at lower raw accuracy. A gaze-based selection system, even at 40% accuracy in prototype form, proves the interaction paradigm is viable and gives us a concrete upgrade path.
MongoDB's document model was the right call for a platform where user data shapes vary significantly by role and accommodation type. Flexible schema meant faster iteration, and the single-document-per-attempt design kept the data access patterns clean.
The goal was never to build a finished product. It was to demonstrate that the gap between standard exam platforms and genuinely accessible ones can be closed — and to show exactly where the hard problems still live.