I Built an Obsidian Plugin to Audit an Entire Vault with AI. Here's How It Works

Most Obsidian vaults start simple.

A few folders. A few dozen notes. Some links. Maybe a graph that still looks understandable.

Then you keep using it.

A few hundred notes later, the situation changes.

You have notes you forgot existed, topics described three different ways, orphan files, inconsistent tags, half-finished ideas, duplicated concepts, and useful information that you remember writing but can no longer find.

At some point I started thinking:

What if I could audit an Obsidian vault almost like you would audit a codebase?

That idea became Vault Audit AI, a free and open-source Obsidian plugin.

It can analyze the structure and content of a vault, generate recommendations, create an Obsidian Canvas with the results, build a semantic search index, and process notes in batches.

But building it introduced an interesting problem:

How do you analyze an entire knowledge base when it doesn't fit into a single LLM context window?

The naive approach

The obvious implementation would be something like:

1. Read every Markdown file.
2. Concatenate everything.
3. Send it to an LLM.
4. Ask for recommendations.

This works surprisingly well for tiny vaults.

It also falls apart surprisingly quickly.

If a vault contains hundreds or thousands of notes, several problems appear:

• context becomes too large
• inference becomes expensive
• important details disappear inside a huge prompt
• one failed request can ruin the entire audit
• there is no useful intermediate representation
• re-running the audit means processing everything again

So I needed another approach.

Treating the vault as a dataset

Instead of thinking of the vault as one giant document, I started treating it as a collection of smaller documents.

The audit pipeline can analyze notes individually or in batches and then combine those intermediate results into a higher-level representation of the vault.

Conceptually, it looks like this:

Vault
  |
  |-- Note 1 --> analysis --\
  |-- Note 2 --> analysis ---|
  |-- Note 3 --> analysis ---|--> global analysis
  |-- ...                    |
  \-- Note N --> analysis --/

It is basically a MapReduce-style approach.

The "map" stage extracts useful information from smaller pieces of the vault.

The "reduce" stage combines those outputs and asks questions about the knowledge base as a whole:

• What thematic clusters exist?
• Which areas are weakly connected?
• Which notes appear redundant?
• Where is organization inconsistent?
• Which notes should probably be linked?
• What structural changes would make the vault easier to navigate?

This avoids requiring the complete raw vault to exist inside one prompt.

Three audit modes

I eventually kept several modes because they solve slightly different problems.

Single Audit processes notes individually and can maintain incremental audit state.

Single Full analyzes everything from scratch when you want a complete reset.

Batch + Report is the deeper MapReduce-style pipeline that combines batch analysis with global insights.

[INSERT SCREENSHOT: AUDIT MODE SELECTION]

For a small vault, the difference is not dramatic.

For a large vault, the architecture becomes much more important.

Turning the analysis into something useful

Another thing I didn't want was this:

AI analysis completed. Here are 4,000 words inside a modal window. Good luck.

The result should become part of the vault itself.

So Vault Audit AI can create a Markdown audit dashboard.

For example, I created a small English demo vault with 47 notes covering AI systems, engineering, product development, research, and a few fictional projects.

I intentionally added problems to it:

• orphan notes
• inconsistent tags
• overlapping topics
• weakly connected areas
• old notes that conflict with newer thinking

The plugin generated this dashboard:

In this case it identified several major clusters:

• AI Core & RAG
• Engineering & Infrastructure
• Product Strategy & Growth
• Cognitive Research
• Active Projects

It also found structural issues and generated reorganization recommendations.

The important part is that the output remains a normal Markdown file.

You can edit it, link to it, search it, version it with Git, or simply delete it.

Generating an Obsidian Canvas

I also wanted a more visual representation.

The audit can generate an Obsidian Canvas containing statistics, folder structure, and AI recommendations.

I like this approach because the analysis does not create some proprietary dashboard outside Obsidian.

The result is still an Obsidian artifact.

Semantic search

Auditing solves one problem.

Finding things is another.

Traditional search works well when you remember the words you used.

Humans, unfortunately, tend to remember ideas instead.

You might remember:

I wrote something about debugging AI systems in production.

But the actual note might be called:

Observability for AI Systems

There may not be an exact keyword match for your query.

So I added optional semantic search.

The plugin creates embeddings for Markdown content and stores a persistent local index.

A query is embedded using the same model and compared with the indexed content.

In my demo vault I searched for:

Ways to make an LLM workflow easier to debug in production

The highest results were:

• Observability for AI Systems
• Testing LLM Integrations
• Prompt Engineering
• AI Feature Launch Checklist

What I like about this example is that the query never says "observability."

The system still finds the note because the concepts are related.

Why not just use a vector database?

For a personal Obsidian vault, I didn't want the first version to require another server or database.

The current implementation therefore keeps the semantic index locally and performs a simple similarity scan.

That has an obvious trade-off.

It is easy to understand, portable, and perfectly usable for smaller knowledge bases.

It is not the architecture I would choose for millions of vectors.

For much larger indexes, something like HNSW or another approximate nearest-neighbor structure would make more sense.

But one principle I tried to follow while building the plugin was:

Don't introduce infrastructure until the problem actually requires it.

A personal knowledge base is not Google Search.

At least, mine isn't.

Yet.

Embeddings are optional

I also didn't want installing the plugin to silently start embedding someone's entire vault.

Semantic functionality is therefore optional.

The user explicitly enables it, chooses an embedding provider, and builds the index.

Language-model inference and embeddings can also use different providers.

The plugin currently supports:

• OpenRouter
• OpenAI
• Groq
• Ollama
• custom OpenAI-compatible endpoints

So it is possible, for example, to use one remote model for generation and another model for embeddings.

Or use Ollama if you prefer local inference.

Batch processing

Once the plugin already knew how to traverse and process a vault, another feature became fairly natural.

Batch operations.

You can select notes using folder, tag, or date filters and run operations such as:

• improve writing style
• summarize
• add examples
• generate tags
• add summaries
• fix grammar
• generate flashcards
• run a custom prompt

There is an important UX problem here though.

AI plus batch processing can become dangerous very quickly.

"Improve all 2,000 notes" sounds convenient right until you realize that the model enthusiastically improved something you wanted to keep exactly as it was.

So I think bulk AI operations should remain visible and deliberate rather than becoming invisible background magic.

The vault as both a graph and a semantic space

One thing I found particularly interesting while building this is that Obsidian effectively gives us two different representations of knowledge.

The first is explicit:

Note A --> [[Note B]]

Humans created those relationships intentionally.

The second is implicit:

embedding(Note A) ≈ embedding(Note C)

Those notes may be conceptually related even if nobody linked them.

These representations are useful for different reasons.

Links tell us what relationships the author explicitly recognized.

Embeddings can expose relationships that may have been missed.

Combining both makes vault analysis much more interesting than simply sending Markdown to an LLM.

Things that still need improvement

The plugin is far from finished.

A few areas I want to improve:

Incremental semantic indexing

The semantic index currently requires explicit updates.

Eventually I'd like file changes to update only the affected parts of the index.

Better similarity search for large vaults

The current local scan is intentionally simple.

An ANN index would make more sense once vault size makes exhaustive similarity comparisons expensive.

Better evaluation

This is probably the biggest one.

AI recommendations can sound reasonable while being useless.

I want to build better ways to measure whether audit recommendations actually improve a real vault rather than merely producing convincing text.

More real-world vaults

Synthetic demo vaults are useful for testing predictable failure cases.

Real personal knowledge bases are much stranger.

That's exactly why I'm interested in seeing what the plugin finds in them.

The project

Vault Audit AI is currently completely free and open source.

It is already available in the official Obsidian Community Plugins directory.

You can install it from:

Settings → Community plugins → Browse → search for "Vault Audit AI"

Community Plugins page:

https://community.obsidian.md/plugins/ai-knowledge-hub

If you use Obsidian and have an old, messy, or simply very large vault, I'd be particularly interested in what the audit finds.

I'm also curious how other people approach this problem.

How do you keep a knowledge base understandable once it grows beyond the point where you can remember what's inside it?