Indexing Obsidian for an AI agent: plugin, MCP server, or build your own?

Once your Obsidian knowledge base grows past a handful of notes, the agent can no longer just read everything — crawling note-by-note gets slow and burns tokens. The fix is an index: a compact map the agent reads first, so it can jump straight to the two or three notes that matter. But there’s more than one way to build that map, and the choice matters more than it looks. This article compares the three real options, lays out the pros and cons in a matrix, and explains why — for a fact-first, own-the-stack setup — we build our own.

The three ways to index a vault

Every approach solves the same problem — give the agent a map instead of making it walk every shelf — but they live in different places and build different kinds of index.

• Build your own (our approach). A script reads the vault and writes an index.json (or an embeddings file). It runs outside Obsidian, and you own every line.
• Community plugins. Plugins that run inside Obsidian and build and maintain the index for you — Smart Connections, Copilot, Lina, Obsidian RAG Search.
• MCP servers. A separate program, outside Obsidian, that exposes your vault to AI tools like Claude Desktop — obsidian-hybrid-search, vault-kb, obsidian-mcp.

The honest core of the comparison is what kind of index each one makes. Our script builds a structural/keyword index: for every note it records the title, tags, headings, and resolved links. Most plugins and MCP servers build a semantic index using embeddings, so they can match meaning, not just words. That single difference explains almost every row in the matrix below.

The pros-and-cons matrix

Criterion	Own indexer (ours)	Community plugins	MCP servers
Where it runs	Standalone script, outside Obsidian	Inside Obsidian (must be open)	Separate program alongside Obsidian
Index type	Structural / keyword (JSON)	Semantic (embeddings)	Often hybrid (keyword + embeddings)
Agent access	Direct — agent reads one JSON file	Indirect — the plugin searches, not the agent	Native — agent talks to the server directly
Cost	Free, no API needed	Often an embeddings API = recurring cost	Usually an embeddings API or a local model
Privacy	Fully local, nothing leaves the machine	Vault text sent to an embeddings provider	Config-dependent; local is possible but heavier
Setup effort	You write / maintain the script	Click-install, lowest barrier	Extra process + config + API keys
Maintenance	Yours — but you understand every line	Plugin author (can age or break)	Server author (often young / experimental)
Semantic search	No (unless you add embeddings yourself)	Yes — core feature	Yes (hybrid)
Token efficiency	High — one compact map, no crawling	Varies — plugin decides what comes back	High — server returns only the hits
Lock-in	None — plain JSON, your code	Plugin format + provider	Server protocol + provider

What we have: a small Python indexer

Our index is produced by one standard-library Python script, index_vault.py. It walks the vault once, records each note’s title, aliases, tags, headings and outgoing [[wikilinks]], resolves those links to real files, computes the reverse backlinks, and writes the whole thing to a single _meta/index.json. No dependencies, no API, runs in under a second on dozens of notes. The agent reads that one file first and instantly has a model of the entire vault — titles, topics, structure, and the link graph — for the cost of a single tool call.

The full build — the complete script, how link resolution works, and the “index-first” rule that makes the agent use it — is its own hands-on article:

Why we build our own

The decision comes straight from how the whole setup is meant to work: a local agent that consults the vault on demand, with no recurring cost and nothing leaving the machine. Measured against that, the own indexer wins on the things that matter here:

• Free and local. No embeddings API, no per-query cost, no vault text shipped to a third party. It runs on the Python you already set up for the agent.
• Token-light and agent-native. The agent reads one compact JSON map, not a pile of note bodies — the single cheapest read replaces dozens of expensive ones.
• Transparent and yours. It’s a short script you can read end to end. When something’s off, you fix it — you’re never waiting on a plugin author or debugging someone else’s server.
• Zero lock-in. The output is plain JSON. Any tool, bot, or future agent you point at the file can use it. Nothing is tied to a vendor or a protocol.

That’s the same reasoning behind everything else in this series: if a thing only works because you rented it, you don’t really own your setup. A twelve-kilobyte JSON file you generate yourself is about as owned as it gets.

When a plugin or MCP server is the better call

Fairness matters, so here’s the flip side. Our approach has one genuine weakness — it’s keyword, not semantic. It finds what you literally name, not “what you meant”. If your day-to-day is fuzzy, associative lookup across hundreds of notes (“find the thing about that pricing idea”) and you don’t want to build anything, a semantic tool earns its keep:

• Reach for a community plugin if you mostly work inside Obsidian yourself, want real semantic search with zero code, and don’t mind an embeddings provider seeing your notes.
• Reach for an MCP server if you want an external AI tool to query the vault natively with semantic recall, and you’re comfortable running and maintaining one more process.

Neither is wrong. They optimise for lowest effort and semantic recall; we optimise for control, cost, and privacy. Know which axis you care about and the choice makes itself.

The growth path: semantic without giving up control

The nice part is that choosing our approach doesn’t lock you out of semantic search forever. If keyword-only ever starts to pinch, the upgrade is additive: bolt an optional embeddings layer onto index_vault.py using a small local sentence-transformer model. You keep the structural index, keep it free, keep it private — and gain “match by meaning” on top. You reach for a plugin or an external server only if you decide the extra machinery is worth it, not because you painted yourself into a corner.

Get your agent to build it

You don’t have to write the indexer by hand. This is a perfect task to hand to the agent — describe exactly what you want and let it produce the script, run it, and adopt the index-first habit. Paste this prompt to Hermes:

Build a vault index for my Obsidian knowledge base so you can read it in one
pass instead of crawling every note.

Write a single standard-library Python script called index_vault.py that:
- Walks every .md file in my vault once (skip .obsidian, .git, .trash, _meta,
  node_modules).
- Records per note: title (filename), aliases and tags from frontmatter plus
  inline #tags, headings, outgoing [[wikilinks]], and word count.
- Resolves each wikilink to the real file (matching by filename and alias, the
  way Obsidian does), computes backlinks (who links to this note), and lists any
  unresolved links.
- Writes it all to <vault>/_meta/index.json.
- Uses PyYAML only if it's already installed, otherwise a light fallback parser.
- Finds the vault from a CLI argument, else the OBSIDIAN_VAULT_PATH env var,
  else a sensible fallback.

Then run it with the project's uv Python and show me the note count. From now on,
read _meta/index.json FIRST before opening individual notes. Repeat that
index-first rule back to me and save it to your Obsidian skill.

Have the agent repeat the index-first rule back before you rely on it — the say-confirm-save pattern from the foundation article is what makes the habit stick across sessions.

Frequently asked questions

Is keyword indexing really enough for an AI agent?

For most personal vaults, yes. The agent doesn’t need semantic recall to navigate — it needs a map of titles, tags, and links to decide which notes to open, then it reads those in full and reasons over the actual text. The structural index gives it exactly that map. Semantic search helps most when you can’t remember what a note is called; if that’s your bottleneck, add the embeddings layer.

Do the plugins and our indexer conflict?

No — they’re independent. A plugin builds its own index inside Obsidian; our script writes a separate JSON file. You can run both, though for an agent-driven workflow the one file the agent reads directly is what does the work.

Why not just use an MCP server since we already use MCP elsewhere?

You could. The trade-off is one more process to run and keep healthy, plus usually an embeddings dependency — for a benefit (semantic recall) that a personal vault often doesn’t need yet. We prefer the smallest thing that works, and add machinery only when a real limit forces it.

Does the embeddings upgrade cost money?

Not if you run a local model. A small sentence-transformer runs on your own machine, so you keep the free-and-private property while gaining semantic search. Paid embeddings APIs are faster to set up but reintroduce the cost and privacy trade-offs we were avoiding.

Bottom line

Three routes, one decision axis. Plugins and MCP servers optimise for lowest effort and semantic recall; our own indexer optimises for control, cost, and privacy — and fits a setup you’re meant to own outright. For a local agent that reads the vault on demand, the small Python indexer is the right foundation, with a clean path to add semantic search later if you ever need it.

Ready to build it? The hands-on vault index article has the full script and the index-first rule. This article is part of the Agentic AI series; start with the Obsidian knowledge base article if you haven’t set up a vault yet.