People now ask an assistant before they open a search box. When someone asks ChatGPT or Claude about something I wrote, the answer comes from whatever the model remembers, or from a web search that may or may not land on my pages. I wanted the assistant to be able to ask my site directly and quote the page it used.
The Model Context Protocol (MCP) is the standard way to do that. A site exposes a server with a few tools, and any MCP client can call them. So I built mcp-my-site: a command-line indexer that crawls a website, and a small ASP.NET Core app that serves the result as a remote MCP server. It is MIT licensed and aimed at small sites, under 1,000 pages.
You can try it before reading further. A live instance indexes my own site:
claude mcp add --transport http pournasserian https://ai.pournasserian.com/mcp
There is also a playground, which is an MCP client in the browser, so you can call the tools without installing anything.
The 60-second version
You need the .NET 10 SDK and, for this first run, nothing else. No API key.
dotnet tool install --global mcpmysite
mcpmysite init https://example.com --keyword-only # writes appsettings.json
mcpmysite build # crawls the site into ./index
mcpmysite search "how can I contact you" # checks the index
docker run --rm -p 5000:8080 -v "$PWD/index:/index:ro" -e Index__Path=/index ghcr.io/pournasserian/mcp-my-site:latest
The server is then at http://localhost:5000. /mcp is the endpoint for agents, /playground lets you try it in a browser, /api/search?q=contact returns JSON, and /healthz shows the loaded index. Connect Claude Code with:
claude mcp add --transport http mysite http://localhost:5000/mcp
Without Docker you can run the server from a clone of the repository.
Design choices
Hybrid search, with a fallback. Keyword search matches words. Vector search matches meaning, so a question phrased differently from the page still finds it. The server runs BM25 and vector search over embeddings and fuses the two rankings per query. If the embedding provider is unavailable at query time, it falls back to keyword search and says so in the response. The live demo runs hybrid.
It works without a key. I did not want the first run to start with signing up for something. --keyword-only builds an index with no vectors, contacts no embeddings endpoint and reads no key. You give up matching by meaning: "how do I reach you" will not find a page that only says "contact". When you want that, change one setting, set AI_API_KEY, and rebuild. The embeddings endpoint can be OpenRouter, which is the default, or any OpenAI-compatible one: OpenAI, Azure OpenAI, Gemini, Ollama, a LiteLLM proxy. The key is read from the environment and never written to a file. With the default model, a small site embeds for a fraction of a cent.
Shaped for the ChatGPT connector contract. The server speaks MCP over Streamable HTTP and exposes two tools, search and fetch. They follow the shape ChatGPT connectors expect: every search result has an id, and fetch takes that id and returns the page as text. Other clients can pass a url to fetch instead. The server also sends instructions generated from the site itself, so the model knows what the site is about and when to use the tools. It works with Claude Code, Claude Desktop, VS Code, Cursor and the MCP Inspector.
One site per server. There is no multi-tenancy and no admin UI. Configuration is a file, appsettings.json. The indexer extracts the main content of each page automatically, so there are no CSS selectors to maintain. The server polls the index source and swaps in a new build without a restart, so updating means building in CI and publishing the index folder to a URL the server polls.
A search API for your own site. The same search code backs GET /api/search, with CORS, rate limiting and an OpenAPI document. The user guide has a drop-in snippet for a search box.
What it does not do
It indexes plain HTML pages reachable over HTTP, or a folder of built HTML. JavaScript-rendered sites, PDFs and pages behind a login are out of scope. A build is always a full rebuild, though unchanged chunks are not re-embedded.
Links
- Repository: https://github.com/pournasserian/mcp-my-site
- Live demo: https://ai.pournasserian.com (playground at
/playground, API reference at/docs) - NuGet: https://www.nuget.org/packages/mcpmysite
- Docker image:
ghcr.io/pournasserian/mcp-my-site
Feedback I would like
Three things would help most. First, whether the quick start worked for you as written, and where it did not. Second, how it behaves on your site: pages that were extracted badly, or questions that returned the wrong page. The indexer writes a report.json that lists low-text pages, and attaching that to an issue helps. Third, which MCP clients you tried it with, especially ChatGPT connectors and anything I have not listed.
Questions and ideas go to Discussions, bugs to Issues.
Top comments (1)
Since unchanged chunks already skip re-embedding, what's left of a full rebuild is the crawl itself. Reading lastmod from sitemap.xml, or sending If-None-Match with the stored ETag and accepting a 304, would let mcpmysite build skip pages that haven't changed. Most static hosts send ETags, so a rebuild of a small site becomes mostly empty responses.