The day the search API market lost half its players
In August 2025, Microsoft killed the Bing Search API. The search API market had two major players. Overnight, it had one.
This wasn't a quiet deprecation notice buried in a changelog. Microsoft announced in May 2025 that Bing Search API would be fully terminated by August. Three months' notice. Thousands of AI applications, RAG pipelines, and third-party search services that depended on Bing's index had a single quarter to find somewhere else to go.
I noticed this shift in my own workflow first. I was debugging why my AI coding agent's web searches returned different results than I expected -- and I realized it wasn't querying Google at all. It was hitting Brave Search. So were Cursor, Claude MCP, and half the tools in my stack. That sent me down a rabbit hole.
Before the shutdown, the market looked like this:
- Google PSE (Programmable Search Engine): $5/1,000 requests, but heavily restricted for AI grounding and training use cases
- Bing Search API: Flexible, widely adopted by AI apps -- but prices had been climbing since Microsoft's OpenAI investment
- Everything else: Scraper APIs that were, structurally, just wrappers around Google or Bing
After the shutdown, the picture was simple. Google still won't open its web index for general AI use. PSE is designed for narrow, site-specific search widgets -- not for powering LLM grounding or RAG applications. Scraper APIs like Tavily and Exa exist, but they carry structural problems: quality instability (they break when the upstream engine changes), legal risk (scraping violates most ToS), privacy leaks (your queries get forwarded to Big Tech), and the ever-present risk of getting blocked.
That left one independent, large-scale web index with a commercial API: Brave Search.
Why Brave Search, specifically?
The thing that separates Brave Search from every other alternative is that it actually owns its index. This isn't a wrapper. It's a full web index built from scratch.
The numbers: 30 billion pages indexed, 100 million pages updated daily, 1.3 billion+ queries per month. Response time under 1 second for 95% of requests.
The technology traces back to Cliqz, a privacy-focused European search engine that shut down in 2020. Its team spun off as Tailcat, which Brave acquired in 2021. From that foundation, Brave built a search engine that doesn't touch Google or Bing infrastructure at any point.
Web Discovery Project: humans curate the index
Here's the part I find most interesting. Brave runs something called the Web Discovery Project. Over 80 million Brave browser users can opt in to anonymously contribute browsing data -- which pages they actually visit and read. This data feeds directly into index freshness and ranking.
Traditional crawlers find pages that are linked to. Web Discovery finds pages that humans actually read. The difference matters: SEO-gamed content that ranks well on Google because of backlink profiles doesn't get the same boost on Brave. Pages with genuine utility do.
This is a different indexing philosophy at its core, and it has real consequences for what AI tools find when they search.
The LLM Context API changes everything
In February 2026, Brave released the LLM Context API, and this is where things get really interesting for engineers.
Traditional search APIs are URL-centric. They return a title, a URL, and a snippet -- designed for a human to click through and read. That's fine for building a search results page. It's terrible for feeding an LLM.
The LLM Context API is data-centric. It's designed from the ground up for LLMs to consume directly. Here's how it works:
flowchart LR
A["Web Search<br/>Brave index identifies<br/>relevant pages"] --> B["Deep Extraction<br/>HTML → structured<br/>content chunks"]
B --> C["Smart Ranking<br/>Chunk-level relevance<br/>scoring"]
C --> D["Compact Output<br/>Token-optimized<br/>compilation"]
Step 1 -- Web Search: Standard search against Brave's independent index to identify the most relevant pages.
Step 2 -- Deep Content Extraction: This is the key innovation. Instead of returning snippets, the API extracts structured content from each page:
- Query-optimized snippets (the most relevant paragraphs)
- JSON-LD and structured data (prioritized extraction)
- Code blocks with context (for technical queries)
- Forum discussions preserving Q&A structure
- YouTube transcript processing
Step 3 -- Smart Chunk Ranking: A dedicated model ranks extracted chunks at the paragraph, table-row, and code-block level -- not at the page level.
Step 4 -- Compact Compilation: Results are compiled into a token-optimized format based on your specified constraints.
Why JSON-LD matters more than ever
Step 2 is where engineers should pay close attention. JSON-LD structured data gets prioritized during extraction. This means schema.org markup on your site directly increases the likelihood that your content gets picked up and cited by AI systems.
Here's a concrete example. If your blog post has this in the <head>:
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "TechArticle",
"headline": "Next.js 15 App Router Migration Guide",
"author": {
"@type": "Person",
"name": "Your Name"
},
"datePublished": "2026-01-15",
"description": "Step-by-step migration from Pages Router to App Router"
}
</script>
The LLM Context API will extract this structured data preferentially and pass it to the LLM as grounding context. Your content becomes machine-readable in a way that plain HTML paragraphs aren't.
This is the single most actionable takeaway in this article. Adding JSON-LD takes 30 minutes. The impact on AI discoverability is disproportionately large.
Data quality beats model size
Brave published benchmark results that should make every engineer rethink their assumptions about AI search.
They evaluated 1,500 queries using Claude Opus 4.5 and Sonnet 4.5 as LLM-as-judge, with pairwise comparisons (both A-vs-B and B-vs-A to control for position bias):
| Service | Avg Score (out of 5) | Win Rate | Loss Rate |
|---|---|---|---|
| Grok | 4.71 | 59.87% | 10.05% |
| Ask Brave (Qwen3) | 4.66 | 49.21% | 15.82% |
| Google AI Mode | 4.39 | 27.07% | 38.17% |
| ChatGPT | 4.32 | 23.87% | 42.22% |
| Perplexity | 4.01 | 10.51% | 64.26% |
Ask Brave uses Qwen3, an open-weight model. It scored 4.66 out of 5, beating ChatGPT (4.32) and Perplexity (4.01).
Read that again. An open-weight model with good search data outperformed frontier models with worse data.
The implication is clear: grounding data quality matters more than model parameters. You can throw the biggest model in the world at a search problem, but if the underlying search index returns low-quality results, the answers will be mediocre. Conversely, a smaller model backed by a high-quality, well-structured index can punch way above its weight.
This flips the conventional wisdom. The AI search race isn't just about who has the best model -- it's about who has the best retrieval pipeline. And retrieval starts with the index.
For content creators, this is encouraging. It means the quality and structure of your content directly influence how well AI systems can use it -- regardless of which model is doing the reasoning. Good structured data on a well-written page will outperform a poorly structured page every time, even if the latter is processed by a more powerful model.
Who's already using Brave Search
This isn't a niche API. The adoption list reads like a directory of tools engineers use every day.
Coding assistants: Cursor, Cline, Windsurf -- three of the most popular AI-powered editors all use Brave Search for web lookups.
AI platforms: OpenClaw (default web search provider), Dify.ai, FlowiseAI.
Enterprise: Snowflake (Cortex Code, Cortex Agents), Chegg (citation search), Turnitin (citation verification).
LLM infrastructure: Anthropic featured Brave Search as one of the first MCP demo servers for Claude. Tens of thousands of developers access Brave Search through Claude MCP integrations.
As of March 2025, Brave Search API had 35,000+ free customers and 2,700+ paid customers. The top 10 AI companies by usage all use Brave Search API for either training or inference.
The point: if you're using AI coding tools, there's a good chance your queries are already hitting Brave's index. Whether your content shows up in those results depends on whether Brave can find and parse it. And if you're building AI tools yourself, Brave Search API is probably the most straightforward path to web search capability right now.
A note on privacy
One thing worth mentioning briefly: Brave owns the entire stack from crawler to API endpoint. When you query a scraper API, your search terms get forwarded to whatever upstream engine powers it -- usually Google or Bing. The scraper can promise zero data retention on their side, but they can't control what happens at the upstream provider.
Brave doesn't have this problem. No third-party intermediary, no query forwarding. For regulated industries (finance, healthcare, legal), this is often the deciding factor.
Try it yourself
curl -s "https://api.search.brave.com/res/v1/web/search?q=your+site+name" \
-H "Accept: application/json" \
-H "X-Subscription-Token: YOUR_API_KEY"
The free tier gives you 2,000 queries per month -- enough to test how your content appears in Brave's index and experiment with the LLM Context API.
What you should do this week
1. Check your site on Brave Search. Go to search.brave.com, search for your site name, your top articles, your product. Compare the results to Google. You might find articles that rank well on Google but are invisible on Brave -- or the reverse. When I first did this, I discovered several of my technical posts were completely absent from Brave's index despite ranking on the first page of Google. Different index, different reality.
2. Add JSON-LD structured data. This is the highest-ROI change you can make for AI discoverability. Use TechArticle for blog posts, SoftwareApplication for tool pages, HowTo for tutorials, or whatever schema.org type fits your content. The LLM Context API prioritizes this data during extraction -- it's not optional anymore.
3. Think multi-index. With Bing gone, the major web indexes are Google, Brave, and Yandex. Google SEO alone won't make your content visible to AI tools that search through Brave's index -- and that list of tools is growing fast. Cursor, Claude MCP, Windsurf, Cline -- these are tools engineers use daily, and they're all powered by Brave.
4. Use clear heading hierarchies and code blocks. The LLM Context API extracts content at the chunk level -- paragraphs, tables, code blocks. Pages with clear h1-h6 structure and properly fenced code examples are easier to extract from, which means they're more likely to surface in AI answers.
Check your site on Brave Search right now. The results might surprise you.
This article is adapted from LLMO Practical Guide: Why ChatGPT Ignores Your Website, a book covering AI search optimization strategies. For the full technical deep-dive on structured data implementation, JSON-LD patterns, and measuring AI search visibility, the book covers all of it.
Top comments (0)