SerpBase + LangChain: a 30-line DocumentLoader that calls Google

Subtitle: A drop-in loader for LangChain that fetches live Google results as LangChain Document objects, ready for any RAG pipeline.

Meta description (~155 chars): A 30-line custom LangChain DocumentLoader for live Google results via SerpBase. Returns Document objects with source, title, rank metadata. Standard library only.

Target length: ~700 words.

Angle fit (per §3.5): #7 "SerpBase + [framework]" — LangChain recipe. Distinct from the on-site "SERP API for AI Agents" post (which is a feature overview) and from the MCP article in this series (which is a different agent stack). This post is a drop-in component.

Suggested target publications (DR 30-70, AI/RAG audience):

• LangChain blog
• LlamaIndex blog
• Pinecone blog (Learn section)
• Weaviate blog
• dev.to (AI / LangChain columns)
• Towards Data Science

Slug convention (for the host): langchain-document-loader-google-serp or serpbase-langchain-recipe if kebab.

---

LangChain ships with 100+ document loaders. Web pages, PDFs, Notion, Slack, GitHub. As of mid-2026 there is still no first-party loader for live Google results, which is exactly what most RAG systems need when the knowledge cutoff is the bottleneck.

This post shows a 30-line custom DocumentLoader that fetches Google results via the SerpBase API and returns them as LangChain Document objects, ready to feed into a splitter, vector store, or retriever.

The loader

import os
import json
import urllib.request
from typing import Iterator
from langchain_core.documents import Document
from langchain_core.document_loaders import BaseLoader

API_KEY = os.environ["SERPBASE_API_KEY"]
BASE = "https://api.serpbase.dev"


class SerpBaseSearchLoader(BaseLoader):
    def __init__(self, query: str, hl: str = "en", gl: str = "us", top_k: int = 5):
        self.query = query
        self.hl = hl
        self.gl = gl
        self.top_k = top_k

    def _search(self) -> dict:
        req = urllib.request.Request(
            f"{BASE}/google/search",
            data=json.dumps({"q": self.query, "hl": self.hl, "gl": self.gl}).encode(),
            headers={"Content-Type": "application/json", "X-API-Key": API_KEY},
            method="POST",
        )
        with urllib.request.urlopen(req, timeout=30) as r:
            return json.loads(r.read())

    def lazy_load(self) -> Iterator[Document]:
        data = self._search()
        for i, r in enumerate(data.get("organic", [])[: self.top_k], 1):
            yield Document(
                page_content=r.get("snippet", ""),
                metadata={
                    "source": r.get("link"),
                    "title": r.get("title"),
                    "rank": i,
                    "query": self.query,
                },
            )


# Usage
loader = SerpBaseSearchLoader("LangChain RAG 2026", top_k=8)
docs = list(loader.lazy_load())

lazy_load is the method LangChain calls when the loader is iterated. Each organic result becomes a Document with page_content set to the snippet and metadata carrying the source URL, title, and rank. Standard library only.

Drop it into a RAG pipeline

from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_chroma import Chroma

splitter = RecursiveCharacterTextSplitter(chunk_size=512, chunk_overlap=64)
splits = splitter.split_documents(docs)

vectorstore = Chroma.from_documents(splits, OpenAIEmbeddings())
retriever = vectorstore.as_retriever(search_kwargs={"k": 4})

Each Document already carries the source URL in metadata["source"], so when the retriever returns chunks to the LLM, the source is preserved end-to-end. Citation at generation time is a one-line addition to the prompt.

Cost math

Workload	Queries / day	Queries / month	Tier	Cost
Personal RAG	30	900	Starter Boost	$3
Team copilot	300	9,000	Starter	$10
Production chat	3,000	90,000	Growth	$50

Standard credits on Starter, Growth, Pro, Business, and Enterprise never expire. The $3 Starter Boost is a 1-month entry pack, available once per account per month.

Why use the loader vs raw curl

Three reasons that matter once you're past the prototype:

1. Standard interface. Anything that takes a BaseLoader (splitters, vector stores, RAG chains, evaluators) works without modification.

2. Metadata preservation. source, title, rank, and query travel with the chunk. Citation at generation time is a one-line addition to the prompt.

3. Composition. You can chain multiple SerpBase loaders for multi-query retrieval — SerpBaseSearchLoader("q1") + SerpBaseSearchLoader("q2") + a web loader — and treat them as one document stream.

What this does not do

• JavaScript-rendered pages. This loader pulls Google SERP data, not arbitrary page content. For page content, pair it with a WebBaseLoader or PlaywrightLoader.
• Multi-hop retrieval. Each loader call is one search. For multi-hop, call the loader multiple times in a chain.
• Knowledge graph entities. The snippet is the page content; the knowledge_graph block from the API response is not extracted into Documents.

Try it

New accounts get 100 free searches on signup, no card. Full endpoint reference is in the SerpBase docs. For agents that don't speak LangChain, the SerpBase agent skill ships a similar pattern as a portable skill.