DEV Community: Tomás Murúa

How to Build a Brand Monitoring Dashboard with SerpApi and Python

Tomás Murúa — Mon, 22 Jun 2026 13:19:43 +0000

Knowing what people say about your product usually means checking Google News, scrolling through YouTube, and digging into different social media threads. That's three tabs, three interfaces, and no way to compare what you find. This tutorial builds a single dashboard that pulls brand mentions from all three sources using Python and SerpApi.

By the end, you'll have a Streamlit app with three tabs, one for news articles, one for YouTube videos, and one for social media and forum discussions. We'll use "serpapi" as the search query, but you can swap the brand or product name.

Brand monitoring dashboard showing metrics row with total mentions, news articles, YouTube videos, and perspectives counts

Set Up Your Environment

Requirements:

Python 3.8+
SerpApi API Key (the free plan includes 250+ searches/month)
Dependencies (serpapi, pandas, streamlit, altair)

The serpapi package is the official Python SDK. It handles request signing, retries, and response parsing.

The complete code, including a Jupyter notebook version, is available in the SerpApi tutorials repository.

tutorials/python_projects/brand-monitoring-dashboard at master · serpapi/tutorials

The Pipeline

The app follows the same three-step pattern from the GitHub Issues dashboard, fetch raw data, transform it, and display the analysis.

Pipeline diagram showing three stages: fetch, transform, and display

The difference this time is three separate engines running in parallel. Each returns a different response structure, so the transform step normalizes everything into DataFrames before the dashboard consumes it.

Fetch the Data

A single SerpApi client instance works for all three engines:

import serpapi
import os

SERPAPI_KEY = os.environ.get("SERPAPI_KEY", "")
client = serpapi.Client(api_key=SERPAPI_KEY)

Google News

The Google News API returns articles through the news_results key. Each result includes title, link, source (a dict with name and icon), date, and snippet.

def fetch_news(client, brand):
    """Fetch news articles mentioning the brand via Google News."""
    results = client.search({
        "engine": "google_news",
        "q": brand,
        "gl": "us",
        "hl": "en",
    })
    return results.get("news_results", [])

For more use cases with this engine, refer to the news monitoring.

YouTube

The YouTube Search API uses search_query instead of q, and the sp parameter controls time filters. The values EgIIAw%3D%3D (this week) and EgIIBA%3D%3D (this month) are YouTube's internal encoding for upload date filters. You can grab these from YouTube's URL bar after applying a filter manually.

We run both filters and deduplicate by link, since the month results include everything from the week:

YT_FILTER_WEEK = "EgIIAw%3D%3D"
YT_FILTER_MONTH = "EgIIBA%3D%3D"

def fetch_youtube(client, brand):
    """Fetch YouTube videos, combining week and month filters."""
    seen = set()
    videos = []

    for sp_filter in (YT_FILTER_WEEK, YT_FILTER_MONTH):
        results = client.search({
            "engine": "youtube",
            "search_query": brand,
            "sp": sp_filter,
        })
        for video in results.get("video_results", []):
            link = video.get("link", "")
            if link and link not in seen:
                seen.add(link)
                videos.append(video)

    return videos

For more examples using the YouTube API, refer to this link.

Google Perspectives

Google Perspectives API surfaces user-generated content from LinkedIn, Reddit, Quora, and blogs. It uses the standard Google engine, and the results appear under the perspectives key:

SerpApi search with the perspectives result

def fetch_perspectives(client, brand):
    """Fetch user-generated content (Reddit, LinkedIn, Quora)."""
    results = client.search({
        "engine": "google",
        "q": brand,
        "google_domain": "google.com",
    })
    return results.get("perspectives", [])

Fetch in Parallel

Three sequential API calls take roughly three seconds. Running them in parallel with Python ThreadPoolExecutor brings that down to about one second. Each call runs in its own thread while the others wait for their response:

from concurrent.futures import ThreadPoolExecutor

@st.cache_data(ttl=300)
def fetch_all_mentions(brand):
    """Fetch all brand mentions from three engines in parallel."""
    client = serpapi.Client(api_key=SERPAPI_KEY)

    with ThreadPoolExecutor(max_workers=3) as pool:
        news_future = pool.submit(fetch_news, client, brand)
        yt_future = pool.submit(fetch_youtube, client, brand)
        persp_future = pool.submit(fetch_perspectives, client, brand)
        return news_future.result(), yt_future.result(), persp_future.result()

SerpApi also offers a server-side async parameter for large-scale batch processing, where you submit searches and retrieve results later. For our three concurrent calls, client-side threading is simpler and equally effective.

The @st.cache_data(ttl=300) decorator caches results for 5 minutes. Without it, every Streamlit interaction would re-trigger the API calls. This works alongside SerpApi's own 1-hour result cache, which serves identical queries from the cache at no extra search cost unless you explicitly pass no_cache=true. Together, these two layers minimize redundant API calls during development and testing.

For more optimization techniques when working with SerpApi at scale, refer to this blog.

Transform the Data

All three engines return dates as relative strings ("3 hours ago", "2 days ago"). We need a shared parser to convert them into datetime objects for sorting.

Parse Relative Dates

Two details worth noting. The regex is compiled once and reused since this function runs for every result in all three engines. And the fallback returns datetime.now(timezone.utc) instead of None, so results without a parseable date sort to the top rather than breaking pandas operations.

import re
from datetime import datetime, timedelta, timezone

RELATIVE_DATE_RE = re.compile(
    r"(\d+)\s+(second|minute|hour|day|week|month|year)s?\s+ago", re.IGNORECASE
)

UNIT_TO_TIMEDELTA = {
    "second": lambda n: timedelta(seconds=n),
    "minute": lambda n: timedelta(minutes=n),
    "hour": lambda n: timedelta(hours=n),
    "day": lambda n: timedelta(days=n),
    "week": lambda n: timedelta(weeks=n),
    "month": lambda n: timedelta(days=n * 30),
    "year": lambda n: timedelta(days=n * 365),
}

def parse_relative_date(text):
    """Convert '3 hours ago' into a datetime object."""
    if not text:
        return datetime.now(timezone.utc)

    match = RELATIVE_DATE_RE.search(str(text))
    if not match:
        return datetime.now(timezone.utc)

    amount = int(match.group(1))
    unit = match.group(2).lower()
    delta = UNIT_TO_TIMEDELTA.get(unit, lambda n: timedelta())(amount)
    return datetime.now(timezone.utc) - delta

Build DataFrames

Each engine gets into its own transformer. Here's the news version:

def transform_news(results):
    """Convert raw Google News results into structured records."""
    records = []
    for item in results:
        source = item.get("source") or {}
        source_name = source.get("name", "Unknown") if isinstance(source, dict) else str(source)

        records.append({
            "title": item.get("title", ""),
            "link": item.get("link", ""),
            "source": source_name,
            "date": parse_relative_date(item.get("date", "")),
            "snippet": item.get("snippet", ""),
        })
    return records

The source field can be a dict or a plain string depending on the result, so the isinstace check handles both. YouTube and Perspectives follow the same pattern, with two differences worth highlighting.

YouTube views come back as strings like "1,234 views", so we strip non-numeric characters before converting:

views = item.get("views") or 0
if isinstance(views, str):
    views = int(re.sub(r"[^\d]", "", views) or 0)

Build the Dashboard

The Streamlit interface starts with a form for the brand query and a row of summary metrics across all three sources:

st.set_page_config(page_title="Brand Monitoring Dashboard", layout="wide")
st.title("Brand Monitoring Dashboard")

with st.form("brand_form"):
    brand = st.text_input("Brand or keyword to monitor", value="serpapi")
    submitted = st.form_submit_button("Search")

Brand or keyword selector to monitor

After fetching, the dashboard shows four metrics at the top for a quick overview, then splits into three tabs:

col1, col2, col3, col4 = st.columns(4)
col1.metric("Total Mentions", total_mentions)
col2.metric("News Articles", len(news_records))
col3.metric("YouTube Videos", len(yt_records))
col4.metric("Perspectives", len(persp_records))

Dashboard metrics row displaying total mentions across three sources

News Tab

The News tab pairs an Altair bar chart of top sources with a sortable table. Altair ships with Streamlit, so there's nothing extra to install. We use it instead of st.bar_chart because it gives control over orientation, tooltips, and styling.

source_df = news_df["source"].value_counts().head(10).reset_index()
source_df.columns = ["source", "count"]

source_chart = alt.Chart(source_df).mark_bar(
    cornerRadiusTopRight=4, cornerRadiusBottomRight=4
).encode(
    x=alt.X("count:Q", title="Articles"),
    y=alt.Y("source:N", sort="-x", title=""),
    color=alt.value("#4A90D9"),
    tooltip=["source:N", "count:Q"],
).properties(height=350)
st.altair_chart(source_chart, use_container_width=True)

News tab with horizontal bar chart of top sources and sortable article table

The table uses st.column_config.LinkColumn so each article title links directly to its source.

YouTube Tab

The YouTube tab shows views by channel and a sorted video table. The chart groups views by channel to surface which creators talk about the brand the most.

channel_df = yt_df.groupby("channel")["views"].sum().reset_index()
channel_df = channel_df.sort_values("views", ascending=False).head(10)

channel_chart = alt.Chart(channel_df).mark_bar(
    cornerRadiusTopRight=4, cornerRadiusBottomRight=4
).encode(
    x=alt.X("views:Q", title="Views", axis=alt.Axis(format="~s")),
    y=alt.Y("channel:N", sort="-x", title=""),
    color=alt.value("#4A90D9"),
    tooltip=["channel:N", alt.Tooltip("views:Q", format=",")],
).properties(height=350)

YouTube tab showing views by channel chart and video table

Perspectives Tab

The Perspectives tab splits the layout between a discussion table on the left, and a donut chart of mentions by platform on the right. The donut chart makes it easy to see where conversations happen, whether it's LinkedIn, Reddit, X, etc.

platform_chart = alt.Chart(platform_df).mark_arc(
    innerRadius=60, outerRadius=120
).encode(
    theta=alt.Theta("count:Q"),
    color=alt.Color("source:N", legend=alt.Legend(title="Platform")),
    tooltip=["source:N", "count:Q"],
).properties(height=350)

Perspectives tab with discussions table on the left and donut chart of mentions by platform on the right

When to use this approach

Ideal for:

Tracking brand mentions across news, video, and social in one view
Monitoring product launches, PR campaigns, or competitor names
Building internal dashboard for marketing or DevRel teams

Not recommended for:

Real-time alerting. The API returns a snapshot, not a stream. For notifications, schedule the script on an interval and compare results.
Historical analysis. Each engine returns recent results, not a complete archive.

If you want to explore the API response before writing code, the SerpApi Playground lets you test any engine interactively. And if you only need news coverage, the Google News API alone handles most brand monitoring use cases.

Where to Go from Here

This dashboard gives you a live snapshot. The natural next step is turning it into a historical record. Store each fetch in a database (SQLite, PostgreSQL, or even a CSV) and you can compare mention volume week over week, track which sources cover your brand consistently, and spot trends that a single snapshot can't show.

With historical data in place, you can layer on more analysis. Identify content gaps by looking at what topics competitors get covered on but you don't. Track which YouTube channels mention your product and how their view counts trend over time. Flag new platforms or authors that start discussing your brand.

The data is yours to work with however fits your needs. The three engines give you the raw material; what you build on top depends on the questions you're trying to answer.

Conclusion

The full application is about 350 lines in a single Python file. Three API calls, three DataFrames, three tabs. The query input at the top lets you switch brands without changing the code.

What started as a way to check where "serpapi" shows up on the web became a tool that surfaces patterns you miss manually. The Perspectives tab pulls in LinkedIn posts, Reddit threads, and Quora answers that don't appear in regular news or video searches, and combining them in one view gives you the full picture.

How to Analyze Issues with the GitHub API and Python

Tomás Murúa — Fri, 19 Jun 2026 13:50:37 +0000

If you maintain an open-source project or track bugs in a public repository, you know how fast GitHub issues can pile up. This tutorial shows you how to pull issue data from the GitHub REST API, transform it into structured metrics, and visualize trends.

By the end, you'll have a Streamlit dashboard that shows open vs closed counts, issue aging, service breakdown, and trends over time. The entire app is a single Python file; I built it against the SerpApi Public Roadmap repository, which has over 3,600 issues, but it works with any public GitHub repo.

Streamlit dashboard analyzing 1,900+ GitHub issues from serpapi/public-roadmap

Set Up Your Environment

We'll use Streamlit, a Python framework that turns scripts into interactive web apps with minimal code. You need:

Python 3.8+
GitHub personal access token
Dependencies (pandas, streamlit, requests)

GitHub recommends the fine-grained tokens over the classic token. Create one with "public repositories" selected under repository access.

Export your GitHub token and run the app:

export GITHUB_TOKEN="your_token_here"
streamlit run app.py

You can find the complete app and notebook in the GitHub repository. The tutorial below walks through each piece of the code.

tutorials/python_projects/github-issues-dashboard at master · serpapi/tutorials

The Pipeline

The app follows three steps:

Fetch the raw issues from GitHub.
Transform them into structured data.
Display the analysis.

Fetch-Transform-Analyze pipeline for GitHub issue data

How to Fetch the Issues

The GitHub REST API exposes issues through GET /repos/{owner}/{repo}/issues. Each request returns up to 100 issues, so repositories with more require pagination.

Here's the fetch function. Two things to note in the script:

The pull_request check filters out PRs (GitHub's Issues endpoint returns both).
The 422 check handles a pagination limit we'll explain below.

def fetch_issues_by_state(owner, repo, state, headers):
    """Fetch all issues for a given state (open/closed), handling pagination."""
    issues = []
    page = 1

    while True:
        response = requests.get(
            f"{BASE_URL}/repos/{owner}/{repo}/issues",
            headers=headers,
            params={"state": state, "per_page": 100, "page": page},
        )

        # GitHub returns 422 when pagination exceeds ~1,000 results
        if response.status_code == 422:
            break

        # Rate limit exceeded
        if response.status_code in (403, 429):
            raise Exception(
                "GitHub API rate limit exceeded. "
                "Wait a few minutes and try again, or check your GITHUB_TOKEN."
            )

        response.raise_for_status()
        data = response.json()

        if not data:
            break

        for issue in data:
            # The Issues endpoint returns PRs too; filter them out
            if "pull_request" not in issue:
                issues.append(issue)

        page += 1

    return issues

Pagination Limit

When I first tested this script, I used state=all to fetch everything in one loop. It worked until page 11, when GitHub returned a 422 Unprocessable Entity. GitHub cap pagination at roughly 1,000 results per query. Page 11 at 100 per page exceeds that limit.

Each query stays under the 1,000-result cap. This gives us roughly 1,900 issues total, enough for a meaningful analysis.

The @st.cache_data(ttl=300) decorator is specific to Streamlit. It caches the return value for 5 minutes. Without it, Streamlit re-executes the entire script on every user interaction. A single click would trigger 20+ API requests.

The SerpApi public roadmap repository has over 3,600 issues. A single query can't reach them all. The fix is to split into two separate queries:

@st.cache_data(ttl=300)
def fetch_all_issues(owner, repo):
    """Fetch all issues from the repo, excluding pull requests."""
    headers = {}
    if GITHUB_TOKEN:
        headers["Authorization"] = f"Bearer {GITHUB_TOKEN}"

    open_issues = fetch_issues_by_state(owner, repo, "open", headers)
    closed_issues = fetch_issues_by_state(owner, repo, "closed", headers)

    return open_issues + closed_issues

Rate limits: without a token, you get 60 requests per hour. With a personal access token, 5,000. For this app, always authenticate. For more details, refer to the rate limit docs.

Transform the Issues

The raw API response has dozens of fields per issue. Most are irrelevant for analysis. The transformation extracts what matters and processes the derived metrics.

Here's what a single issue looks like when we fetch it from the API:

{
    "number": 3654,
    "title": "[Google search API] - shows only images, FAQ, AI snippet or video on a valid
  search (Location error)",
    "state": "closed",
    "created_at": "2026-03-26T14:03:30Z",
    "closed_at": "2026-03-26T14:50:58Z",
    "labels": [
      {
        "name": "type: bug",
        "color": "FEF2C0",
        "description": "Something is broken"
      }
    ],
    "user": {
      "login": "annieiscorrie"
    },
    "comments": 4,
    "html_url": "https://github.com/serpapi/public-roadmap/issues/3654",
    "pull_request": null
  }

The basic fields (number, title, state, created_at) come straight from the API response. The interesting part is these specific fields:

age_days: days since creation. This is the metric that surfaces forgotten issues. Sorting by age immediately shows what's been sitting unresolved the longest.
service: the serpapi/public-roadmap repository encodes the affected component in the title using square brackets, like [Google Search API] Results missing pagination becomes "Google Search API". The regex \[(.+?)\] captures the text inside the first pair of brackets.

Not every repository uses brackets in titles, the other commons alternatives include label prefixes area/component/, milestones, or GitHub projects. Adjust the services extraction in transform_issue to match your repo's conventions.

status and type: parsed from labels with status: and type: prefixes. Check your target repository's labels before assuming a format.

def transform_issue(issue):
    """Convert a raw GitHub issue dict into a structured record."""
    created = datetime.fromisoformat(issue["created_at"].replace("Z", "+00:00"))
    age_days = (datetime.now(timezone.utc) - created).days
    labels = [label["name"] for label in issue.get("labels", [])]

    # Extract service from title prefix like "[Google Search]"
    match = re.search(r"\[(.+?)\]", issue["title"])
    service = match.group(1) if match else "General"

    # Extract status and type from label prefixes
    status = next(
        (l.split(": ", 1)[1] for l in labels if l.startswith("status:")), "none"
    )
    type_ = next(
        (l.split(": ", 1)[1] for l in labels if l.startswith("type:")), "none"
    )

    return {
        "number": issue["number"],
        "title": issue["title"],
        "state": issue["state"],
        "created_at": created.strftime("%Y-%m-%d"),
        "age_days": age_days,
        "labels": labels,
        "service": service,
        "status": status,
        "type": type_,
    }

Analyze the Issues

With the data transformed into a pandas DataFrame, Streamlit handles the display. The dashboard starts with a repository selector so you can analyze any public repo without changing the code:

with st.form("repo_form"):
    col_owner, col_repo, col_btn = st.columns([2, 2, 1])
    with col_owner:
        owner = st.text_input("Owner", value="serpapi")
    with col_repo:
        repo = st.text_input("Repository", value="public-roadmap")
    with col_btn:
        submitted = st.form_submit_button("Fetch Issues")

Repository selector for switching between public GitHub repos

After submitting, the dashboard displays four sections.

Display a Metrics Summary

Three numbers at the top for a quick overview:

col1, col2, col3 = st.columns(3)
col1.metric("Total Issues", total)
col2.metric("Open", open_count)
col3.metric("Closed", closed_count)

Summary metrics: total, open, and closed issue counts

Chart Status and Type Distribution

A bar chart for statuses and a donut for types side by side. We use Altair instead of Streamlit's built-in st.bar_chart because it gives control over chart type, orientation, tooltips, and styling. Altair ships with Streamlit, so there's nothing extra to install.

col_left, col_right = st.columns(2)

with col_left:
    st.subheader("Issues by Status")
    status_df = df["status"].value_counts().reset_index()
    status_df.columns = ["status", "count"]

    status_chart = (
        alt.Chart(status_df)
        .mark_bar(cornerRadiusTopLeft=4, cornerRadiusTopRight=4)
        .encode(
            x=alt.X("status:N", sort="-y", title=""),
            y=alt.Y("count:Q", title="Issues"),
            color=alt.value("#4A90D9"),
        )
        .properties(height=350)
    )
    st.altair_chart(status_chart, use_container_width=True)

Issues by status (bar chart) and by type (donut chart) using Altair

Track Issues Over Time

An area chart showing issues opened per month, split by state:

df["month"] = pd.to_datetime(df["created_at"]).dt.to_period("M").astype(str)
monthly_state = df.groupby(["month", "state"]).size().reset_index(name="count")

timeline = (
    alt.Chart(monthly_state)
    .mark_area(opacity=0.6)
    .encode(
        x=alt.X("month:N", title=""),
        y=alt.Y("count:Q", title="Issues"),
        color=alt.Color("state:N"),
        tooltip=["month:N", "state:N", "count:Q"],
    )
    .properties(height=350)
)
st.altair_chart(timeline, use_container_width=True)

Monthly issue activity split by open and closed state

Compare Services and Measure Aging

The last row pairs two charts, one with services that have the most issues and the other with how old the open ones are. And for aging, we bucket open issues into time ranges:

def age_bucket(days):
    if days < 7:
        return "< 7 days"
    elif days < 30:
        return "7-30 days"
    elif days < 90:
        return "30-90 days"
    elif days < 180:
        return "90-180 days"
    elif days < 365:
        return "180-365 days"
    else:
        return "> 365 days"

Top 15 services by issue count and open issue aging distribution

Dashboard Complete View

Here's the full dashboard running against serpapi/public-roadmap. The repository selector at the top lets you switch to any public repo. All five sections update together when you fetch a new repository.

Complete dashboard view running against serpapi/public-roadmap

When to Use This Approach

Ideal for:

Track the health of open-source projects you depend on or maintain.
Building internal dashboards for product roadmap visibility.

Not recommended for:

Repositories with fewer than 50 issues, because the manual review will be faster.
Real-time monitoring and the API rate limits make the polling impractical.
Fetching complete history from repositories with 10,000+ issues.

Conclusion

The full application is 220 lines in a single Python file. The dashboard includes a repository selector, so you can point it at any public GitHub repository without touching any code, and everything runs locally, so you have control over everything.

What started as a quick experiment to understand a public roadmap became a tool I use daily. The GitHub API gives everything you need; the work is in structuring the data to answer the question that the web interface doesn't provide.

How to Connect Your Local LLM with Web Search Data

Tomás Murúa — Thu, 04 Jun 2026 05:48:28 +0000

This blog will show you how to give a local LLM access to real-time web data. We'll build a small chat app in TypeScript; the model will decide what to search for, your code will run the search, and the model will write an answer with the fresh data in hand. The inference runs on your hardware, and the only thing that leaves your local environment is the search query the model chose to issue.

By the end of this blog, you'll understand the full request flow, you'll know how to keep a large search response from blowing past your context window, and you'll have a public repository you can clone and run against your own local model.

Why LLMs Don't Have Access to Real-Time Data

Every large language model is limited by its training data cutoff. Anything that happened after that date is invisible to the model. That's true whether you run the model locally or call a hosted API.

Cloud chat products hide this from most users. ChatGPT, Claude, and Gemini ship the model wrapped in a product, and that product includes a built-in search tool. When you ask ChatGPT, "What's the stock price of Nvidia right now?" the model doesn't answer on its own; the product called a search tool, fed the result back to the model, and the model wrote the up-to-date reply.

The moment you skip that product and run a model locally, none of it is wired up. You get the model on its own; no tools, no search, no real-time data. The fix is what the big AI labs already do under the hood. Give the model a tool and let it decide when to call it.

The tool, in our case, is a web search API; one API call returns structured search results instead of raw HTML. We'll use SerpApi, which returns clean JSON across Google Search, Finance, News, Maps, Flights, and 100+ other search engines.

Run a Local Model with LM Studio

There are different ways to use LLMs locally, but in this guide, we're going to use LM Studio. This is a desktop app for downloading and running open-weight models on your own hardware. It matters for this project for one reason; it exposes the model through OpenAI and Anthropic standards on localhost. You point the SDK you already use at that endpoint, and the same code that talks to GPT or Claude now talks to your local model.

Here's how to get a model running locally in a few minutes:

Download and install LM Studio from lmstudio.ai.
Open the model browser and download a model with native tool calling. I'm using Qwen3.5-9B, which runs comfortably on an M4 Max with 36 GB of unified memory.

Open the Developer tab, load the model, and click Start Server.

LM Studio now serves an OpenAI-compatible endpoint at http://localhost:1234/v1.

The server has to stay running for the rest of this guide; that local endpoint is what our code talks to. It is not necessary to pre-load the model because the app requests it on demand, so LM Studio loads it only when a call actually comes in.

Any model with tool calling works here, so favor one built for agentic workflows; that realibility is what the whole loop leans on. Google's Gemma 4 is another strong option. Which mode and size you pick comes down to your hardware; it has to fit in memory alongside a context window large enough to hold the search result.
If you're not sure what your hardware can handle, LM Studio's model search flags the best match for your hardware and warns you when a model is too large to run. I chose Qwen3.5-9B because it offered a good balance between speed and the quality of its responses.

Build the Chat App

We'll build a chat app that wraps a local model in the same tool-calling loop that the cloud products use. The loop is the back-and-forth between the model and your code: the model asks for a search, your code runs it, and the model writes the answer.

The full project is a TypeScript app you can clone from:

GitHub Repository

Currently, it supports five SerpApi engines (Search, Finance, News, Maps, and Flights), and you can add more. The architecture has three moving parts:

The third part is the one most tutorials skip, and it's the one that makes a local setup actually work: a search response is far too big to fit in a model with a small context window. We'll fix that with a single SerpApi parameter called JSON Restrictor.

Set Up the Project

Clone the repo, install dependencies, and grab an API key from your SerpApi dashboard:

git clone https://github.com/serpapi/local-llm-web-search.git
cd local-llm-web-search
bun install
echo "SERPAPI_API_KEY=your_key_here" > .env

Make sure that the server is running in LM Studio, then start the app.

bun dev

With the local server running and the key in place, you're ready to walk through how the loop works.

How the Loop Works

The pattern is the one OpenAI documents for function calling and Anthropic documents for tool use. Five steps, in order. The code below is lifted from the app's agent.ts and restrictors.ts, trimmed to the essentials.

Step 1. Declare the Tools to the Model

You hand the model a list of tools it can call. Each tool has a name, a description, and a JSON Schema for its parameters. Here is an example of declaring SerpApi as the web search tool the model can reach for:

import OpenAI from "openai"

const TOOLS: OpenAI.Chat.Completions.ChatCompletionTool[] = [
  {
    type: "function",
    function: {
      name: "google_search",
      description:
        "Search Google for current web results. Use for news, facts, or anything time-sensitive.",
      parameters: {
        type: "object",
        properties: {
          query: { type: "string", description: "What to search for" },
          gl: { type: "string", description: "Two-letter country code, e.g. 'us'" },
          hl: { type: "string", description: "Two-letter language code, e.g. 'en'" },
        },
        required: ["query"],
      },
    },
  },
  {
    type: "function",
    function: {
      name: "google_flights_search",
      description: "Find flight options between two airports for a given date.",
      parameters: {
        type: "object",
        properties: {
          departure_id: { type: "string", description: "IATA code, e.g. 'JFK'" },
          arrival_id: { type: "string", description: "IATA code, e.g. 'LAX'" },
          outbound_date: { type: "string", description: "YYYY-MM-DD" },
          return_date: { type: "string", description: "YYYY-MM-DD, round-trip" },
        },
        required: ["departure_id", "arrival_id", "outbound_date"],
      },
    },
  },
  // ...one entry per SerpApi engine you want to support
]

A few things worth noting:

The descriptions matter more than they look; the model picks tools by reading them, so write each one like documentation for a developer who has never seen your code before.
Each tool name maps to one SerpApi engine. The app supports google_search, google_finance_search, google_news_search, google_maps_search, and google_flights_search.
These definitions go to the model on every request, so keep them lean. They count against the same context window budget as everything else.

Step 2. Ask the Model What It Needs

You send the user's prompt to the model along with the tools list. The model returns one of two things:

A direct answer when it already knows
Tools calls when it needs the web data

Point the OpenAI client at the LM Studio server you started earlier:

const client = new OpenAI({
  baseURL: "http://localhost:1234/v1", // LM Studio's OpenAI-compatible endpoint
  apiKey: "lm-studio",                 // any non-empty string works locally
})

const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
  { role: "system", content: "You are an assistant with access to web-search tools." },
  { role: "user", content: userPrompt },
]

const first = await client.chat.completions.create({
  model,
  messages,
  tools: TOOLS,
  tool_choice: "auto",
  temperature: 0.1,
})

A few things worth noting:

The LM Studio server must be running in the background for this call to connect; the model itself loads on demand when the request arrives. If you get a connection refused error, that's the first thing to check.
tool_choice: "auto" lets the model decide between answering directly and calling tools. A low temperature (0.1) keeps tool selection deterministic; we'll raise it for the second call when the model writes the answer.
If first.choices[0].message.tool_calls is non-empty, the model wants the web, and you have work to do.
Before this call, the app trims the conversation history with a sliding window, dropping the oldest turns so a long chat doesn't overflow the budget. json_restrictor keeps each response small; the window keeps the cumulative history small.

Step 3. Run the Tool Calls

For each tool call, validate the arguments and run the matching SerpApi engine. The model can pick more than one tool in the same turn. For example, "Compare Apple and Tesla stock" is going to trigger two finance lookups, so run them concurrently:

const toolCalls = first.choices[0].message.tool_calls ?? []

const results = await Promise.all(
  toolCalls.map(async (call) => {
    const args = validate(call.function.name, JSON.parse(call.function.arguments))
    const data = await executeTool(call.function.name, args)
    return { call_id: call.id, data }
  }),
)

A few things worth noting:

validate runs the arguments through a Zod schema before any network call. Smaller models occasionally emit malformed JSON in arguments, and the check catches it before you spend a SerpApi search on garbage input.
executeTool is where the response gets trimmed. That's the next step, and it's the part that makes the whole thing viable on a local model.

Step 4. Trim the Response with the JSON Restrictor

SerpApi returns everything an engine knows. A single Google Flights response can return over 14,000 tokens once airport metadata, every price tier, layover details, carbon estimates, and baggage policies come along. A Google Finance response can top 46,000 tokens. A Qwen3.5-9B running with a 32K context window can't carry that, especially once you add the system prompt, the conversation history, and the tool definitions themselves.

SerpApi solves this for you with the JSON Restrictor parameter. You pass a field-selector string with the request, and SerpApi filters the response on its servers before sending it back. The trimming happens before the data crosses the network, so the giant payload never reaches your machine at all. Applying it is one line:

import { getJson } from "serpapi"

// Run a tool's SerpApi call with its server-side restrictor applied.
function executeTool(name: RestrictorTool, args: ToolArgs): Promise<SerpApiJson> {
  return getJson({ ...toolParams(name, args), json_restrictor: RESTRICTORS[name] })
}

The restrictor strings live in their own file, one per engine. The syntax is jq-like: .field selects an object field, [] selects every array item, [0:5] takes a slice, and .{a, b} projects multiple fields at once:

export const RESTRICTORS: Record<RestrictorTool, string> = {
  google_search: [
    "answer_box.{answer,snippet,title}",
    "knowledge_graph.{title,type,description,source.{name,link}}",
    "organic_results[0:5].{title,link,displayed_link,snippet,date}",
  ].join(","),

  google_flights_search: [
    "best_flights[0:3].{price,total_duration,type,flights[].{airline,departure_airport.{id,time},arrival_airport.{id,time},duration},layovers[].{name,duration}}",
    "other_flights[0:3].{price,total_duration,type,flights[].{airline,departure_airport.{id,time},arrival_airport.{id,time},duration}}",
  ].join(","),

  // ...one restrictor per engine
}

A few things worth noting:

Fields absent from a given response are silently skipped, so one restrictor can list every branch it might want (an answer_box for factual queries, a knowledge_graph for entities, organic_results as the fallback) without any conditional logic.
json_restrictor works on every SerpApi engine, so the pattern is the same whether you're filtering flights, finance, or maps.
No client-side formatting is needed; the response arrives already shaped for the model.

Here's an example of the before and after from the prompt "What's the current price of NVDA?":

The token reduction is the whole point. These are measured on real queries from the app's benchmark:

Step 5. Send the Second Call and Get the Answer

Push the restricted results back into the conversation and send a second call. The model writes the final answer using the trimmed data:

messages.push(first.choices[0].message) // the assistant's tool_calls turn
for (const { call_id, data } of results) {
  messages.push({
    role: "tool",
    tool_call_id: call_id,
    content: JSON.stringify(data),
  })
}

const second = await client.chat.completions.create({
  model,
  messages,
  temperature: 0.3,
})

return second.choices[0].message.content

A few things worth noting:

Send the assistant's original tool-call message before the role: "tool" results, in that order. The model needs the call context to bind each result back to its tool_call_id.
Temperature climbs from 0.1 to 0.3 here; this is a prose-writing step, not a tool-selection step.
Every tool result must be present before the model can answer. Miss one and the second call fails.

A Full Run: Flights from JFK to LAX

Here's the loop end-to-end with a single prompt:

The model parses the question, picks google_flights_search, and emits one tool call with departure_id: "JFK", arrival_id: "LAX", and a date. The app validates the arguments, then calls SerpApi with the flights restrictor attached. SerpApi returns roughly 800 tokens of pre-filtered JSON instead of the 14,000+ it would otherwise send. The app pushes that into the message history and sends the second call.

The model writes a three-itinerary comparison with airline, price, schedule, duration, and stops. On an M4 Max running Qwen3.5-9B, the full turn (pick tool, fetch, write) lands in a handful of seconds, with the SerpApi call completing well inside the inference time.

One Catch: Open Models Sometimes Emit Tool Calls as Text

The OpenAI spec defines a structured tool_calls array on the model's response. Qwen3.5, gpt-oss, and several other open models will sometimes ignore the array and emit tool calls inline as <tool_call>...</tool_call> text inside message.content instead. Your code has to parse both shapes.

The fix is a fallback that runs whenever the structured array comes back empty:

function parseInlineToolCalls(content: string) {
  const calls = []
  for (const match of content.matchAll(/<tool_call>\s*([\s\S]*?)\s*<\/tool_call>/gi)) {
    try {
      calls.push(JSON.parse(match[1]))
    } catch {
      // skip malformed blocks rather than fail the whole turn
    }
  }
  return calls
}

A few things worth noting:

Run the fallback only when the structured tool_calls array is empty. If both are empty, the model genuinely answered without a tool.
Swallow parse errors per block; one malformed tag shouldn't kill the turn.
Synthesized calls need a `tool_call_id (any unique string works) so you can map results back when you push them into history.

Why Not Use The SerpApi MCP Server?

A reasonable question, since SerpApi also ships an MCP server. Both paths hit the same endpoints; the difference is who drives the loop.

The MCP server is the right transport when the user is the consumer (Claude Desktop, Cursor, or any MCP-compatible client) and the model has a large context window to spare. The client connects to the server and the model calls the tools through it; you don't write the loop yourself.

The direct API path in this guide is the right shape when you're building the agent yourself and your runtime has a tight context budget, which is exactly the case for a local model. You own the loop, so you decide what reaches the model per turn, and json_restrictor lets you make that decision per field. On a 32K context window, that control is the difference between an answer and an overflow.

Where to Go from Here

In this blog we connected a local model to live web data through a five-step loop: declare the tools, ask the model what it needs, run the searches, trim each response with json_restrictor, and send a second call for the answer. The restrictor is doing the quiet, heavy work; without it, a single flights query overflows the context window of a 9B model.

The same loop scales to any SerpApi engine. Add one tool definition and one restrictor string per engine, and the rest of the code stays exactly as it is. Clone the repo, point it at your own local model, and try swapping in an engine the app doesn't cover yet; the shape will already be familiar.