DEV Community: dodou

Know Your Balance Before You Spend: GET /account/credits in Practice

dodou — Sat, 26 Sep 2026 03:19:06 +0000

Every SERP API bill ends the same way: you find out you're out of credits when a request fails with 1020. But there's a way to see the number before that happens, and it costs nothing to check — GET /account/credits is a documented utility endpoint that doesn't consume search credits at all.

I wired this into my monitoring scripts last month after a nightly job died at 2am on an empty balance. Now the job checks the balance first, alerts before the threshold, and I haven't been surprised since.

What the endpoint gives you

Per the docs, GET /account/credits returns the current balance for the account that owns an active API key. The response separates what you have into:

credits — the total you can spend right now
permanent_credits — the portion with no calendar expiry
expiring_credits — the portion tied to a subscription cycle, with expiring_expires_at as a Unix timestamp in seconds; it's 0 when no limited-time credits are active

That split matters. If your balance looks healthy but most of it is expiring credits, you have a scheduling problem, not a budget problem — use the permanent balance for long-term plans and burn the expiring one first.

Auth is the same as everywhere else: the key goes in the X-API-Key header. URL query parameters are not accepted, so don't try ?api_key=. The endpoint is rate limited to 60 requests per minute per account — plenty for a pre-flight check, and the docs confirm it doesn't consume search credits. Field definitions are in SerpBase's endpoint documentation.

The pre-flight check

import requests

API = "https://api.serpbase.dev"
KEY = "your_api_key"   # your key here

def balance() -> dict:
    resp = requests.get(f"{API}/account/credits",
                        headers={"X-API-Key": KEY}, timeout=15)
    data = resp.json()
    if data.get("status") != 0:
        raise RuntimeError(f"{data.get('status')}: {data.get('error')}")
    return data

if __name__ == "__main__":
    b = balance()
    print(f"total: {b['credits']}")
    print(f"permanent: {b['permanent_credits']}")
    print(f"expiring: {b['expiring_credits']} (expires_at={b['expiring_expires_at']})")

Three ways I use it in practice:

Pre-flight guard. Before a batch job starts, fetch the balance and compare it against len(keywords) * credits_per_request. If the balance is short, alert and skip the run instead of dying halfway through with a partial CSV.
Expiry awareness. When expiring_credits is non-zero, log expiring_expires_at next to the daily run — if you're burning 40 credits a day with 400 expiring, you have ten days to use them, and that's a signal to move monitoring to a weekly cadence rather than buy more.
Post-run reconciliation. After a batch, subtract the run's successful request count from the starting balance. If the numbers don't line up, the request_id on each response is the stable identifier to correlate with support.

FAQ

Does checking the balance really cost nothing? Yes — the docs list it as 0 credits. That's the whole point: you can check as often as you need inside the 60 requests/minute limit.

Why did my balance check fail with 1001? Same as any other endpoint — missing, invalid, or revoked key. The header name is X-API-Key; query-string keys are not accepted.

Can I check the balance from the MCP server? The MCP tools cover the six search endpoints. For balance checks, call this endpoint directly — it's a plain GET with the same key.

How often should a scheduled job check? Once per run is enough for a daily job. If you run hourly batches, once an hour stays far below the 60/min limit.

Add the pre-flight guard to your next batch script tonight — it's four lines, and it turns "why did my job die" into "my job told me it was low before it started".

Google Images via SERP API: Field Quirks and 2-Credit Discipline

dodou — Fri, 25 Sep 2026 01:28:52 +0000

The images endpoint is the one that burns credits fastest — 2 credits per successful request, double the search endpoint — and it's also the one whose response shape will surprise you if you assume it mirrors web search. Two things to know before you write a line of code: the real endpoint path, and which fields you can actually rely on.

I hit both on a product-monitoring script last month. The endpoint path isn't what you'd guess, and title turned out to be missing on half my results. Here's the shape of it.

The endpoint path

It's POST /google/image/search — singular "image", not "images". The docs list six endpoints under /google/ and the images one is the odd one out, so if you're copy-pasting from a route list, check it. Same base URL, same X-API-Key header, same JSON body.

What you can rely on

Per the docs, every image result is guaranteed to have rank and link — "Primary link returned by the parser" — plus image_url, the full image URL. Everything else is explicitly optional:

Field	Status	What the docs say
`rank`	required	"1-based ranking position in this response."
`link`	required	"Primary link returned by the parser."
`image_url`	required	"Full image URL."
`title`	optional	"When image title or alt text is parsed."
`position`	optional	"Alias for rank when available."
`url`	optional	"Present for normalized result links."
`source_url`	optional	"Only when Google redirect/source URL is observed."
`display_url`	optional	"When Google display text or a domain can be derived."
`thumbnail_url` / `thumbnail`	optional	"When Google exposes a thumbnail."
`source` / `domain`	optional	"When source text is parsed."

The docs also warn that "some Google Images payload shapes expose rank/link but not every normalized alias" — so write your parser defensively: read the three required fields, and treat everything else as a bonus. Field definitions live in SerpBase's endpoint documentation.

The script

import requests

API = "https://api.serpbase.dev/google/image/search"
KEY = "your_api_key"   # your key here

def image_search(q: str, page: int = 1) -> list:
    resp = requests.post(API,
        headers={"X-API-Key": KEY, "Content-Type": "application/json"},
        json={"q": q, "hl": "zh-CN", "gl": "cn", "page": page},
        timeout=30)
    data = resp.json()
    if data.get("status") != 0:
        raise RuntimeError(f"{data.get('status')}: {data.get('error')}")
    return data.get("images", [])

for item in image_search("机械键盘 产品图"):
    # only trust the required fields; optional ones get .get() fallbacks
    print(item["rank"], item["image_url"][:60])
    print("  title:", item.get("title", "(none)"))
    print("  source:", item.get("source", "(none)"), item.get("domain", ""))

Three habits that keep the cost down:

Check status before touching images. The array is optional — "When image results are parsed" — so a zero-result query can come back with status: 0 and no images key at all. data.get("images", []) handles it; data["images"] will KeyError.
Budget before you loop. Each successful request is 2 credits. A 20-keyword monitoring pass is 40 credits; the 100-search free trial covers about two and a half of those. Failed requests come back with credits_charged: 0, so retries are free — but a loop that retries 20 times is still 20 round trips.
Don't re-fetch the same query. Unlike the search endpoint, image results for the same query are stable enough that caching by (q, hl, gl, page) pays off quickly at 2 credits a hit.

FAQ

Why does my title come back empty? It's optional — "When image title or alt text is parsed." Alt text is missing on plenty of real pages, so the parser can't always fill it. Design your display logic to work without it.

Can I use link or image_url in an <img> tag? image_url is the full image URL — that's the one for <img src>. link is the parser's primary result link, which may be a source page rather than the image file itself. And hotlinking third-party images has its own etiquette and legal considerations — check the source site's terms before embedding.

Is there a device parameter? No. device is documented only for the search endpoint. Images takes q, hl, gl, page.

Does the free trial cover images? The 100 free searches are trial credits; images requests cost 2 each, so plan roughly 50 image requests from the trial budget.

Point the script at a product keyword, print the required fields only, and see how much of the optional data your niche actually returns.

Build a Local Docs Search Bot: SQLite FTS5 over Your Documentation

dodou — Thu, 24 Sep 2026 01:09:43 +0000

You don't need a vector database to search your own docs. SQLite ships with FTS5, a full-text search engine that runs inside the same file as your data — no server, no embeddings, no API bill. For a personal knowledge base or a small project's documentation, it beats most "semantic search" setups I've seen on latency and cost, and the whole thing fits in one Python file.

I built this for my notes after getting tired of grep. It ranks by relevance instead of matching substrings, and it handles typos the way people actually search — by words, not exact strings.

What FTS5 gives you

BM25 ranking out of the box — results come back ordered by relevance, not by file path.
Word-level matching: docker networking matches documents containing both words, in any order.
No external service: it's a table in the same SQLite file as everything else, so it backups with your data.

What it doesn't give you is semantic understanding — searching "container port mapping" won't find a doc that only says "expose ports with -p". If you need that, you need embeddings. For most personal docs, you don't.

The script

Two parts: an indexer that loads your markdown files, and a query function.

import sqlite3, pathlib, re

DB = "docs.db"

def index_docs(root: str) -> int:
    conn = sqlite3.connect(DB)
    conn.execute("CREATE VIRTUAL TABLE IF NOT EXISTS docs USING fts5(path, title, body)")
    conn.execute("DELETE FROM docs")

    n = 0
    for p in pathlib.Path(root).rglob("*.md"):
        text = p.read_text(encoding="utf-8", errors="ignore")
        title = text.splitlines()[0].lstrip("# ").strip() if text else p.stem
        body = re.sub(r"[#*`>\[\]()]", " ", text)      # strip markdown syntax
        conn.execute("INSERT INTO docs VALUES (?, ?, ?)", (str(p), title, body))
        n += 1
    conn.commit()
    conn.close()
    return n

def search(query: str, limit: int = 5) -> list:
    conn = sqlite3.connect(DB)
    rows = conn.execute(
        "SELECT path, title, snippet(docs, 2, '<b>', '</b>', '…', 12) "
        "FROM docs WHERE docs MATCH ? ORDER BY rank LIMIT ?",
        (query, limit),
    ).fetchall()
    conn.close()
    return rows

if __name__ == "__main__":
    n = index_docs("./notes")
    print(f"indexed {n} files")
    for path, title, snip in search("docker networking"):
        print(f"\n{title}  ({path})\n  {snip}")

Run it once to build the index, then import search() from your bot, a REPL, or a small UI. ORDER BY rank is the BM25 score — lower is better.

When local search isn't enough

FTS5 covers the case where the answer already exists in your own files. The gap is everything outside them: release notes, changelogs, competitor pages, anything on the web. Two options there, and they're not exclusive:

Keep a web fallback. When local search returns nothing, hit a SERP API — search requests on SerpBase cost 1 credit each and standard credits don't expire, so a fallback path costs almost nothing on a low-traffic bot. Field definitions live in SerpBase's search endpoint documentation.
Hybrid ranking. Try local first; if the top BM25 score is weak or empty, merge in web results. Most "which library does X" questions resolve locally; "what happened with X last week" needs the network.

The MCP route works too if your client supports it — the same key can drive a server that exposes search to Claude or Cursor, which is convenient for interactive use. For a scheduled bot, direct HTTP is simpler.

Cost and limits

SQLite FTS5 has no per-query cost and no network hop — indexing a thousand markdown files takes under a second on a laptop. The practical limit is your disk, not the engine. If your docs are behind auth or scattered across services, local indexing gets awkward fast, and that's when a hosted search API earns its keep.

FAQ

Do I need to re-index every run? Only when files change. For a bot, watch the directory's mtime and re-index on change, or index on startup — with a thousand files it's fast enough not to matter.

How is this different from grep? grep returns lines; FTS5 returns documents ranked by relevance, with highlighted snippets. It also handles multi-word queries as "contains all these words" rather than exact phrase.

Can I search PDFs or HTML too? Yes, after extraction — convert to text, store the text. FTS5 only sees the body column you insert.

What about embeddings for semantic search? Worth it when users phrase queries differently from the docs' wording. Start with FTS5; if logs show people searching "how do I restart the thing" when the doc says "service restart procedure", add embeddings then.

Index your notes tonight and ask the bot a question you'd normally open three browser tabs for. If it answers, you've saved yourself the tabs.

Give Claude Live Google Search with an MCP Server

dodou — Tue, 22 Sep 2026 03:20:31 +0000

An agent's answers are only as fresh as the data it can reach. Wire a SERP API into Claude — or Cursor, Cline, Continue, Codex — through MCP and it can pull live Google results mid-conversation, with sources it can cite. The config is about ten lines of JSON; no glue code, no custom tool schemas to write.

I set this up last week for release-note checking and competitor-watching questions. It's the fastest "my agent knows what happened on the web today" upgrade I've done.

What you need

An MCP-capable client: Claude Desktop, Claude Code, Cursor, Cline/Roo, Continue, or Codex.
Python on your PATH (the server runs via python -m serpbase_mcp).
An API key from a SERP API provider. I use SerpBase — new accounts get 100 free searches without a card, search requests cost 1 credit, and standard credits don't expire. The server, its tool list, and the endpoint parameters are documented in SerpBase's MCP integration docs; the code itself lives in the serpbase-dev/serpbase-mcp repo on GitHub.

The config

Install the server per its repo README, then add it to your client's MCP config (for Claude Desktop that's claude_desktop_config.json):

{
  "mcpServers": {
    "serpbase": {
      "command": "python",
      "args": ["-m", "serpbase_mcp"],
      "env": {
        "SERPBASE_API_KEY": "your_api_key"
      }
    }
  }
}

Restart the client and you get six tools exposed to the agent: Google Search, Images, News, Videos, Maps Search, and Maps Detail — all returning structured results the model can read and cite directly. The key rides in the process environment, not in the chat, and not in your repo.

On Windows, if python isn't found, try "command": "python3" or the full path to your interpreter — that was the only hiccup in my setup.

Prompts that work well

The value shows up with questions that have a time dimension:

"Search Google for the latest release notes of and summarize what changed, with sources."
"Check Google News for from this week and tell me if anything looks important."
"Search for ' pricing' and compare what's on their page today with what I pasted below."

One habit worth keeping: ask the agent to do one search per question instead of letting it chain ten. Every successful tool call is a billed API request (search/news/videos are 1 credit, images/maps are 2), and failed requests come back with credits_charged: 0 — so retries don't cost you, but enthusiasm does.

MCP server or agent skill?

The same org also publishes a portable agent skill (serpbase-dev/serpbase-skill) — a skill folder plus a shell fallback for agents that prefer scripts over an MCP server (opencode-style agents, Claude Code memory, shell workflows). If your tool reads local instructions instead of speaking MCP, use the skill; otherwise the server above is the cleaner path.

FAQ

Does this replace calling the API from my code? No. The MCP server is for interactive agent use — research, monitoring questions, cited answers. Production pipelines should still call the endpoints directly with the same key.

Is my API key exposed to the model? The model sees tool results, not the key. The key lives in the server process's environment via the env block in your config.

What does it cost to try? A hundred free searches cover a real evaluation — wire it up, ask your team's actual questions for a day, and check the credit log afterward.

Add the config, restart your client, then ask it something you'd normally google. When the answer comes back with today's sources attached, you'll know it's working.

Track Keyword Ranks in Google Sheets with Apps Script

dodou — Mon, 21 Sep 2026 01:40:31 +0000

You can track Google rankings for a keyword list inside Google Sheets — no server, no database, no monthly tracker subscription — with about 50 lines of Apps Script that calls a SERP API once per run and writes each keyword's best rank into a dated column. Set a time-driven trigger and the sheet builds your rank history by itself.

I keep one of these for a 50-word client list. The client gets a live doc they already know how to read, and I get a new column every morning without touching anything.

What you need

A Google Sheet with one tab named Ranks. Row 1 can hold dates (the script fills it); column A holds your keywords, one per row.
An API key from a SERP API provider. I use SerpBase — search requests cost 1 credit each, new accounts get 100 free searches, and the entry paid pack is $10 for 20,000 credits with no expiry. Parameter and field definitions live in SerpBase's search endpoint documentation.

The script

Extensions → Apps Script, paste this:

const API_URL = 'https://api.serpbase.dev/google/search';
const DOMAIN  = 'example.com';   // the site you're tracking

function onOpen() {
  SpreadsheetApp.getUi()
    .createMenu('Rank check')
    .addItem('Check now', 'checkRanks')
    .addToUi();
}

function checkRanks() {
  const key = PropertiesService.getScriptProperties().getProperty('SERP_KEY');
  if (!key) throw new Error('Save your key first: Project Settings → Script Properties → SERP_KEY');

  const sheet  = SpreadsheetApp.getActive().getSheetByName('Ranks');
  const values = sheet.getDataRange().getValues();
  const outCol = values[0].length + 1;          // one new column per run

  sheet.getRange(1, outCol).setValue(new Date());   // header = run timestamp

  for (let r = 1; r < values.length; r++) {
    const q = values[r][0];
    if (!q) continue;
    sheet.getRange(r + 1, outCol).setValue(bestRank(q, key));
    Utilities.sleep(500);                        // be gentle on long lists
  }
}

function bestRank(q, key) {
  const resp = UrlFetchApp.fetch(API_URL, {
    method: 'post',
    contentType: 'application/json',
    headers: { 'X-API-Key': key },
    payload: JSON.stringify({ q: q, hl: 'en', gl: 'us' }),
    muteHttpExceptions: true,
  });
  const data = JSON.parse(resp.getContentText());
  if (data.status !== 0) {
    if (data.status === 1020) throw new Error('Out of credits');
    if (data.status === 1029) return 'rate-limited';  // rerun in a minute
    return 'err ' + data.status;
  }
  const hits = (data.organic || [])
    .filter(i => (i.link || '').includes(DOMAIN))
    .map(i => i.rank);                           // 1-based position within this page
  return hits.length ? Math.min(...hits) : '';   // blank = not on page 1
}

Setup, in order:

Put your keywords in column A of the Ranks tab (row 1 stays empty for dates).
In Apps Script, open Project Settings → Script Properties, add SERP_KEY with your key. The key never lives in the sheet itself, so sharing the doc with a client is safe.
Reload the spreadsheet, run Rank check → Check now, and authorize once.
Optional: in Apps Script → Triggers, add a time-driven trigger (e.g. every day at 6am) on checkRanks. Each run appends a new dated column — scroll right to see your history.

A blank cell means the domain didn't appear on page 1 for that keyword. If a run hits the rate limit, the cell says rate-limited — rerun and that column gets refreshed.

What this costs

One request per keyword per run, 1 credit each; failed requests come back with credits_charged: 0, so retries and rate-limit hiccups don't burn balance. Fifty words checked daily is about 1,500 credits a month — the $10 / 20k pack covers that pace for over a year. If daily is overkill for your list, trigger weekly instead; weekly is usually enough to spot real movement anyway.

FAQ

Why not just use Search Console? Different question. Search Console shows impressions and clicks for pages it already knows about; this sheet answers "where do I rank today for this specific keyword list" — including keywords you haven't built pages for yet, which is the useful half when you're planning content.

Can I track page 2 and beyond? The script only requests page 1, so it tracks the top 10. The endpoint takes a page parameter if you want to go deeper, at one credit per page — I've never needed it for a health-check sheet.

Can I watch a competitor at the same time? Duplicate the bestRank filter with a second domain and write it to the adjacent column set. Two domains, one request per keyword — the response already contains everyone's positions.

Paste the script, set the trigger, and tomorrow morning your sheet has its first real column.

Mine people_also_ask and related_searches for Free Content Ideas

dodou — Sun, 20 Sep 2026 01:32:46 +0000

Every SERP you fetch for a client or a report comes with free homework attached: the People Also Ask box and the Related searches strip at the bottom. Most pipelines throw both away. They shouldn't — one is Google telling you exactly which questions its users ask around your topic, the other is a list of adjacent queries worth ranking for. This post adds a small mining layer to an existing SERP pipeline.

What the API gives you

Alongside organic, a search response may carry two optional modules: people_also_ask — an array of question/answer/source entries — and related_searches — a plain array of query strings. Their presence varies per query, so treat both as "maybe there": check before iterating, and when in doubt, store the raw JSON first and model later. Shapes and conditions are defined in the SerpBase search endpoint docs.

The miner

import json, pathlib, requests

API = "https://api.serpbase.dev/google/search"
KEY = "你的 API Key"
SEEDS = ["mechanical keyboard", "mechanical keyboard vs membrane",
         "quiet keyboard for office"]

def fetch(q: str) -> dict:
    resp = requests.post(API,
        headers={"X-API-Key": KEY, "Content-Type": "application/json"},
        json={"q": q, "hl": "en", "gl": "us"}, timeout=30)
    data = resp.json()
    if data.get("status") != 0:
        raise RuntimeError(f"{q}: {data.get('error')}")
    return data

paa, related = {}, set()
for q in SEEDS:
    data = fetch(q)
    # PAA 条目字段以实际返回为准:先原样存,再按需读取 question/title
    paa[q] = data.get("people_also_ask", [])
    for s in data.get("related_searches", []):
        related.add(s.strip().lower())
    print(f"{q}: paa={len(paa[q])} related={len(data.get('related_searches', []))}")

pathlib.Path("paa_raw.json").write_text(
    json.dumps(paa, ensure_ascii=False, indent=1), encoding="utf-8")
pathlib.Path("related.txt").write_text("\n".join(sorted(related)), encoding="utf-8")
print(f"去重后相关搜索 {len(related)} 条")

The raw PAA dump is deliberate: item fields aren't frozen in the docs, so saving the JSON as-is means schema drift costs you a re-read, not a re-crawl. Second-round seeds come straight from related.txt — each fetched query hands you the next three.

Two outputs, two uses

Output	What it is	Feed it into
`paa_raw.json`	Real questions users ask, with context	FAQ sections, headings, "answering the next question" internal links
`related.txt`	Adjacent queries Google associates with your seeds	Keyword clustering, gap analysis seeds, topic clusters

The two compose: cluster the related queries first (they define the topic space), then place PAA questions as sub-headings inside each cluster's page. A page that answers the cluster's main query and the three questions under it earns sitelinks-sized coverage from one URL.

Cost discipline

One fetch per seed = 1 credit. Ten seeds plus a second round of ten = 20 credits — a rounding error at pack prices ($0.50/1k entry, $0.30/1k at the top tier; the $3/month Starter Boost's 10,000 searches would fund 500 rounds). The 100-search free trial covers this entire workflow several times over.

Two mistakes to avoid

PAA from page 1 only. Results shift between pages and hl/gl settings; if your audience is multilingual, mine each market separately — the questions genuinely differ.
Treating related searches as long-tail gold without filtering. They include navigational junk (brand names, "login", "download") and near-duplicates. Dedupe, drop anything containing your competitors' brands, and cluster what survives.

FAQ

Do PAA answers come from my competitors? Often, yes — the source links point at pages Google trusts for that question. That's competitive intel: note who answers what, and where their answer is thin.

How often should I re-mine? PAA rotates faster than organic rankings. Monthly for stable topics; before starting any new content cluster, always.

Can I fetch PAA for a page instead of a query? No — modules attach to queries. Pick the query your page targets, not the page itself.

Run ten seeds tonight and you'll have a month of FAQ headings queued — from Google, for the price of a coffee refill.

Ingest SERP Data on a Schedule with Cloudflare Workers (Cron + KV)

dodou — Sat, 19 Sep 2026 01:57:08 +0000

You don't need a VPS to track rankings overnight. A Cloudflare Worker with a cron trigger can call a SERP API once a day, store each snapshot in KV, and fire a webhook when something moves — for roughly zero dollars at a dozen-keywords scale. This post ships the whole worker in one file.

The design in three lines

A cron trigger fires scheduled() once per day.
Each run fetches results for every keyword in config, writing one KV key per keyword per day.
A rank beyond your threshold (or an API error) posts a compact alert to a webhook.

The worker

// src/index.js — wrangler deploy and forget
const API_URL = "https://api.serpbase.dev/google/search";

const KEYWORDS = [
  { q: "best mechanical keyboard", threshold: 10 },
  { q: "mechanical keyboard vs membrane", threshold: 15 },
  { q: "quiet office keyboard", threshold: 10 },
];
const OWN_DOMAIN = "example.com";
const ALERT_WEBHOOK = "https://hooks.example.com/abc"; // Slack/Discord style

export default {
  async scheduled(controller, env, ctx) {
    const today = new Date().toISOString().slice(0, 10);
    const alerts = [];

    for (const { q, threshold } of KEYWORDS) {
      const resp = await fetch(API_URL, {
        method: "POST",
        headers: {
          "X-API-Key": env.SERPBASE_API_KEY,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({ q, hl: "en", gl: "us" }),
      });
      const data = await resp.json();

      // Errors carry credits_charged: 0 — log and move on, don't spam the webhook
      if (data.status !== 0) {
        console.error(`${q}: ${data.status} ${data.error}`);
        continue;
      }

      const hit = (data.organic || []).find((r) => (r.link || "").includes(OWN_DOMAIN));
      const rank = hit ? hit.rank : null;

      await env.SNAPS.put(`${q}:${today}`, JSON.stringify({
        rank,
        top5: (data.organic || []).slice(0, 5).map((r) => r.link),
      }));

      if (rank === null || rank > threshold) {
        alerts.push(`⚠️ ${q}: ${rank === null ? "not in top 10" : `#${rank}`} (limit #${threshold})`);
      }
    }

    if (alerts.length) {
      await fetch(ALERT_WEBHOOK, {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ text: `SERP alerts ${today}\n` + alerts.join("\n") }),
      });
    }
  },
};

Request params, the status/error envelope, and the organic field table are documented in the SerpBase search endpoint docs — the worker adds scheduling and storage, nothing else. The wrangler.toml you need is six lines: name, main, compatibility_date, and a [triggers] crons = ["0 6 * * *"] plus the KV namespace binding SNAPS.

The KV layout (and why one key per day)

Key	Value	Used by
`{q}:{date}`	`{rank, top5}`	The daily write; the alert check
`{q}:history` (optional)	appended array	Trend charts, week-over-week diffs

One key per keyword per day keeps writes atomic and reads trivial: list({ prefix: q + ":" }) walks the whole history. Keep top5 alongside your own rank — when your position drops, the question "who took my slot?" is answerable from the same snapshot without a second API call.

What this costs

Workers' free tier covers this workload comfortably (a dozen fetches a day is nothing). The SERP side is 1 credit per successful request; the prepaid packs run $0.50/1k at entry down to $0.30/1k at the top tier, the $3/month Starter Boost covers 10,000 searches, and new accounts start with 100 free searches — at one keyword per day, that trial alone runs you three months.

Two failure modes to plan for

The webhook becomes the SPOF. If the alert POST fails, the data is still in KV — log the failure and let the next run re-alert only if the condition still holds. Don't build retry state in the worker; KV already remembers yesterday.
Cron runs in UTC. 0 6 * * * is 6:00 UTC. If your users search at 9am in Berlin, your "daily snapshot" should probably land before that, not after.

FAQ

Why KV instead of D1/R2? At one JSON blob per keyword per day, KV is the least machinery that works. Move to D1 when you want SQL across keywords (that's the analytics problem, not the ingestion problem).

Why not check 100 keywords per run? Each is 1 credit; 100/day is ~3,000/month — real money at pack prices, and Worker CPU time starts to matter. Daily snapshots for your top 10–20 head terms, and let the long tail live in your batch pipeline.

Can I add more endpoints (news, maps)? Yes — same envelope, different payload key, and note images/maps cost 2 credits. Factor that into the cron budget.

Deploy it, wire the webhook to your team chat, and the next time a ranking moves you'll hear about it from your own bot — not from a client.

serp: a 60-Line Python CLI for Google Search Results

dodou — Fri, 18 Sep 2026 06:36:04 +0000

Between "I wonder who ranks for X" and opening a notebook to answer it sits about twenty minutes of boilerplate. Close that gap with a small CLI: type serp "best mechanical keyboard" --gl us, get the top results printed as a table, and pipe the same output to grep, jq-adjacent tooling, or a cron job. No framework, no config file — one file, standard library plus requests.

The whole thing

#!/usr/bin/env python3
"""serp.py — query a SERP API from the command line."""
import argparse
import json
import os
import sys

import requests

API_URL = "https://api.serpbase.dev/google/search"


def search(q: str, hl: str, gl: str, page: int, device: str | None) -> dict:
    body = {"q": q, "hl": hl, "gl": gl, "page": page}
    if device:
        body["device"] = device  # search endpoint only: default/pc/mobile
    resp = requests.post(
        API_URL,
        headers={"X-API-Key": os.environ["SERPBASE_API_KEY"],
                 "Content-Type": "application/json"},
        json=body,
        timeout=30,
    )
    resp.raise_for_status()
    data = resp.json()
    if data.get("status") != 0:
        sys.exit(f"API error {data.get('status')}: {data.get('error')}")
    return data


def main() -> None:
    p = argparse.ArgumentParser(description="Query Google results via a SERP API")
    p.add_argument("query")
    p.add_argument("--hl", default="en")
    p.add_argument("--gl", default="us")
    p.add_argument("--page", type=int, default=1)
    p.add_argument("--device", choices=["default", "pc", "mobile"])
    p.add_argument("--json", action="store_true", help="dump full response")
    args = p.parse_args()

    data = search(args.query, args.hl, args.gl, args.page, args.device)
    if args.json:
        print(json.dumps(data, ensure_ascii=False, indent=2))
        return

    for r in data.get("organic", []):
        print(f"{r.get('rank', ''):>3}  {r.get('title', '')}")
        print(f"     {r.get('link', '')}")


if __name__ == "__main__":
    main()

Save it, chmod +x serp.py, export SERPBASE_API_KEY, and you have a working search tool. Params, the status/error envelope, and the organic field table are specified in the SerpBase search endpoint docs — the CLI adds a flag parser on top and nothing else.

The flags you'll actually use

Flag	What for
`--json`	Full response for piping into other tools (`serp "..." --json \
{% raw %}`--gl us` / `--hl en`	Market and language; defaults match the API
`--page 2`	Second page of results
`--device mobile`	See the mobile SERP when debugging layout-sensitive topics

Why this beats a notebook for small questions

It composes. serp "site:news.ycombinator.com serp api" --json | jq '.organic | length' answers "how many results" without editing any code.
It's cron-able. The same command in a scheduled task becomes a poor-man's monitor; add a timestamp and redirect to a file and you have snapshots.
It fails loudly. A missing key raises immediately (env var lookup), and API errors exit non-zero with the code — so cron logs tell you what happened instead of silently writing empty output.

One habit worth keeping: the CLI never retries. For interactive use you'll see the error and rerun yourself; for automation, wrap the call in your own backoff so the retry policy lives in one place, not inside a "quick tool".

Cost discipline

Every invocation is one search request = 1 credit (errors don't charge). At $0.50/1k starting tier — with the $3/month Starter Boost covering 10,000 searches and 100 free searches for new accounts — casual CLI use is effectively free; volume belongs in your batch scripts, where you already control pacing.

FAQ

Why not argparse + urllib to drop requests entirely? You can; requests just makes the headers/timeout story shorter. The rest of the file has no other dependency.

Where do I put long-running features (caching, exports)? Not here. This file's whole value is that it does one request and prints one answer. Features belong in the module you import from scripts — the CLI stays a thin front door.

Windows? Works under python serp.py "query" in PowerShell; the | piping examples assume a POSIX shell, so on Windows use --json plus a redirect if you need to chain.

Steal the file, rename it, delete the flags you don't use — a tool you can hold in your head is the one you'll reach for.

Build a Type-Safe SERP API Client in TypeScript (fetch + Zero Dependencies)

dodou — Thu, 17 Sep 2026 06:07:56 +0000

By the time you wire a SERP API into a real product, you've usually made the same deal three times in three files: a fetch call, a type assertion, and a prayer that the response shape matches. This post collapses all three into one small, dependency-free client. ~80 lines of TypeScript gives you typed organic results, discriminated errors, and a call site that reads like const results = await serp.search({ q }).

Types first, code second

The response has two layers worth modeling: the envelope (status, timings, credits) and the payload (the organic array). Model both:

// serp.ts
export interface SearchParams {
  q: string;
  hl?: string;      // defaults to "en" server-side
  gl?: string;      // defaults to "us" server-side
  page?: number;    // 1-based
  device?: "default" | "pc" | "mobile"; // search endpoint only
}

export interface OrganicResult {
  rank: number;        // 1-based position within this response
  title: string;
  link: string;        // always present
  display_url?: string;
  snippet?: string;
  date?: string;       // raw date text as extracted from the snippet
}

export interface SearchOk {
  status: 0;
  request_id: string;
  elapsed_ms: number;
  credits_charged: number;
  organic: OrganicResult[];
}

export interface SerpError {
  status: number;      // non-zero
  error: string;
  request_id?: string;
  credits_charged?: 0;
}

export type SearchResponse = SearchOk | SerpError;

Two deliberate choices: status: 0 on the success branch turns the envelope into a discriminated union, so every consumer must narrow before touching organic. And link is required while snippet is optional — that matches the documented field table for the SerpBase search endpoint docs, where link is the only guaranteed field per result.

The client

const API_URL = "https://api.serpbase.dev/google/search";

export class SerpApiError extends Error {
  constructor(
    public code: number,
    public requestId: string | undefined,
    message: string,
  ) {
    super(`SERP API error ${code}: ${message}`);
    this.name = "SerpApiError";
  }
}

export function createClient(apiKey = process.env.SERPBASE_API_KEY!) {
  async function search(params: SearchParams): Promise<SearchOk> {
    if (!params.q?.trim()) throw new RangeError("q is required");

    const resp = await fetch(API_URL, {
      method: "POST",
      headers: {
        "X-API-Key": apiKey,
        "Content-Type": "application/json",
      },
      body: JSON.stringify(params),
    });

    const data = (await resp.json()) as SearchResponse;

    if (data.status !== 0) {
      throw new SerpApiError(data.status, data.request_id, data.error);
    }
    return data;
  }

  /** First organic hit for a domain, or undefined. */
  async function positionOf(q: string, domain: string) {
    const { organic } = await search({ q });
    return organic.find((r) => r.link.includes(domain));
  }

  return { search, positionOf };
}

The call site — the part your teammates will read a hundred times:

const serp = createClient();

const best = await serp.positionOf("best mechanical keyboard", "example.com");
if (best) console.log(`#${best.rank}: ${best.title}`);

Reading the error surface

The error codes worth branching on in application code are few: 1001 means your key is wrong, 1020 means the account is out of credits, and 1029 means rate limiting — the only one that merits automatic retry, with backoff. Everything else (upstream timeouts, malformed requests) deserves a logged request_id and a failed job, not a silent retry.

Note what the client does not do: it doesn't retry on status !== 0 responses that consumed no credits (credits_charged is 0 on errors), and it doesn't hide failures behind undefined. Errors here are loud so that dashboards can't quietly show stale data.

Test it without spending credits

Inject the fetch or run against a recorded fixture — one JSON file per query is enough:

import { search } from "./fixture"; // recorded response

const client = createClient("test-key");
jest.spyOn(global, "fetch").mockResolvedValue(new Response(JSON.stringify(search)));

Because the parser boundary is a single function from Response to SearchOk, one fixture per status covers the whole unit surface. Keep a single live test (mirroring the fixture approach) to catch schema drift.

FAQ

Why not generate types from the docs with an OpenAPI spec? If the API published one, sure. Hand-rolled interfaces for a single endpoint are cheaper than a codegen pipeline — and the field table is small enough to review by eye.

Why throw instead of returning a Result type? For an internal client, exceptions at the boundary are fine; the discriminated union already forces narrowing on the shape check. If you're building a library, swap the throw for Result<SearchOk, SerpApiError> — the union type stays the same.

What about the other endpoints (news, images, maps)? Same envelope, different payload key and params. Copy the SearchOk interface, rename organic to news/images/places, and tighten the param type — the structure repeats.

Start with the 80 lines above, keep the fixtures next to the client, and resist the urge to wrap it in a class hierarchy — a function and a union type are the whole design.

Cache Your SERP API Responses: One Tiny Node.js Module, Fewer Credits

dodou — Thu, 17 Sep 2026 05:34:27 +0000

Every repeat query you send to a SERP API costs a credit and returns roughly the same organic results as it did an hour ago. Most of that traffic is your own doing: a dashboard refresh, a re-run of the same report, a test that reads the same keyword. Cache the response and the second call is free — same JSON, no request, no credit.

You don't need Redis for this. A folder of JSON files and a 30-line module will carry a small pipeline for months.

The cache key is the whole request

Two calls that differ only in gl are different answers. Build the key from every parameter you send, in a fixed order:

// serp-cache.mjs — disk cache in front of a SERP API
import { createHash } from "node:crypto";
import { readFile, writeFile, mkdir } from "node:fs/promises";
import path from "node:path";

const API_URL = "https://api.serpbase.dev/google/search";
const CACHE_DIR = ".serp-cache";
const TTL_MS = 6 * 60 * 60 * 1000; // 6 hours; set to Infinity for stable snapshots

function cacheKey(params) {
  const canon = Object.keys(params)      // only the params that shape results
    .filter((k) => params[k] !== undefined)
    .sort()
    .map((k) => `${k}=${params[k]}`)
    .join("&");
  return createHash("sha1").update(canon).digest("hex").slice(0, 20);
}

export async function search(params, apiKey = process.env.SERPBASE_API_KEY) {
  const file = path.join(CACHE_DIR, `${cacheKey(params)}.json`);
  try {
    const hit = JSON.parse(await readFile(file, "utf8"));
    if (Date.now() - hit.at < TTL_MS) return hit.data;
  } catch {
    /* miss: fall through to the network */
  }

  const resp = await fetch(API_URL, {
    method: "POST",
    headers: { "X-API-Key": apiKey, "Content-Type": "application/json" },
    body: JSON.stringify(params),
  });
  if (!resp.ok) throw new Error(`HTTP ${resp.status}`);
  const data = await resp.json();
  if (data.status !== 0) throw new Error(`API error: ${data.error}`);

  await mkdir(CACHE_DIR, { recursive: true });
  await writeFile(file, JSON.stringify({ at: Date.now(), params, data }));
  return data;
}

Params, billing, and the response envelope are defined in the SerpBase API docs — the wrapper above adds nothing to the request, it just replays the last answer while it's still fresh.

What one afternoon of caching saves

Say you track 20 keywords and reload a dashboard 5 times a day:

Usage pattern	Requests/day	With 6h TTL	Credits saved
Dashboard, reloaded often	100	~20	80%
Weekly report re-runs	20	20 (first run) then 0	~100% after Monday
Rank tracking, once daily	20	20	0 — don't cache this

The last row matters: if the number you care about is the position right now, a cache is lying to you. Rank tracking wants fresh rank fields every run. Caching is for everything that reads results more often than results change.

Delete like you mean it

A cache is stale data wearing a friendly name. Two habits keep it honest:

Invalidate on purpose. Re-run with TTL_MS = 0 when you upgrade parsing code or want a clean snapshot — the module re-records every key once.
Ship the cache folder in CI artifacts, not in git. Fixtures of real responses are gold for tests; a 40 MB .serp-cache in your repo is not.

What it costs anyway

Standard searches are 1 credit per successful request; packs start around $0.50/1k with volume tiers down to $0.30/1k, the $3/month Starter Boost includes 10,000 searches, and new accounts get 100 free searches — enough to fill this cache several times over and verify the wrapper end to end. Exact numbers live on the pricing page; if they've moved, the math above still holds: a hit rate of X% cuts your bill by X%.

FAQ

Why disk files instead of SQLite or Redis? One JSON per query greps well, diffs well, and survives process restarts. Move to SQLite when you want queries across cached responses (that's a data problem, not a caching problem); move to Redis when latency matters more than simplicity.

Should the TTL differ per endpoint? Yes. News and video results go stale in hours; a gl-pinned organic snapshot can sit for days if you only study structure. Cache people_also_ask and related_searches aggressively — those modules move slowly.

Does the cache hide API errors? It shouldn't — the code above only writes on status === 0. If a request fails, nothing is cached and the error propagates.

Start with the 20-line version, watch the hit rate for a week, and only then decide whether you need something fancier.

Mocking a SERP API in Tests: Fixtures Over Live Calls

dodou — Tue, 15 Sep 2026 01:07:03 +0000

Tests that call a live SERP API are slow, cost credits, and fail whenever rankings move. The fix is the usual one: record a real response to a JSON fixture once, replay it through a fake transport in your tests, and keep a single opt-in live test that watches for schema drift. No vendor SDK required — the boundary is just one POST.

The boundary you're faking

POST https://api.serpbase.dev/google/search with an X-API-Key header and a JSON body (q required; hl, gl, page, device optional). A successful request costs 1 credit and returns an envelope with status (0 means success), error, request_id, elapsed_ms, credits_charged, plus an organic array whose items carry rank (1-based position in the response), title, link and an optional snippet.

Step 1: record a fixture

import json
import os
import pathlib

import requests

API_URL = "https://api.serpbase.dev/google/search"


def record(query: str, out: pathlib.Path) -> None:
    resp = requests.post(
        API_URL,
        headers={"X-API-Key": os.environ["SERPBASE_API_KEY"],
                 "Content-Type": "application/json"},
        json={"q": query, "hl": "en", "gl": "us", "page": 1},
        timeout=30,
    )
    resp.raise_for_status()
    out.parent.mkdir(parents=True, exist_ok=True)
    out.write_text(json.dumps(resp.json(), ensure_ascii=False, indent=2),
                   encoding="utf-8")


if __name__ == "__main__":
    record("python asyncio tutorial", pathlib.Path("tests/fixtures/search_python.json"))

Every fixture costs 1 credit, so recording five representative queries costs five credits — then your test suite runs forever for free. Field names and the status convention above come from the SerpBase API docs.

Step 2: a client with an injectable session

import requests


class SerpClient:
    def __init__(self, api_key: str, session: requests.Session | None = None,
                 base_url: str = "https://api.serpbase.dev"):
        self.api_key = api_key
        self.session = session or requests.Session()
        self.base_url = base_url

    def search(self, query: str, page: int = 1) -> list[dict]:
        resp = self.session.post(
            f"{self.base_url}/google/search",
            headers={"X-API-Key": self.api_key},
            json={"q": query, "page": page},
            timeout=30,
        )
        resp.raise_for_status()
        data = resp.json()
        if data.get("status") != 0:
            raise RuntimeError(f"status={data.get('status')} error={data.get('error')}")
        return data.get("organic", [])

    @staticmethod
    def position_of(results: list[dict], domain: str) -> int | None:
        for item in results:
            if domain in item.get("link", ""):
                return item.get("rank", item.get("position"))
        return None

Two design choices make this testable: the session is a parameter (so tests pass a fake with the same post signature), and position_of is a pure function — no HTTP, trivially testable.

Step 3: replay the fixture in pytest

import json
import pathlib

import pytest

from client import SerpClient

FIXTURE = json.loads(
    pathlib.Path("tests/fixtures/search_python.json").read_text(encoding="utf-8")
)


class FakeResponse:
    status_code = 200

    def __init__(self, payload):
        self._payload = payload

    def raise_for_status(self):
        pass

    def json(self):
        return self._payload


class FakeSession:
    def __init__(self, payload):
        self.payload = payload
        self.calls = []

    def post(self, url, headers=None, json=None, timeout=None):
        self.calls.append({"url": url, "json": json})
        return FakeResponse(self.payload)


def make_client(payload):
    fake = FakeSession(payload)
    return SerpClient("test-key", session=fake), fake


def test_position_is_found():
    client, _ = make_client(FIXTURE)
    results = client.search("python asyncio tutorial")
    assert SerpClient.position_of(results, "docs.python.org") is not None


def test_missing_domain_returns_none():
    client, _ = make_client(FIXTURE)
    results = client.search("python asyncio tutorial")
    assert SerpClient.position_of(results, "not-in-results.example") is None


def test_api_error_raises():
    client, _ = make_client({"status": 1020, "error": "INSUFFICIENT_CREDITS",
                             "request_id": "req_x"})
    with pytest.raises(RuntimeError, match="1020"):
        client.search("anything")


def test_request_shape():
    client, fake = make_client(FIXTURE)
    client.search("python asyncio tutorial")
    assert fake.calls[0]["json"] == {"q": "python asyncio tutorial", "page": 1}

That last test is the quiet win: the request your code sends is asserted against the documented shape, so a refactor that drops page or renames q fails in CI instead of in production.

Step 4: one live test, opt-in

Fixtures are snapshots; the API can evolve. Keep a single test that hits the real endpoint, marked so it never runs by default:

@pytest.mark.live
def test_live_contract():
    """Run with: pytest -m live   (costs 1 credit)"""
    client = SerpClient(os.environ["SERPBASE_API_KEY"])
    data = client.search("python asyncio tutorial")
    assert data and "rank" in data[0]

Run pytest -m "not live" in CI and pytest -m live weekly, or when you touch parsing code. Register the marker in pytest.ini (markers = live: hits the real API) so the suite stays warning-free.

What it costs

Recording fixtures: 1 credit per query. Everything above then runs at 0 credits — tests, CI, refactors. New accounts start with 100 free searches, which is plenty to record a fixture set and validate the client once against the live API.

FAQ

Why not VCR.py or a record/replay library? For a single endpoint, one JSON file and a twenty-line fake session are easier to read and have no matching rules to debug. Reach for a library when you're recording many hosts.

How often should fixtures be refreshed? Re-record when the API docs change, when you add fields to your parser, or quarterly — a stale fixture still tests your parsing, just not the current payload.

Should I mock the API or run against a sandbox? Mock the HTTP boundary in unit tests; keep one live test as the contract check. That combination catches both your bugs and their schema changes.

Record one fixture for a query you actually use, wire the fake session into your suite, and your SERP tests stop being flaky overnight.

Call a SERP API from Rust: reqwest and serde in Practice

dodou — Mon, 14 Sep 2026 01:05:18 +0000

A SERP API call from Rust is a blocking reqwest POST with an X-API-Key header, serde for the JSON, and a small loop for pagination. Two crates, about seventy lines, no async runtime needed if you're just scripting a handful of queries.

The request you're making

POST https://api.serpbase.dev/google/search with a JSON body: q is required; hl (defaults to en), gl (defaults to us), page (1-based, defaults to 1) and device (default / pc / mobile, search endpoint only) are optional. One successful request costs 1 credit. The response envelope carries status, request_id, credits_charged and search_type, plus the organic array.

Add the two crates:

[dependencies]
reqwest = { version = "0.12", features = ["blocking", "json"] }
serde = { version = "1", features = ["derive"] }

The client

use serde::{Deserialize, Serialize};

#[derive(Serialize)]
struct SearchRequest<'a> {
    q: &'a str,
    hl: &'a str,
    gl: &'a str,
    page: u32,
    device: &'a str,
}

#[derive(Deserialize, Debug)]
struct OrganicItem {
    rank: u32,
    title: String,
    link: String,
    #[serde(default)]
    snippet: String,
    #[serde(default)]
    date: String,
}

#[derive(Deserialize, Debug)]
struct SearchResponse {
    status: i64,
    #[serde(default)]
    error: String,
    #[serde(default)]
    request_id: String,
    #[serde(default)]
    credits_charged: f64,
    #[serde(default)]
    organic: Vec<OrganicItem>,
}

fn search(
    client: &reqwest::blocking::Client,
    key: &str,
    query: &str,
    page: u32,
) -> Result<SearchResponse, Box<dyn std::error::Error>> {
    let body = SearchRequest {
        q: query,
        hl: "en",
        gl: "us",
        page,
        device: "default",
    };
    let data: SearchResponse = client
        .post("https://api.serpbase.dev/google/search")
        .header("X-API-Key", key)
        .json(&body)
        .send()?
        .json()?;
    if data.status != 0 {
        return Err(format!(
            "status={} error={} request_id={}",
            data.status, data.error, data.request_id
        )
        .into());
    }
    Ok(data)
}

The endpoint shape above, including the status convention (0 means success) and the field names, comes from the SerpBase docs for the search endpoint. I only map the fields I need: rank (1-based position in the response), title, link, and optionally snippet and date. Everything else in the payload — url, display_url, sitelinks, and friends — is ignored by serde unless you add it to the struct.

Two details worth copying:

#[serde(default)] on optional fields. The docs mark snippet, date and friends as optional; without the attribute, a single missing key would fail deserialization for the whole response.
Check status, not just HTTP. A transport-level failure gives you a non-2xx status; API-level problems (invalid key, insufficient credits, rate limit) still return JSON with a numeric status and an error string, so branch on the body.

Paginate and print

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let key = std::env::var("SERPBASE_API_KEY")?;
    let client = reqwest::blocking::Client::builder()
        .timeout(std::time::Duration::from_secs(30))
        .build()?;

    for page in 1..=2 {
        let data = search(&client, &key, "rust reqwest example", page)?;
        if data.organic.is_empty() {
            break; // no more results
        }
        for item in &data.organic {
            println!("{}. {} — {}", item.rank, item.title, item.link);
        }
        println!("credits charged: {}", data.credits_charged);
        std::thread::sleep(std::time::Duration::from_millis(1500));
    }
    Ok(())
}

Reuse one Client for every request — building it per call throws away the connection pool, which is the whole reason to keep it around.

What it costs to run

1 credit per successful request, and credits_charged in each response is what actually got billed. New accounts come with 100 free searches, enough to validate the client. If you scale this up with tokio and many concurrent requests, watch for 1029 RATE_LIMITED — the docs map it to QPS or concurrency limits, so cap in-flight requests rather than retrying harder.

FAQ

Blocking or async? Blocking for scripts and cron jobs; reach for tokio + reqwest::Client when you're fetching hundreds of queries and want concurrency with a semaphore.

Why #[serde(default)] instead of Option<String>? Either works. I use defaults with empty strings for display fields and reserve Option<T> for values where "missing" and "empty" mean different things to the caller.

How do I get sitelinks? It's an optional nested array of { title, link, url, ... }; add a Vec<Sitelink> field and serde handles the nesting.

cargo run with SERPBASE_API_KEY set is enough to see real results — the structs are the part you'll reuse. All parameters used here are documented at the link above.