DEV Community

Cover image for Google Forums Search by API: Pull Reddit and Quora Threads as JSON in 2026
Truffle Pig Data
Truffle Pig Data

Posted on

Google Forums Search by API: Pull Reddit and Quora Threads as JSON in 2026

The honest answers about any product live in forum threads, and the fastest index of those threads is the Forums tab on Google Search. It surfaces Reddit, Quora, and niche boards in one ranked list, and it offers no way to export any of it. I'll show the scripted route and its failure points, then the tool I use instead: the Google Forums API on Apify, which turns a query into structured JSON threads.

Disclosure: the Apify links in this post are affiliate links. If you run the Actor, I may earn a referral commission at no extra cost to you.

Is there an API for Google's Forums tab?

No. Google's official search products do not expose the Forums vertical at all, so there is no endpoint that returns "the discussion threads Google ranks for this query." That gap matters because the Forums tab does something hard: it already picked the relevant threads across Reddit, Quora, Stack Exchange, and forums you have never heard of. A scraper-as-API gives you that ranking as data: send q, get forum_results back.

What the Google Forums API returns

The Google Forums API returns each page of forum results as one JSON item with the ranked threads, their sources, snippets, and engagement metadata.

Field Example Notes
forum_results[].title For you what is the best programming language? With position in the ranking
forum_results[].link https://www.reddit.com/r/learnprogramming/... Direct thread URL
forum_results[].source Reddit . r/learnprogramming Site and community
forum_results[].displayed_meta 70+ comments . 2 years ago Engagement and recency
forum_results[].snippet C++ It's the biggest language... Text preview
search_metadata.forums_count 20 With pages_processed and totals

Localization covers 40+ countries through gl and hl, with device emulation and duplicate filtering, and the JSON contract stays the same everywhere.

Who this is for

Brand and community folks hunting mentions across boards they do not read daily. Market researchers who mine Reddit and Quora for unfiltered opinions before a launch. And SEO or AI-corpus builders who want the discussion landscape for a topic as rows rather than tabs.

The manual way, and where it breaks

Scripting a Google results page is a rite of passage, and it rarely survives contact. The Forums tab is a JavaScript-rendered vertical with markup that shifts, localization that depends on cookies and IP, and rate limits that arrive fast once you paginate. You end up managing proxies and parsing regressions for a page you do not control. I keep a graveyard of my own dead SERP parsers, so I say this with some feeling.

The faster way: run the Google Forums scraper

Apify Console

  1. Open the Google Forums API and click Try for free.
  2. Type your query in q, optionally set gl, hl, and max_pages.
  3. Run it and export the threads as JSON, CSV, or Excel.

REST

curl -X POST "https://api.apify.com/v2/acts/johnvc~google-forums-search-api/runs?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "q": "what is the best programming language", "gl": "us", "hl": "en", "max_pages": 2 }'
Enter fullscreen mode Exit fullscreen mode

Reference: the Apify API docs.

Collect forum threads in Python

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("johnvc/google-forums-search-api").call(
    run_input={"q": "what is the best programming language", "max_pages": 2}
)

for page in client.dataset(run["defaultDatasetId"]).iterate_items():
    for thread in page.get("forum_results", []):
        print(thread["position"], thread["source"], thread["title"], thread["displayed_meta"])
Enter fullscreen mode Exit fullscreen mode

Each item is one results page, roughly ten threads, with the ranking preserved in position.

Find Reddit threads mentioning your brand

Search your brand name and the ranked threads arrive with comment counts and recency, ready for triage: Find Reddit threads mentioning your brand.

Run a Reddit competitor analysis

Query a competitor instead of yourself and read what their users complain about. The task Reddit competitor analysis is the preconfigured version.

Mine discussions for market research

Category queries like "best budgeting app" return the threads where buying decisions actually get argued: Reddit discussions for market research.

Scrape the raw Forums results

The general-purpose starting task, query in and every forum thread out, is Scrape Google Forums and discussions results.

Ask Claude to search the boards over MCP

Connected via the Model Context Protocol, Claude, Claude Code, and Cursor can run a live forums search as a tool call, so "what are people saying about X this month" gets grounded in actual threads. Setup lives in Search Reddit and Quora from Claude via MCP, and you can read more about Claude at claude.ai.

FAQ about scraping Google forum results

Does Google offer this data anywhere, or is a scraper required?

No official Google product exposes the Forums vertical, so a scraper is the only programmatic route. This one returns the same ranked list the tab shows, as a stable JSON contract across countries.

How is the Google Forums scraper priced?

Per event: a $0.03 setup fee per run plus one charge per page of results pushed, about ten threads per page, with the current per-page price on the Store card. The Actor estimates the maximum cost up front and aborts cleanly if it would exceed your remaining budget, and failed pages are never charged.

Why did the scraper return an empty forum_results array?

Some queries simply have no Forums-tab results, especially narrow or freshly trending ones. Rephrase toward how people actually ask the question and results usually appear; the metadata still records the attempt honestly.

Can Claude or Cursor drive this scraper through MCP?

Yes. Exposed through the Apify MCP server, the Actor is a callable tool for any MCP client, which makes live community search available inside an agent conversation.

Can I schedule the scraper to monitor mentions weekly?

That is the intended pattern: save your query as a task, attach an Apify schedule, and each run appends fresh threads with timestamps. Start from the Google Forums API.

Is it legal to point a scraper at Google forum results?

The Actor collects public search results, the same list any signed-out visitor sees, and links out to public threads. As always, how you use collected data commercially is on you to clear for your jurisdiction.

More from Truffle Pig Data

Topic monitoring works best across surfaces: the Google News API lines up press coverage against the community chatter, the Google AI Overview API captures what Google's AI summary claims for the same query, and the YouTube Transcripts API adds spoken content to the corpus.

Wrapping up

The Forums tab already found the threads that matter; the missing piece was an export button. The Google Forums API is that button, one query away from JSON.

Top comments (0)