DEV Community

Kevan
Kevan

Posted on

How I find people asking for help on niche forums (free Discourse API, 30 lines)

#javascript #webdev #typescript #api

I'm building a tool that finds people who just asked online for what a founder sells. Web search APIs are good at blogs and big sites. They're weak on niche forums, and that's where a lot of people actually ask for help.
Many of those forums run on Discourse: the Audacity forum, the Steinberg forum (Cubase), the Ardour forum, the n8n community, the Obsidian forum. Discourse has a public search endpoint, no key needed.
The search

type Topic = { id: number; title: string; slug: string; created_at: string };
type Post = { topic_id: number; post_number: number; username: string };

async function recentQuestions(forum: string, keywords: string, days = 30) {
  const since = new Date(Date.now() - days * 86_400_000).toISOString().slice(0, 10);
  // in:first = only opening posts, after: = only topics started since that day
  const q = `${keywords} in:first after:${since} order:latest`;
  const res = await fetch(`https://${forum}/search.json?q=${encodeURIComponent(q)}`, {
    headers: { Accept: "application/json" },
  });
  if (!res.ok) return [];
  const data = (await res.json()) as { topics?: Topic[]; posts?: Post[] };
  return (data.topics ?? []).map((t) => {
    const opener = data.posts?.find((p) => p.topic_id === t.id && p.post_number === 1);
    return {
      url: `https://${forum}/t/${t.slug}/${t.id}`,
      title: t.title,
      author: opener?.username ?? null,
      posted: t.created_at.slice(0, 10),
    };
  });
}
Enter fullscreen mode Exit fullscreen mode

Two filters do most of the work:
in:first only matches opening posts. You get the person who asked, not someone who replied.
after:YYYY-MM-DD only keeps topics started after that day.
The response has topics (title, slug, created_at) and posts (who wrote each match). With in:first, the matching post is post #1, so its username is the person asking.
The date you can trust
The date a search index gives you isn't always the post date. It can be the last crawl or the last reply. A thread from two years ago with one fresh reply can look brand new.
On Discourse, /t/<id>.json has the real created_at, so I check it before showing anyone a post:

async function postedOn(topicUrl: string): Promise<string | null> {
  const m = /^(https:\/\/[^/]+)\/t\/[^/]+\/(\d+)/.exec(topicUrl);
  if (!m) return null;
  const res = await fetch(`${m[1]}/t/${m[2]}.json`, { headers: { Accept: "application/json" } });
  if (!res.ok) return null;
  const topic = (await res.json()) as { created_at?: string };
  return topic.created_at?.slice(0, 10) ?? null;
}
Enter fullscreen mode Exit fullscreen mode

My rule now: if the date can't be read on the forum itself, the post isn't shown.
Try it

const found = await recentQuestions("forum.audacityteam.org", "podcast sound");
console.log(found);
if (found[0]) console.log("checked date:", await postedOn(found[0].url));
Enter fullscreen mode Exit fullscreen mode

On three audio forums (Audacity, Steinberg, Ardour) with three keywords each, the whole search took about 2 seconds. It found a podcaster asking for help with EQ and compression, someone whose stem separation came out with an almost empty drum track, and someone who wants to remaster old tracks with only the CD masters left.
Things to know
Keep it light: a few requests per search, a timeout, and a clear User-Agent.
Not every forum is Discourse, and some sit behind bot protection that blocks scripts. Skip those.
Read each forum's rules before replying to anyone there. Many don't allow vendors, so answer the question first.
This now runs inside FounderPulse (founder-pulse.com): for each search, the AI suggests up to 3 forums that fit the product, and these searches run for free next to the web search. If your users ask for help on a Discourse forum, try the snippet on it and tell me what you find.

Top comments (0)