I scored 942 user complaints for commercial intent. The data embarrassed me.

I have abandoned enough side projects to accept that my ideas were the problem. So instead of having ideas, I spent a few months reading other people's complaints: Reddit, Hacker News, GitHub, Stack Exchange, Trustpilot, app store reviews, product forums, X.

The pipeline is six stages.

1. Source from public complaints only. No surveys, no private data.
2. Prefilter the obvious noise with keywords and heuristics, before spending money on inference.
3. Deduplicate. This mattered far more than I expected. One thread with 200 upvotes is one person's problem amplified 200 times, not 200 people with the problem. Frequency across distinct authors and sources is the actual signal.
4. Two LLM passes. The first confirms it is a real, software-shaped problem. The second extracts the opportunity, the persona, and what people currently do instead.
5. Score for commercial intent, confidence, and willingness to pay.
6. Editorial gate. Nothing publishes automatically. I read every one.

942 have cleared that.

Three things in the data I did not expect

My corpus is mostly about developers, and that is a bias, not a finding.

Dev Tools and SaaS Infrastructure is 44% of everything I have. My single largest source is GitHub with 563 items, ahead of Reddit at 299. That is not because developer problems are the most valuable. It is because developers complain in public, in writing, on platforms with APIs.

Anyone mining public text has this skew. I had it for weeks before I went looking for it.

The highest-intent category is the one I have almost nothing in.

HR and recruiting averages 80.1/100 commercial intent, the highest of any category with meaningful volume. I have 28 items in it. Dev tools averages 71.9 across 560.

When I dug into why, the answer was worse than "not enough sources". The HR targets existed and were enabled. 11 of the 13 on active sources had never been fetched once, because the ingest budget kept getting consumed by the same high-priority targets every run. I had been mining where it was easy, and the ranking made that permanent.

The good ones are boring.

Top of this week:

[intent 100 / confidence 90]  Automate 40% of property manager after-hours calls
[intent 100 / confidence 90]  Stop Stripe Connect fraud before platform shutdown
[intent  90 / confidence 95]  60-second mobile-first e-signatures, no account

Nobody is going to tweet about building after-hours call automation for property managers. The ideas that sound exciting score badly, over and over.

Querying it from the editor

Your agent can already build almost anything, so the bottleneck moved to deciding what to build. I put the corpus behind an MCP server so I can ask while I am already in the editor:

claude mcp add --transport http ddmarketer https://www.ddmarketer.com/api/mcp

Or in any MCP client config:

{
  "mcpServers": {
    "ddmarketer": {
      "type": "http",
      "url": "https://www.ddmarketer.com/api/mcp"
    }
  }
}

Three tools need no key and no account: search_gaps, get_top_gaps, and validate_idea. That last one scores an idea you already have, and it will tell you when there is nothing there — it returns a weak verdict when it only finds loosely related complaints, rather than pretending a near-miss is validation.

What this does not do

Complaint volume is not demand. Somebody being annoyed on Reddit does not mean they will pay to fix it. The corpus narrows where to look. You still have to talk to people.

The scores come from a language model. They are consistent enough to rank against each other, but I would not read 80 vs 75 as meaningful.

And the editorial gate is one person, which is a quality gate, not peer review.

The question I cannot answer

How do you separate "people complain about this" from "people would pay to fix this"? I score willingness to pay as its own dimension, but I have no ground truth to check it against, and one paying customer is not a validation set.

If you have solved this, I would genuinely like to hear how.