DEV Community

Eyvind Barthelemy
Eyvind Barthelemy

Posted on

The API promised industry keywords. Modern sites kept returning [].

#devchallenge #bugsmash #javascript #web3

Summer Bug Smash: Smash Stories 🐛🛹

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry.

The quietest kind of production bug

The endpoint was healthy. Payment gating worked. The response matched its JSON schema. Nothing crashed, timed out, or lit up a red dashboard.

It was still broken.

I had launched a pay-per-call /enrich endpoint for AI agents. Give it a domain and it returns structured public company intelligence: identity, contact surfaces, technology signals, DNS and email infrastructure, social profiles, AI-search readiness—and company.keywords, advertised as industry keywords an agent could use for routing or qualification.

For modern sites, that last field kept coming back as this:

{
  "company": {
    "name": "...",
    "keywords": []
  }
}
Enter fullscreen mode Exit fullscreen mode

The API had returned a successful, paid, structurally valid answer that omitted one of the signals it promised.

That made this bug more interesting than a 500 error. Every shallow health check passed. Only checking whether the output was useful revealed the failure.

Root cause: a legacy assumption hiding behind a modern schema

The original implementation populated the field from one HTML tag:

const keywords = (meta.keywords || "")
  .split(",")
  .map((value) => clean(value))
  .filter(Boolean)
  .slice(0, 15);
Enter fullscreen mode Exit fullscreen mode

That means the entire field depended on:

<meta name="keywords" content="payments, infrastructure, fintech">
Enter fullscreen mode Exit fullscreen mode

Many contemporary sites do not publish that legacy tag. They express topics elsewhere: JSON-LD keywords, knowsAbout, applicationCategory, article tags, titles, and descriptions.

My parser had confused one old representation with the underlying concept. The schema said “industry keywords”; the code actually meant “the value of this one optional meta tag.”

The first tempting fix was the wrong one

The easy patch would have been to split the page title and return the first few long words. That would make the array non-empty, but it would blur together two different kinds of evidence:

  • topics the publisher explicitly declared; and
  • terms the service derived from public copy.

For an autonomous buyer, that distinction matters. A derived label should not masquerade as a company-authored claim.

So the fix needed to improve coverage and preserve provenance.

The repair: declared first, deterministic fallback second

The corrected pipeline has three stages.

1. Collect declared signals

It reads:

  • meta[name="keywords"];
  • news_keywords;
  • article:tag;
  • JSON-LD keywords;
  • JSON-LD knowsAbout; and
  • JSON-LD applicationCategory.

The JSON-LD walk handles nested graphs, arrays, strings, and { "name": "..." } objects.

2. Normalize without laundering the evidence

Declared values are cleaned, capped at 40 characters, deduplicated case-insensitively, and limited to 15 public response items.

If fewer than six declared topics exist, the service deterministically derives a top-up from title and description text. It removes stopwords and numeric noise, ranks by frequency, and preserves first-appearance order as the tie-breaker.

There is no model call and no random generation. The same public input produces the same result.

3. Label the provenance

The response now states where its list came from:

{
  "company": {
    "keywords": [
      "stripe",
      "financial",
      "infrastructure",
      "grow",
      "revenue",
      "services",
      "platform",
      "types"
    ],
    "keywordsSource": "derived"
  }
}
Enter fullscreen mode Exit fullscreen mode

The label is one of:

  • declared: the page provided a complete set;
  • mixed: declared topics plus a deterministic top-up; or
  • derived: no declared topics were available.

That small field keeps the improvement honest. A consuming agent can accept derived signals for discovery, or require declared signals for a higher-stakes decision.

Before and after 

BEFORE
legacy meta keywords absent
        └─ company.keywords = []
             └─ HTTP 200, schema valid, weak answer

AFTER
declared meta + JSON-LD topics
        ├─ enough signals ──────────────> declared
        └─ sparse or absent
             └─ deterministic top-up ──> mixed / derived

all paths: case-insensitive dedupe + 15-item cap + provenance label
Enter fullscreen mode Exit fullscreen mode

The endpoint still uses only public page data. The payment rail, input contract, URL security guard, and all unrelated response fields are unchanged.

Locking the semantic contract down

The original repair shipped on June 24. While preparing this retrospective, I noticed the lesson had not been encoded in a focused test. That is exactly how a quiet semantic bug can return during a later refactor.

I extracted the keyword pipeline behind a pure keywordSignals(...) seam and added four deterministic regression cases:

  1. No legacy metadata still produces a non-empty derived set.
  2. Meta and nested JSON-LD topics merge and deduplicate before a top-up.
  3. Six declared topics remain purely declared; marketing copy is not appended.
  4. The public response cap remains 15 after normalization.

The full current suite also preserves the gateway's referral safety tests:

$ npm test
tests 8
pass 8
fail 0
Enter fullscreen mode Exit fullscreen mode

A live keyless CLI probe against stripe.com now returns eight keywords and keywordsSource: "derived" rather than the empty field that exposed the bug.

Code and evidence

What I learned

A valid payload can still violate the product promise

Status codes, JSON parsing, and schema validation prove transport and shape. They do not prove that the signal a customer is buying is present or useful.

For data products, a test should assert semantic invariants: the advertised field is populated for representative modern inputs, its provenance is visible, and its bounds remain stable.

Prefer concepts over historical representations

“Industry topics” can appear in several public formats. Binding the concept to one legacy tag made the implementation brittle even though the code looked reasonable in isolation.

Derived data needs an honesty label

Filling an empty field is not enough. The consumer should know whether a value was declared, inferred deterministically, or generated. Provenance turns a fallback from a hidden guess into a usable choice.

The lasting fix was not merely making [] non-empty. It was making the endpoint say what it knows, how it knows it, and keeping that contract executable in tests.

Top comments (0)