DEV Community

Hsinghsien Chen
Hsinghsien Chen

Posted on

How to Compare a Long Keyword List in Google Trends on One 0-100 Scale

If you've ever tried to chart 12 keywords from Google Trends, you've hit the wall: one comparison only holds a handful of terms (the current Explore page allows up to 8; the older compare box allowed 5). The obvious workaround (run several smaller requests and paste the results together) gives you numbers that look right but mean nothing when you put them side by side.

This post explains why, and walks through the anchor-term rescaling method that fixes it, with a small pandas implementation you can run offline.

Why separate requests aren't comparable

Google Trends doesn't return search volume. It returns a relative index: within a single request, the highest point across all terms in that request becomes 100, and everything else is scaled against it and rounded to an integer.

So "60" in request A and "60" in request B are on different scales. If request A contains a giant term, everything else in A gets squashed toward zero. If request B contains only small terms, one of them gets stretched to 100. Concatenating the two is like merging two charts that use different y-axes without telling anyone.

The fix: a shared anchor term

The trick is to put the same term (the anchor) in every request. Because the anchor's real interest is the same no matter which request it's in, any difference in its numbers between requests comes from the different scaling. That gives you a conversion factor.

The procedure:

  1. Pick an anchor term.
  2. Split your other keywords into groups small enough that each group plus the anchor fits in one request (groups of 4 plus the anchor, 5 terms per request, work under both the old and the new limit).
  3. Fetch each group with identical geo, timeframe, and category.
  4. Pick a reference group (e.g. the first one where the anchor isn't all zeros).
  5. For each group g, compute a factor:
   factor_g = sum(anchor in reference group) / sum(anchor in group g)
Enter fullscreen mode Exit fullscreen mode
  1. Multiply every series in group g by factor_g. Report the anchor once, from the reference group.
  2. Renormalize globally: divide everything by the overall max and multiply by 100, so the biggest point across all keywords is 100 again.

Why sums instead of per-point ratios?

You could compute a ratio for each date and average them, but each data point is an integer from 0 to 100. A point where the anchor reads 2 could really be anywhere from 1.5 to 2.5. That's ±25% on one point. Summing the whole series first and dividing once spreads that rounding error across the full timeline. Points where the anchor reads 0 would also break a per-point ratio (division by zero), but they're harmless in a sum.

A runnable implementation in pandas

The code below has no network calls. The DataFrames are made-up example data so you can see the math work. In real use you'd fill them with whatever your fetcher returns (one DataFrame per request, indexed by date).

import pandas as pd

LOW_ANCHOR_MEAN = 5  # below this, integer rounding error gets amplified


def rescale_groups(groups: list[pd.DataFrame], anchor: str):
    """Put several Google Trends requests (each with its own 0-100 scale)
    onto one shared 0-100 scale using an anchor term present in every group.

    Returns (combined DataFrame, factors dict, warnings list).
    """
    warnings, factors, scaled = [], {}, []

    # Reference = first group where the anchor has any signal at all.
    ref_idx = next((i for i, g in enumerate(groups) if g[anchor].sum() > 0), None)
    if ref_idx is None:
        raise ValueError("anchor is zero in every group; cannot rescale")
    ref = groups[ref_idx]
    ref_sum = ref[anchor].sum()

    for i, g in enumerate(groups):
        # Groups must share the exact same date axis to be comparable.
        if not g.index.equals(ref.index):
            warnings.append(f"group {i}: date axis differs from reference; skipped")
            continue
        s = g[anchor].sum()
        if s == 0:
            warnings.append(f"group {i}: anchor is all zeros; skipped")
            continue
        mean = s / len(g)
        if mean < LOW_ANCHOR_MEAN:
            warnings.append(
                f"group {i}: anchor mean {mean:.2f} < {LOW_ANCHOR_MEAN}; "
                "rounding error is amplified, consider a more popular anchor"
            )
        factors[i] = float(ref_sum / s)
        cols = g.columns if i == ref_idx else g.columns.drop(anchor)
        scaled.append(g[cols] * factors[i])

    combined = pd.concat(scaled, axis=1)
    peak = combined.to_numpy().max()
    combined = (combined * (100 / peak if peak > 0 else 0)).round(2)
    return combined, factors, warnings


# ---- EXAMPLE DATA (made up for illustration, not real Google Trends values) ----
dates = pd.date_range("2026-01-04", periods=6, freq="W")

# Request 1: anchor + two big terms. "term_a" sets this request's 100.
group0 = pd.DataFrame(
    {"anchor": [20, 22, 21, 19, 23, 20],
     "term_a": [80, 90, 100, 85, 95, 88],
     "term_b": [40, 42, 45, 41, 39, 44]},
    index=dates,
)

# Request 2: anchor + two small terms. With no giant term here,
# the anchor itself is stretched toward 100.
group1 = pd.DataFrame(
    {"anchor": [87, 96, 91, 83, 100, 87],
     "term_c": [30, 35, 33, 31, 36, 34],
     "term_d": [10, 12, 11, 9, 13, 10]},
    index=dates,
)

combined, factors, warnings = rescale_groups([group0, group1], anchor="anchor")

print("factors:", {k: round(v, 4) for k, v in factors.items()})
print("warnings:", warnings or "none")
print(combined)
Enter fullscreen mode Exit fullscreen mode

Running it (Python 3, pandas) prints:

factors: {0: 1.0, 1: 0.2298}
warnings: none
            anchor  term_a  term_b  term_c  term_d
2026-01-04    20.0    80.0    40.0    6.89    2.30
2026-01-11    22.0    90.0    42.0    8.04    2.76
2026-01-18    21.0   100.0    45.0    7.58    2.53
2026-01-25    19.0    85.0    41.0    7.12    2.07
2026-02-01    23.0    95.0    39.0    8.27    2.99
2026-02-08    20.0    88.0    44.0    7.81    2.30
Enter fullscreen mode Exit fullscreen mode

In raw request 2, term_c peaked at 36 and looked like it might be close to term_b. Once it's on the shared scale you can see it's really around 7-8, a small fraction of term_b. That's exactly the kind of wrong conclusion that naive concatenation produces.

Note the output has decimals. That's expected: rescaled values are estimates, not Google's native integers.

Pitfalls (the part that actually bites)

1. An anchor with too little volume

This is the big one. If your anchor averages, say, 3 in a group, each point's ±0.5 rounding is a huge relative error, and the factor inherits it. Every keyword in that group is then multiplied by a noisy factor. If the anchor rounds to 0 across a whole group, you can't rescale that group at all.

In the implementation I base this on, a group whose anchor mean is below 5 triggers a warning, and a group whose anchor is all zeros is marked as failed instead of guessed. I'd recommend both behaviors: flag it, don't invent numbers.

2. Choosing the anchor

A good anchor is:

  • Steady: no huge spikes, so it isn't the 100 in some groups and tiny in others for reasons unrelated to scaling.
  • Fairly popular in your geo and timeframe, so it stays well above zero in every group.
  • Not dwarfed by your biggest keyword. If one keyword is 50× the anchor, the anchor gets crushed toward 0 in that keyword's group.

A practical check: after a run, look at the anchor's mean per group and at the factors. Factors far from 1 combined with a low anchor mean are where the error lives.

3. Rounding error is real, even when everything is done right

Rescaling can't recover precision that Google rounded away. Low-volume keywords in a group with a large factor carry roughly ±0.5 × factor of uncertainty per point. Treat small differences between small keywords with suspicion.

4. Sampling noise

Google Trends is computed from a sample of searches, and recent data gets revised (the latest point is often flagged as partial). Two requests made at different times can return slightly different values for the same term. Fetch all your groups in one session, close together in time, and consider dropping partial points before comparing.

5. Timeframe, geo, and category must match exactly

The anchor only links groups if every request uses the same geo, timeframe, and category. Watch out for the date axis too: if one group comes back with a different set of dates, don't try to line it up. Exclude it. The implementation above (and the one I based it on) requires an identical date axis and skips any group that doesn't match.

6. Rate limits and HTTP 429

If you're using pytrends, you will meet TooManyRequestsError / HTTP 429 sooner or later, especially in loops. What helps:

  • Send requests one at a time with a delay between them (seconds, not milliseconds).
  • Retry only on 429 / 5xx / timeouts, with exponential backoff plus jitter, and honor Retry-After if present.
  • Don't retry a 400: that's a bad request and retrying won't fix it.
  • Cap the number of retries, and record a failed keyword as failed rather than silently dropping or zero-filling it.

7. Browser scrapers and UI changes

Google has redesigned the Trends pages more than once. Any scraper that drives a browser and reads values out of the rendered page depends on that page's layout and selectors, so a redesign is exactly the kind of change that breaks it. The approach I've had better luck with is to skip the DOM entirely and use the JSON endpoints that the Trends UI itself calls (the same data the "download CSV" button gives you). These are still unofficial and can change without notice, so build in validation either way.

A quick validation you should run

Before you trust a pipeline, compare it to ground truth: pick 5 terms, fetch them in one direct request (Google's own scale), then fetch them again split into anchored groups and rescale. The difference between the two tells you how much error your anchor choice introduces. With a weak anchor you'll see small but non-zero errors on the low-volume terms. That's the rounding issue from pitfall 1, and it's the best way I know to pick an anchor with data instead of guessing.

Wrap-up

  • Each Google Trends request is scaled to its own 0-100 peak, so separate requests can't be compared directly.
  • Put a shared anchor in every request, scale each group by sum(anchor_ref) / sum(anchor_group), then renormalize so the global max is 100.
  • Most of the error comes from the anchor: choose a steady, popular one, warn when its mean is low, and never rescale a group where it's all zeros.
  • Keep geo, timeframe, category, and date axis identical across groups, fetch close together in time, and back off politely on 429s.

This article was written with AI assistance. The code was reviewed before publishing.

Top comments (0)