A keyword gap is the list of Google searches your competitors rank for while your site doesn't show up at all. It's one of the quickest ways to find content worth writing: someone has already shown there's demand and that the query can be won.
Most SEO suites put this report behind a monthly plan. This post shows a pay-per-result way to get the same list from Python, what a real run returns, and where the data falls short.
How a keyword gap is built
The method is simple once you have ranking data:
- Pull the keywords each competitor ranks for in the top N (say, top 20) in one country.
- Pull what your own domain ranks for in the top 100.
- Keep the competitor keywords your domain is missing from, merge duplicates across competitors, and sort by volume or estimated traffic.
The hard part is step 1: you need a database of Google rankings per domain. Scraping Google for thousands of queries yourself is slow and gets blocked, so in practice you query a ranking database.
Example: one site against two competitors
Disclosure: we (SteadyScrape) built the tool below and get paid when it's used.
Competitor Keyword Gap: Missing & Ranked Keywords (steady-scrape/competitor-keywords-gap) is an Apify Actor that runs those steps against a licensed Google ranking database and returns one row per keyword.
import os
from apify_client import ApifyClient
client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("steady-scrape/competitor-keywords-gap").call(run_input={
"yourDomain": "mangools.com",
"competitors": ["semrush.com", "moz.com"], # leave empty to use your top 3 search competitors
"mode": "gap",
"location": "US",
"language": "en",
"maxRowsPerDomain": 150, # keywords fetched per competitor, biggest volume first
})
for row in client.dataset(run["defaultDatasetId"]).iterate_items():
print(row["keyword"], row["searchVolume"], row["keywordDifficulty"],
row["bestCompetitor"], row["bestPosition"], row["competitorCount"])
That's the Actor's prefilled example input. Here's what it returned in our test run on November 3, 2026:
- 287 gap keywords in about 4 seconds: 150 fetched for each competitor (out of 44,530 and 8,144 keywords available for them in the US top 20), merged into one list.
- 13 keywords were ranked by both competitors, mostly backlink-checker searches. For example, "check incoming links" (135,000 monthly US searches, difficulty 51) has semrush.com at #3 and moz.com at #5, and mangools.com isn't in the top 100.
- Median search volume across the 287 rows: 40,500.
- Cost: $1.19 at list price (breakdown below).
Each row includes search volume, CPC, competition, keyword difficulty (0-100), search intent, and for every competitor that ranks: position, ranking URL and estimated traffic. trafficPotential is the best competitor's estimated traffic for that keyword.
Turning the raw list into a content plan
The raw gap is noisy. Big domains rank for plenty of searches that have nothing to do with your product: in this run, some of the highest-volume rows were names of unrelated websites and generic tools that one competitor happens to rank for. A few filters fix most of it:
-
minCompetitors: 2keeps only keywords several competitors rank for. Here that cut 287 rows to the 13 shared ones, and nearly all of them were on-topic. -
excludeNavigational: truedrops searches for a specific site or brand. Intent labels come from a model, so a few will still slip through. -
keywordContains(e.g."backlink") andmaxKeywordDifficultynarrow it to topics you can realistically win. -
searchIntentofcommercialortransactionalusually maps to landing pages;informationalmaps to blog posts.
The Actor also has three other modes: shared (keywords you both rank for, and who's ahead), ranked (everything one domain ranks for, up to 10,000 keywords) and competitors (who your real search competitors are).
Tracking it over time
With compareWithPreviousRun on (the default), the second and later runs with the same settings mark each row new, up, down or same, and the run summary lists keywords that dropped out. Schedule it weekly and you get a running view of what competitors are gaining.
Limits worth knowing
- The data is a monthly-refreshed snapshot of Google rankings, not a live search. Positions can be a few weeks old.
- Organic Google results only: no ads and no other search engines. One country and language per run.
- Search volumes, difficulty and traffic are estimates, not the competitors' analytics.
-
maxRowsPerDomainfetches the biggest-volume keywords first, so a small limit gives you the head of the list, not the long tail. - Small or new domains may have few ranked keywords to compare.
What it costs
| Event | Price |
|---|---|
| Keyword result (one row in your dataset) | $0.004 ($4 per 1,000) on the Apify Free plan; $0.0036 on Starter, $0.0032 on Scale, $0.003 on Business and above |
| Domain analyzed (each competitor or domain query that returned data) | $0.02 |
Apify also adds a $0.00005 start fee. Domains with no data or errors aren't charged, and in gap mode a keyword ranked by several competitors is one row, so you pay for it once.
Examples at the Free-plan price:
- The test run above: $0.00005 + 2 × $0.02 + 287 × $0.004 = $1.19.
- Gap against 1 competitor, 100 keywords: $0.02 + 100 × $0.004 = $0.42 (plus the start fee).
- Every keyword one domain ranks for, 1,000 rows: $0.02 + 1,000 × $0.004 = $4.02 (plus the start fee).
You can set a maximum cost per run in Apify's run options; the Actor then fetches only as many keywords as that limit covers.
Which to use
If you already pay for an SEO suite, its gap report does the same job in a UI. If you only need a gap list now and then, want it in a script, a spreadsheet or a scheduled job, or are feeding it into another pipeline, paying per keyword is usually cheaper. Either way, filter the raw list before you write anything: the useful part is the shared, on-topic keywords, not the biggest numbers.
Try it on the Apify Store: the prefilled example input runs the same comparison shown above. To check search volume for a longer list of ideas afterwards, there's also Keyword Search Volume.
Top comments (1)
월별 스냅샷이며 maxRowsPerDomain이 큰 검색량부터 자른다는 한계를 적어 둔 점이 유용하네요. compareWithPreviousRun에서 키워드가 사라졌을 때, 실제 순위 하락과 이번 조회 상한 밖으로 밀려난 경우를 어떻게 구분하나요? 데이터 기준 월·조회 상한·필터를 결과 요약에 함께 남기면 주간 실행끼리의 차이를 해석하기 쉬울 것 같습니다.