Most SEO dashboards look reassuring right up until someone asks a very ordinary question: “Why is this keyword missing today?”
That is where a lot of home-grown rank trackers become awkward. A blank cell might mean the site dropped out of the results you fetched. It might also mean the request timed out, the response changed shape, or yesterday’s check used a different locale. If all of those cases become 0, the chart is tidy but the data is fiction.
The useful part of a small rank monitor is not the graph. It is the contract behind every observation.
Treat a ranking as an observation, not a universal fact
A search position only makes sense alongside the query, country, language, result type, page, requested result count, and timestamp. Change one of those and you have started a different series.
This also means “not present in the response” is not the same as “ranked 11th.” If you requested ten organic results and did not find your domain, all you know is that it was absent from those ten results. You do not know its next position.
A practical monitor should preserve three outcomes:
-
found: the request was valid and the target host appeared -
not_found: the request was valid, but the host was absent from the returned organic results -
error: transport, decoding, authentication, or schema validation failed
I normally keep position nullable. Values such as 0 and 999 are tempting because spreadsheets like numbers, but they contaminate averages and make genuine failures look like spectacular ranking losses.
Keep matching boring and explicit
Substring checks are an easy source of false positives. Searching for example.com inside a URL can also match notexample.com. Parse the hostname instead, normalize it, then decide whether subdomains count.
Here is the small piece I would test before building anything else:
from urllib.parse import urlparse
def hostname(value: str) -> str:
candidate = value if "://" in value else f"https://{value}"
parsed = urlparse(candidate)
return (parsed.hostname or "").lower().rstrip(".")
def belongs_to(url: str, target: str, include_subdomains=True) -> bool:
result_host = hostname(url)
target_host = hostname(target)
if not result_host or not target_host:
return False
if result_host == target_host:
return True
return include_subdomains and result_host.endswith(f".{target_host}")
Whether shop.example.com belongs to example.com is a product decision, not a parsing decision. Make that choice visible in code. If you are tracking a specific landing page rather than an entire site, add a separate path-normalization rule instead of quietly mixing page-level and domain-level positions.
Store evidence before you design the dashboard
For a lightweight setup, SQLite is often enough. The table matters more than the chart:
CREATE TABLE IF NOT EXISTS rank_checks (
checked_at TEXT NOT NULL,
keyword TEXT NOT NULL,
target_host TEXT NOT NULL,
country TEXT NOT NULL,
language TEXT NOT NULL,
result_page INTEGER NOT NULL,
requested_count INTEGER NOT NULL,
returned_count INTEGER,
status TEXT NOT NULL,
position INTEGER,
landing_url TEXT,
error_kind TEXT,
raw_response TEXT
);
Saving returned_count helps explain odd days when a provider returns fewer records than requested. Keeping the raw response gives you something to inspect when your parser and the API disagree. That payload should stay in protected storage, of course; it should not be dumped into a public report or an unrestricted log.
The collection loop can stay deliberately plain. Send a fixed request to the Google SERP endpoint, validate that the organic collection exists and has the expected types, match hosts, and write exactly one row for the check. Credentials belong in an environment variable or secret manager, never in the script or keyword file.
import os
import requests
payload = {
"type": "search",
"query": "reusable travel tumbler",
"country": "us",
"language": "en",
"page": 1,
"number": 10,
}
response = requests.post(
"https://api.acedata.cloud/serp/google",
headers={"Authorization": f"Bearer {os.environ['ACEDATACLOUD_API_KEY']}"},
json=payload,
timeout=(10, 60),
)
response.raise_for_status()
data = response.json()
organic = data.get("organic")
if not isinstance(organic, list):
raise ValueError("The response does not contain a valid organic list")
Do not swallow that exception and write not_found. A missing or malformed organic field is a broken observation. An empty but valid list is different: the request succeeded and returned no organic rows.
Scheduling is the easy part
Once a single-keyword integration test behaves correctly, running it daily is just operations work. Cron, a CI scheduler, or a managed job runner can all do it. The important detail is that the scheduled process must use the same configuration every day and must actually receive the secret.
A shell session having ACEDATACLOUD_API_KEY does not mean cron inherits it. Run the job as the deployment user, lock down logs, prevent overlapping executions, and make the process exit non-zero if any request fails. That last detail turns silent holes into alerts.
Retries deserve restraint. A bounded retry with backoff can help with transient failures, but endless retries can inflate usage and hide an outage. I prefer to make the failure visible first, then introduce a small retry policy based on the provider’s current rate limits.
Read reliability before movement
When the data reaches a dashboard, start with collection health. How many checks succeeded? Did the returned result count change? Were the locale and request size identical?
Only compare positions when both observations are found and their search conditions match. Moving from position eight to five is a three-position improvement between those two observations. Moving from eight to not_found is not a measurable drop to some invented rank; it is simply an absence from the fetched set.
That distinction sounds fussy until someone uses the report to judge a release, a writer, or an agency. Honest uncertainty is more useful than a precise-looking number that the system never observed.
A small tracker built this way will not prove that a page edit caused a ranking change, and it will not improve SEO by itself. What it gives you is a stable timeline that can be inspected later alongside deployments, content changes, and analytics. Start with a handful of business-critical terms, let it run for a week, and check the strange rows before adding more keywords or a prettier chart.
Top comments (0)