DEV Community

Oleg Starnikov
Oleg Starnikov

Posted on

A key that isn't there: reading job postings without KeyError

#python #webdev #api #tutorial

Your loop works for nine hundred rows and then throws KeyError: 'city'.

Nothing broke. The row simply has no city, and the API left the key out instead of sending null. A posting that is remote only, or whose vendor never filled the field, arrives without it. Same for salary_text, for region, for remote_flag.

This is a small thing that costs an afternoon the first time, so here is the shape of it, with code you can run.

Read the optional fields with a fallback that means something

try/except around the whole loop hides which field was missing. .get() per field, with a fallback you choose deliberately, keeps the row usable:

import os, requests

HOST = "hiringindex.p.rapidapi.com"
H = {"x-rapidapi-key": os.environ["RAPIDAPI_KEY"], "x-rapidapi-host": HOST,
     "content-type": "application/json"}

# One page of postings. A key is left out of a row when that posting has no
# value for it, so every optional field is read with .get() and a fallback.
# row["city"] would raise KeyError on a remote-only posting.
r = requests.post(f"https://{HOST}/jobs/search", headers=H,
                  json={"keywords": ["python"], "limit": 20})
r.raise_for_status()

for row in r.json()["jobs"][:3]:
    where = row.get("city") or row.get("location_raw") or "location not given"
    print(f"{row['title'][:36]:<36} {where[:20]:<20} {row.get('salary_text', 'not stated')}")
Enter fullscreen mode Exit fullscreen mode

Note the order in where: city first, then location_raw, which is the string the employer's board actually published, and only then a constant. Falling back to the raw location keeps rows that a naive city lookup would have dropped.

Output from that run:

Thermofluids Analysis Intern - Summe Los Angeles          $34 per hour
Robotics Engineer: Foundation Model  Redmond              not stated
GPU Kernel Engineer                  San Francisco        $190K – $250K • Offers Equity
Enter fullscreen mode Exit fullscreen mode

When you want numbers, do not page for them

The second mistake is paging through rows to compute an average. The aggregates come from one call against the same filter, and they arrive already grouped by currency, city, platform and seniority.

One detail worth knowing before you write the retry: the endpoint answers 202 while the slice is still being computed, with a hint for how long to wait. Retry once after that delay rather than hammering it.

body = {"keywords": ["python"], "percentiles": True}
r = requests.post(f"https://{HOST}/jobs/insights", headers=H, json=body)
if r.status_code == 202:
    import time
    time.sleep(30)
    r = requests.post(f"https://{HOST}/jobs/insights", headers=H, json=body)

d = r.json()
head = d["headline"]
usd = next(s for s in d["salary"] if s["currency"] == "USD")
print(f"{head['row_count']} rows, {head['company_count']} employers, "
      f"{head['with_salary']} rows state a salary")
print(f"USD n={usd['count']}: lower band p50 {usd['min']['p50']}, "
      f"upper band p50 {usd['max']['p50']}")
print(f"median days a posting stays live: {d['freshness']['median_days_live']}")
Enter fullscreen mode Exit fullscreen mode

Output, measured 2026-09-14 09:48 UTC on the keywords=python slice:

78115 rows, 7184 employers, 4667 rows state a salary
USD n=3248: lower band p50 168000, upper band p50 237550
median days a posting stays live: 47
Enter fullscreen mode Exit fullscreen mode

Three numbers worth reading twice. The USD group holds 3 248 postings that state a band, and the median of their lower bounds is 168 000 while the median of their upper bounds is 237 550, so the middle posting advertises roughly 168k to 237k. And the median posting in that slice has been live for 47 days, which is a useful sanity check on anything you build on top: a "new this week" feature over a corpus with a 47 day median needs the posting date, not the fetch date.

What to take away

  • A missing key is not an error in the data. Read optional fields with .get() and pick a fallback per field, not one for the whole row.
  • Prefer the published raw value over a parsed one as your fallback. location_raw saved two of the three rows above.
  • Aggregates belong in the aggregate call. Paging to compute a median is slower and gives you a worse median.

The API used here is HiringIndex, which reads postings from thirteen applicant tracking systems and keeps the employer's own apply link on every row. I work on it. The free plan is 200 postings plus 5 insights calls a month, no card: rapidapi.com/starnikovoleg/api/hiringindex

Written with AI assistance (Claude). Every number above, and both output blocks, come from the live calls shown in the code.

Written with an AI assistant; every number and code snippet was run against the live HiringIndex API by the team before publishing. <!-- ai-disclosure -->

Top comments (0)