DEV Community

Get Anything
Get Anything

Posted on

How to build a hiring intent score that doesn't lie to you

You have a list of companies and their open roles. You want one number per company: how hard is this company hiring, right now?

That number is easy to compute and easy to get wrong in ways that look plausible. Here's a design that survives contact with real data, and the specific mistakes that inflate every naive version.

The three inputs

Volume. How many roles are open. The most obvious input and the least interesting on its own, because it scales with company size. Ten open roles at a fifty-person startup means something very different from ten at a ten-thousand-person bank.

Seniority mix. What kind of roles. Weight senior and executive postings higher. A company creating a director-level position it never had is starting something; a company posting three interns is running a graduate scheme. Seniority is the closest proxy for "new initiative" that a job title gives you.

Momentum. How the count changed since last time. This is the input everyone skips, because it requires storing state across runs, and it's the one that carries the most information. A company that went from two roles to eight is in a different state from one that has had eight open for a year.

A workable formula

SENIORITY_WEIGHTS = {
    "Internship": 0.2,
    "Entry level": 0.4,
    "Mid level": 0.6,
    "Mid-Senior level": 0.8,
    "Director": 1.0,
    "Executive": 1.0,
}

def hiring_intent(job_count, seniority_mix, delta):
    # Volume: log-ish, so 40 roles doesn't drown out everything else
    volume = min(job_count / 10, 1.0)

    # Seniority: weighted average of the mix
    total = sum(seniority_mix.values()) or 1
    seniority = sum(SENIORITY_WEIGHTS.get(k, 0.5) * v
                    for k, v in seniority_mix.items()) / total

    # Momentum: unknown on the first run, and that's not the same as zero
    momentum = 0.5 if delta is None else min(max(delta, 0) / 5, 1.0)

    return round(100 * (0.5 * volume + 0.3 * seniority + 0.2 * momentum), 1)
Enter fullscreen mode Exit fullscreen mode

Cap volume rather than letting it run linearly, or the largest company in your list wins every time and the score becomes a headcount proxy.

Note delta is None. On the first run you have no previous state. A first run is not a company with zero momentum — it's a company whose momentum you don't know. Scoring those identically means every company looks stagnant on day one, and your first Monday's call list is garbage. Give unknown momentum a neutral value and label the run as a baseline.

The trap that will inflate your score

Seniority weighting is where this breaks, because it depends on a classifier that reads job titles, and job titles are adversarial.

Three bugs I shipped, all of which made companies look more senior than they were:

"coo" in title.lower() matches "Coordinator". Every programme coordinator became a chief operating officer.

"partner" in title.lower() matches "Partner Onboarding Specialist" and "Partnerships Lead". Support staff became firm partners.

"Assistant Vice President" is not an executive. In banking and real estate it's an upper-mid individual contributor title. A single company's AVP postings pushed my executive count up by seven.

The first two are fixed with word boundaries:

re.search(r"\b(ceo|cfo|cto|cmo|coo)\b", title)   # not `"coo" in title`
Enter fullscreen mode Exit fullscreen mode

The third is fixed by ordering your rules so the specific case is caught before the general one:

if "assistant vice president" in t or "avp" in t:
    return "Mid-Senior level"          # must come first
if "vice president" in t or "chief" in t:
    return "Executive"
Enter fullscreen mode Exit fullscreen mode

Together these took my "executive-level roles" figure from 18% of a market down to 16%, and changed which companies ranked highest. I nearly published the wrong number.

Validate by reading, not by eyeballing percentages

The check that catches all three of the above takes one minute:

from collections import Counter
for bucket, count in Counter(r["seniority"] for r in records).most_common():
    print(f"\n=== {bucket}: {count}")
    for r in [x for x in records if x["seniority"] == bucket][:10]:
        print("   ", r["title"])
Enter fullscreen mode Exit fullscreen mode

Print the titles inside each bucket. Every classification bug I have written was invisible as a percentage and obvious as a list. If "Executive" is 18% of your dataset, look at who's in it before you tell anyone.

Make the score legible

A score of 73 is useless on its own. Ship the components alongside it:

{
  "company": "Example Co",
  "hiringIntentScore": 73.0,
  "openRoles": 8,
  "trendDelta": 3,
  "seniorityMix": {"Mid-Senior level": 5, "Director": 2, "Entry level": 1},
  "sampleTitles": ["Director of Engineering", "Senior Data Engineer"]
}
Enter fullscreen mode Exit fullscreen mode

Anyone using the score to decide where to spend their week needs to see why a company ranks where it does. A high score driven by twenty interns is not the same as one driven by three director hires, and the person making the call should be able to tell those apart at a glance.

Weekly, or it means nothing

Momentum requires a previous run. Score once and you have a snapshot with the most valuable input missing. Run it on a fixed weekly schedule, store each run's per-company counts, and compare consecutive runs.

One caution: never persist state from a run where a source failed. A partial fetch writes a truncated baseline, and next week every missing role looks like it closed. Suppress the closure signal on any incomplete run.

Ready-made

I maintain a tracker that does exactly this: aggregates open roles per company, infers seniority and function from titles, compares against the previous scheduled run, and returns a 0–100 hiring intent score with all its components attached.

It's on my Apify page

Top comments (0)