Your annual salary median can mix currencies and pay types

Your remote-pay chart can have a perfectly valid median and no coherent unit.

You're about to put salaryAnnualMin on a slide comparing roles. The column name suggests the awkward conversion work is finished, which is a comforting thought until someone asks what the numbers actually measure.

Annualised salary fields standardise time, not currency or compensation type. The Actor's salary filters compare published annualised amounts without converting currencies, while salaryType distinguishes base, ote, total-comp and hourly.

That median needs more than a successful numeric cast.

What does a salary record represent?

With Dataspryva's salary Actor, a record represents a job posting, deduplicated by its canonical job URL. It represents advertised terms, rather than an employee, an accepted offer or a payment.

That distinction matters before any arithmetic. A median of advertised lower bounds answers a different question from a median of employee earnings.

This Example input keeps postings without parsed pay and leaves salary thresholds unset, so the collection preserves the population needed to measure missingness.

import os

import duckdb
import pandas as pd
from apify_client import ApifyClient

run_input = {
    "mode": "browse",
    "categories": [],
    "requireSalary": False,
    "includeCompanyProfile": False,
    "maxItems": 5000,
}

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor(
    "dataspryva/we-work-remotely-job-listings-and-salary-extractor"
).call(run_input=run_input)

if run["status"] != "SUCCEEDED":
    raise RuntimeError(f"Actor ended with status {run['status']}")

items = []
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    items.append(item)

The input's maxItems is a collection ceiling. Your eventual denominator is the returned collection under those settings, not a claim about every remote vacancy.

The fields separate amount from meaning

The salary threshold descriptions use USD wording, but the currency limitation still applies. Setting minSalary doesn't create a common-currency sample.

Compensation needs its own split. OTE includes contingent earnings, so it can't stand in for guaranteed base pay. Total compensation can include components beyond base salary, while hourly compensation needs a conversion before an annual comparison.

The useful path through these fields is a split, followed by arithmetic:

advertised pay
    |
    +-- currency label
           |
           +-- compensation type
                  |
                  +-- annual lower and upper bounds

Here's the field map we need, including the pandas types we'll use and the question each field leaves open.

Field	Value and pandas treatment	Analytical catch
salary	Original salary text, kept as string alongside the untouched record	Useful for inspection, unsuitable for direct aggregation
salaryDetails	Object containing min, max, currency, unit; retain as object	A nested representation of the range, not an additional salary observation
salaryMin, salaryMax	Flattened range amounts, parsed to nullable Float64	Their time basis still comes from the period fields
salaryCurrency	Currency label, normalised to uppercase string	A currency symbol alone would not establish a unique currency
salaryUnit	YEAR, MONTH, WEEK, DAY or HOUR, kept as uppercase string	Equal amounts with different units are different quantities
salaryPeriod	Lowercase period text, kept as string	A period label does not establish paid hours or weeks
salaryType	base, ote, total-comp or hourly, kept as lowercase string	A common currency does not make these compensation concepts interchangeable
salaryAnnualMin, salaryAnnualMax	Annualised endpoints, parsed to nullable Float64	These remain advertised bounds, not observed earnings
salarySource	description or detail, kept as lowercase string	Extraction provenance does not certify the employer's pay claim

Empty fields are omitted. On jobs without a confidently parsed range, salary fields can therefore be absent rather than explicitly null.

Absence doesn't come with a reason code that separates undisclosed pay from text that could not be parsed confidently. We'd rather preserve that uncertainty than turn a blank into a salary.

Parse amounts without guessing their meaning

The loader below handles omitted keys explicitly. It retains the original records and keeps salaryDetails intact, rather than merging overlapping representations and quietly choosing a winner.

It also normalises labels without trying to infer currency from the salary text.

raw = pd.DataFrame(items)

text_fields = [
    "salary", "salaryCurrency", "salaryUnit",
    "salaryPeriod", "salaryType", "salarySource",
]
amount_fields = [
    "salaryMin", "salaryMax", "salaryAnnualMin", "salaryAnnualMax",
]

pay = raw.reindex(
    columns=text_fields + amount_fields + ["salaryDetails"]
).copy()

def clean_text(values):
    return values.astype("string").str.strip().replace("", pd.NA)

pay["salary"] = pay["salary"].astype("string")

for field in text_fields:
    if field != "salary":
        pay[field] = clean_text(pay[field])

for field in ["salaryCurrency", "salaryUnit"]:
    pay[field] = pay[field].str.upper()

for field in ["salaryPeriod", "salaryType", "salarySource"]:
    pay[field] = pay[field].str.lower()

For amounts, we accept ordinary decimal notation and correctly grouped commas. Currency symbols, abbreviated amounts and unfamiliar separators remain parse failures.

That's deliberately conservative: removing every non-numeric character is an excellent way to manufacture a number with confidence.

This cell parses the flattened amounts while recording which non-empty values failed conversion.

def parse_amount(values):
    text = clean_text(values)
    valid = text.str.fullmatch(
        r"[+-]?(?:[0-9]+|[0-9]{1,3}(?:,[0-9]{3})+)(?:\.[0-9]+)?",
        na=False,
    )
    cleaned = text.where(valid).str.replace(",", "", regex=False)
    return pd.to_numeric(cleaned, errors="coerce").astype("Float64")

parse_failures = pd.DataFrame(index=pay.index)

for field in amount_fields:
    supplied = clean_text(pay[field]).notna()
    pay[field] = parse_amount(pay[field])
    parse_failures[field] = supplied & pay[field].isna()

We don't recompute annual amounts from salaryMin or salaryMax. In particular, annualising hourly pay requires assumptions about paid hours and weeks that aren't supplied here.

Hourly records remain identifiable through salaryType; accepting their supplied annualisation doesn't verify those assumptions.

Missing pay changes who the median describes

Setting requireSalary to true keeps only confidently parsed salary ranges. That's useful for a pay-only export, but it removes the records you need to calculate salary coverage within the collected postings.

Salary thresholds have another wrinkle: jobs without disclosed pay remain unless requireSalary is enabled. A salary-filtered collection can therefore mix threshold-matching salaries with jobs whose pay is unknown.

For this analysis, leaving those filters off lets you distinguish the collection denominator from the salary denominator.

The distinction survives after parsing. A lower-bound median uses records with usable lower bounds; an upper-bound median uses records with usable upper bounds. Those populations need not match.

A missing upper bound isn't permission to copy the lower bound. That would convert uncertainty into a point estimate, neatly enough to pass through most charts.

Keep exceptions beside the comparison

Our comparison policy requires a currency label with the expected letter-code shape and a recognised compensation type. The currency check validates shape, not membership in an authoritative currency registry.

We also hold out malformed amounts, negative annual amounts, reversed annual endpoints and unexpected salaryDetails shapes. A record with a usable endpoint can remain eligible when the other endpoint is missing.

This cell builds an exceptions table and a coverage table, retaining an explicit unknown group for unusable classifications.

currency_ok = pay["salaryCurrency"].str.fullmatch(
    r"[A-Z]{3}", na=False
)
type_ok = pay["salaryType"].isin(
    ["base", "ote", "total-comp", "hourly"]
)

annual_fields = ["salaryAnnualMin", "salaryAnnualMax"]

flags = pd.DataFrame({
    "missing_currency": pay["salaryCurrency"].isna(),
    "malformed_currency": pay["salaryCurrency"].notna() & ~currency_ok,
    "missing_type": pay["salaryType"].isna(),
    "unrecognised_type": pay["salaryType"].notna() & ~type_ok,
    "amount_parse_failure": parse_failures.any(axis=1),
    "no_annual_bound": pay[annual_fields].isna().all(axis=1),
    "negative_annual_bound": pay[annual_fields].lt(0).any(axis=1),
    "reversed_annual_bounds": (
        pay["salaryAnnualMin"] > pay["salaryAnnualMax"]
    ).fillna(False),
    "malformed_details": (
        pay["salaryDetails"].notna()
        & ~pay["salaryDetails"].map(lambda value: isinstance(value, dict))
    ),
})

eligible = ~flags.any(axis=1)
exceptions = raw.join(flags.add_prefix("audit_")).loc[~eligible].copy()

coverage_rows = pay[["salaryCurrency", "salaryType"]].copy()
coverage_rows["salaryCurrency"] = pay["salaryCurrency"].where(
    currency_ok, "unknown"
)
coverage_rows["salaryType"] = pay["salaryType"].where(
    type_ok, "unknown"
)
coverage_rows["eligible"] = eligible
coverage_rows["usable_lower"] = eligible & pay["salaryAnnualMin"].notna()
coverage_rows["usable_upper"] = eligible & pay["salaryAnnualMax"].notna()

coverage = coverage_rows.groupby(
    ["salaryCurrency", "salaryType"], dropna=False
).agg(
    collected_postings=("eligible", "size"),
    eligible_postings=("eligible", "sum"),
    usable_lower_bounds=("usable_lower", "sum"),
    usable_upper_bounds=("usable_upper", "sum"),
).reset_index()

The original values remain in exceptions, alongside the reasons for holding them out. Missing classifications stay visible in coverage instead of disappearing through a group-by default.

These exclusions are an analyst's policy, not a discovery that the underlying posting is wrong. A reversed range deserves inspection before it deserves a swap.

Compare lower bounds with lower bounds

Now the statistic can have a precise name: the median advertised annual lower bound within a currency and compensation group.

The upper-bound distribution gets a separate measure. Averaging each posting's endpoints would create an analyst-derived midpoint, not recover an employer's expected offer.

This DuckDB query calculates separate medians and endpoint denominators from eligible records.

comparison = pay.loc[
    eligible,
    ["salaryCurrency", "salaryType"] + annual_fields,
].copy()

connection = duckdb.connect()
try:
    connection.register("salary_comparison", comparison)
    bound_summary = connection.execute("""
        SELECT
            salaryCurrency,
            salaryType,
            COUNT(*) AS eligible_postings,
            COUNT(salaryAnnualMin) AS lower_bound_n,
            MEDIAN(salaryAnnualMin) AS median_annual_lower_bound,
            COUNT(salaryAnnualMax) AS upper_bound_n,
            MEDIAN(salaryAnnualMax) AS median_annual_upper_bound
        FROM salary_comparison
        GROUP BY salaryCurrency, salaryType
        ORDER BY salaryCurrency, salaryType
    """).df()
finally:
    connection.close()

For a role comparison, those groups are necessary but insufficient. You still need a defensible role classification and comparable employment arrangements; a title string alone doesn't establish equivalent work.

Keep coverage beside the resulting table. A median without its contributing count conceals whether your comparison is mostly describing disclosed pay or mostly describing missingness.

Where the pay claim stops

The worked record comes from We Work Remotely Job Listings & Salary Extractor.

Its salary fields describe published posting terms. They can't establish what applicants were offered, what employees earned, whether contingent compensation paid out, or which undisclosed salaries would change your result.

Nor does grouping by currency resolve purchasing power or local labour-market differences. Those require additional evidence and an explicitly chosen comparison method.

Your chart caption can be precise: median advertised annual lower bounds among eligible, salary-parsed postings in each currency and compensation group, with endpoint counts and collection coverage shown separately.

That's a narrower claim than “remote roles pay more”.

It's also a claim you can defend.