Quick answer
NIH RePORTER's search API is public, keyless, and free — and also happy to silently hand you its entire 2.9-million-project history if you forget to filter, or to cap you at roughly 15,000 records deep on a broad search regardless of how many actually match. The NIH RePORTER Grants Scraper requires a real filter before it starts, pages within the platform's own offset ceiling, and flattens every project — title, abstract, award amount, institution, PI, agency — into one clean row at $6.05 per 1,000 grants.
The traps in an API that looks trivially open 🧬
No filter means no limit — and that's the expensive part. NIH RePORTER doesn't reject an unfiltered /v2/projects/search call. It just returns matches against its full corpus, which currently sits at 2.9M+ funded projects. A script that pages that "for fun" will run for a very long time and, on a pay-per-result Actor, would bill for every row it emits. This Actor validates the input before a single request goes out: at least one of fiscal year, text, institution, PI, or agency is required, or the run fails immediately with no charge.
The offset ceiling doesn't scale with your query. Confirmed live against the API: offset > 14,999 returns HTTP 400 — "System doesn't support offset value greater than 14,999" — no matter how many projects actually match your criteria. A search for a broad term like "cancer" can report tens of thousands of hits and still only let you page ~15,000 of them. Treating that 400 as a fatal error would kill the run and lose whatever it already scraped; instead it's read as a clean stop signal, and the run reports how many rows it got plus a note to narrow your filters (add a fiscal year, tighten the text search) if you need to go deeper. Page size has its own hard ceiling too — 500 per request; ask for 501 and NIH hands back a 400 before you see a single result.
The nested shapes aren't optional parsing — they're where the useful data lives. principal_investigators is a list of objects, each with its own is_contact_pi flag, not a flat name string — a project can (and often does) carry several PIs, and only one is the contact. Indexed research terms come back as a single bracket-packed string, <Term A><Term B><Term C>, which is useless for filtering or grouping until it's split. Both get flattened into structured fields (investigators as an array, terms as a real JSON array) rather than left as strings you'd have to re-parse downstream.
The judgement call: we block the run you didn't mean to start 🚧
It would be easy to let maxResults be the only guardrail and leave filtering optional — more flexible input, fewer validation errors to explain. This Actor doesn't do that. If none of fiscalYears, textSearch, orgNames, piNames, or agencies is set, the run refuses to start.
The reasoning: NIH RePORTER's unfiltered corpus is large enough that an accidental blank-input run isn't a small mistake, it's a five- or six-figure row count on a metered Actor. A required filter costs a first-time user one extra field. Not having one costs them a surprise invoice. We picked the cheaper mistake to make possible.
What you get per row
Application ID, full and core project numbers, project title and abstract, fiscal year, total/direct/indirect award amounts, recipient institution (name, city, state, country), funding NIH institute/center (code and full name), the contact PI plus the full investigators list with each person's contact-PI flag, project start/end dates, active status, human-readable spending categories, the parsed terms array, and the NIH RePORTER project detail URL.
Filter by fiscal year, free text against title/abstract/terms, institution name, PI name, agency code, and a min/max award amount range.
What it costs
Pay-per-event: $0.05 per run plus $0.006 per grant row. A thousand grants comes to $6.05. No key to request, no subscription, and a run that hits the offset ceiling or an unparseable record doesn't silently overcharge you for it — you get a status line saying exactly how many rows landed and why it stopped.
Where it fits
Tracking who's funded in a therapeutic area before a partnership or licensing conversation, building a competitive feed on a rival institution's or PI's active grants, sizing a research niche's funding trend across fiscal years for a market thesis, or feeding fresh award data into a grant-writing or BD CRM instead of a stale manual export.
Top comments (0)