DEV Community: Juan de Pablos

US Federal Contract Opportunities as Clean JSON (SAM.gov Without the Pain)

Juan de Pablos — Mon, 13 Jul 2026 08:00:18 +0000

Every business day the US federal government publishes hundreds of new contract opportunities on SAM.gov. It is the single largest buyer in the world, and unlike most B2B pipelines, everything is public: what they want to buy, when responses are due, who the contracting officer is, and whether the contract is reserved for small businesses.

The problem is not access. The problem is that actually working with SAM.gov means refreshing a slow portal, fighting saved-search emails that arrive late, or wrestling an official API that returns deeply nested records, cryptic set-aside codes and HTML-polluted descriptions.

I got tired of it and built a feed that does one thing: give me the opportunities matching my filters as clean JSON, with the fields I actually need for a go/no-go decision. This post shows what it returns and how to use it.

What a useful opportunity record looks like

Here is a real record from a cybersecurity keyword scan (trimmed for length):

{
  "title": "HHS/OCI West Publishing Clear Software or EQUAL",
  "solicitationNumber": "7571TEHHSOCIWPCS",
  "noticeType": "Solicitation",
  "publishDate": "2026-07-02T15:19:03.194+00:00",
  "responseDeadline": "2026-07-06T16:30:00-04:00",
  "daysUntilDeadline": 1,
  "naicsCodes": ["541511"],
  "setAside": "NONE",
  "placeOfPerformance": "Silver Spring, MD, USA",
  "contactName": "Raja Seshadri",
  "contactEmail": "raja.seshadri@hhs.gov",
  "url": "https://sam.gov/opp/a57a5914739646ccbd70cbbecac11b5b/view"
}

Three details do most of the work here:

daysUntilDeadline is computed against now. SAM.gov gives you a timestamp; what you actually want to know is "is this alive, and how urgent is it". A record with daysUntilDeadline: 1 goes to the top of today's list. Anything negative never reaches you.

setAside is a label, not a code. The raw API speaks in codes like SBA, 8A, WOSBC, SDVOSBC. The feed maps them to what they mean, so you can filter for "total small business set-aside" or "service-disabled veteran-owned" without a lookup table. If you qualify for a set-aside category, these are the opportunities where your win rate is structurally higher, because most of the market is excluded by law.

The contracting officer is right there. Name and email, from the official record. For subcontracting or capability questions, that contact is the difference between a bid and a shot in the dark.

Running a scan

The feed runs as an Apify Actor, so you can call it from anywhere that can make an HTTP request:

curl -X POST "https://api.apify.com/v2/acts/jdepablos~samgov-opportunities-feed/run-sync-get-dataset-items?timeout=300" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "keywords": "cybersecurity",
    "dueWithinDays": 30,
    "setAsides": ["SBA"],
    "maxOpportunities": 25
  }'

You can filter by keywords, NAICS codes, notice types (solicitations vs sources-sought vs award notices), deadline window and set-aside categories. Descriptions come back with the HTML stripped.

Schedule that daily and you have a private deal-flow feed for exactly your niche, hitting your inbox or your CRM before you would ever find the opportunity by browsing.

What it costs

Pay per event: $0.004 per opportunity delivered. A daily scan returning 25 matches costs 10 cents. There is no subscription, and runs that return nothing cost you nothing beyond Apify's tiny platform fee. Compare that with the enterprise gov-con intelligence suites, which start in the hundreds per month and mostly resell the same public data.

Who this is for

Small businesses with set-aside eligibility. Filter for your categories and stop competing with everyone.
Subcontractors. Sources-sought notices tell you which primes are hunting for partners, weeks before an award.
Anyone building agents. The Actor is exposed as an MCP tool, so an AI agent can query live federal opportunities mid-conversation. Ask it "find cybersecurity contracts closing this month with small-business set-asides" and it has the answer in one call.

The Actor is here: US Contract Opportunities Feed on Apify. Runnable examples for this and 16 other data tools live in my Data Actors Cookbook.

If you work gov-con and there is a field or filter you are missing, tell me in the comments. The last two features I shipped on another feed came straight from a reader request.

Scraping Booking.com Without Touching the Search Box (City Landings + the Apollo Cache)

Juan de Pablos — Sat, 11 Jul 2026 08:00:14 +0000

Booking.com does not want bots, and it shows. Every plain HTTP request, any client, any proxy, gets absorbed by AWS WAF with a 202 that never resolves into content. There is no getting around a real browser here, which makes this the most expensive scrape in my portfolio. What kept the project alive was discovering that the two hard parts, search and data extraction, both have quiet side doors.

Don't touch the search box

My first version drove the UI like a user: open the homepage, type the city, click search. It was a disaster. The autocomplete is timing-sensitive, the results panel loads lazily, and about a third of runs ended with zero hotels for no visible reason. Debugging flaky UI automation is the worst kind of debugging because every failure looks different.

The side door: Booking maintains SEO landing pages per city with stable URLs:

https://www.booking.com/city/{countryCode}/{citySlug}.{lang}.html

Navigate straight there and the page arrives populated with a hotel list. No typing, no clicking, no autocomplete race. Runs went from flaky to boring, which is the highest compliment a scraper can receive.

To build the URL from a free-text city name I geocode with Open-Meteo's public API (no key): city name in, country code and a sluggable name out.

The data is not in the DOM

Second surprise: scraping the rendered page gets you almost nothing coherent. Prices and hotel names live in visually adjacent but structurally unrelated containers, and the class names are minified soup.

The actual data source is Booking's Apollo GraphQL cache, embedded in a <script type="application/json"> block with a ROOT_QUERY key. Every hotel on the page is there as a typed object: name, review score, coordinates, and the price in three formats.

About those formats: roundedValue and value are display strings like "€ 128". The numeric price is at minPrice.AMOUNT. If you parse the display strings you inherit every currency symbol and thousands separator problem for free. Take the number the frontend itself uses.

Know what price you are looking at

One honesty note that shapes the whole product: city landing prices are indicative nightly rates, not date-specific quotes. Booking shows "from X per night" without you having picked dates. For market-level questions (median price level of a city, how hotel A sits against the market) that is exactly what you want. For "what does a room cost on August 14" you need a second navigation with dates, which is a different, more expensive product.

My feed stamps priceBasis: "indicative-nightly" on every response so nobody mistakes one for the other. Label your price basis; blurred numbers look precise and mean nothing.

Try it

The result is booking-hotel-price-analyzer on Apify. One call gives you a city's hotel market: median and percentile prices, supply depth, and the hotel list behind the numbers.

curl -s -X POST \
  "https://api.apify.com/v2/acts/jdepablos~booking-hotel-price-analyzer/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"location": "Madrid"}'

Browser-based, so it costs more per scan than my HTTP actors, and it is priced accordingly. A scan that returns no hotels is never charged.

MCP config for agents:

https://mcp.apify.com?tools=jdepablos/booking-hotel-price-analyzer

Takeaways

When a site fights HTTP clients, the browser is the entry fee, but you still choose how much of the UI you depend on. Direct navigation to stable URLs beats interactive flows every time you can find them. And before you fight the DOM, check for an embedded state blob: Apollo caches, __NEXT_DATA__, Redux snapshots. Frontends increasingly ship their data model in one script tag, cleaner than any HTML you will ever parse.

Hotel folks: if you need date-specific quotes or a market I have not covered, tell me in the comments and it goes on the roadmap.

Tracking New Patent Filings Without an API Key (Google Patents Has a Hidden XHR Endpoint)

Juan de Pablos — Thu, 09 Jul 2026 08:00:17 +0000

I wanted a simple thing: a weekly feed of new patent filings matching a keyword, so I could watch what competitors in a given technology are up to. The official routes all said no. The USPTO Open Data Portal returned 401 without an API key. The bulk data endpoint answered 503 for days. PatentsView had just moved to key-only access with an approval queue.

Then I opened DevTools on Google Patents and found that the whole site runs on one undocumented XHR endpoint. No key, no login, worldwide coverage.

The endpoint

When you search on patents.google.com, the frontend calls:

https://patents.google.com/xhr/query?url=q%3D(solid%2Bstate%2Bbattery)%26sort%3Dnew

The url parameter is itself a URL-encoded query string. Decode it and you get the same syntax the search box uses: q=(solid+state+battery)&sort=new, plus operators like assignee=Nintendo or country=US. The response is JSON with results.cluster[0].result[], each entry carrying the publication number, title, snippet, assignee, inventor and dates.

It works with plain HTTP requests. I have been running it on a schedule for a couple of days now without drama, which is more than I can say for my week with the official APIs.

Three traps I hit so you don't have to

1. Server-side date filters return zero. The query syntax accepts after=publication:20260101, and on the website it works. Through the XHR endpoint the same filter quietly returns an empty result set. Not an error, just nothing. The fix is to drop the filter, fetch pages sorted by newest, and filter publication_date client-side. Annoying, but reliable.

2. sort=new is not actually sorted. I assumed I could stop paginating once I saw a date older than my window. Wrong: results within sort=new are shuffled inside a rough two-month band, so an old date does not mean the rest of the page is old too. If you early-exit on the first stale date, you silently lose fresh publications. You have to walk the whole window and let the client-side filter do its job.

3. The index lags reality by about two months. A patent published last week will usually not be in Google Patents yet. If you set your watch window to 30 days, most weeks return nothing and you conclude the tool is broken. Ninety days is the sweet spot: you see everything, slightly deferred. For patents that trade-off is usually fine, since the filings themselves are 18 months old by the time they publish.

There is also the ordinary anti-abuse stuff: burst requests from datacenter IPs get 503s, so pace your calls and use decent proxies. And the JSON fields arrive with <b> highlight tags around your search terms, which you need to strip before the data is clean.

Try it

I packaged all of this as an Apify Actor, patent-watch. One call:

curl -s -X POST \
  "https://api.apify.com/v2/acts/jdepablos~patent-watch/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"term": "solid state battery", "daysBack": 90}'

You can watch a technology (term), a company (assignee: "Nintendo"), or both at once, up to 20 watches per run. Pricing is per watch that returns hits; a quiet week costs nothing.

Wiring it into an AI agent (MCP)

One line of config through Apify's MCP server:

https://mcp.apify.com?tools=jdepablos/patent-watch

Your agent gets a "what's new in patents for X?" tool with structured output. Put it on a weekly schedule and the agent writes your competitive R&D brief for you.

What I took away

Official APIs are the right first choice, and I always probe them before touching anything else. But when the official route is keyed, queued or down, the data-bearing endpoint behind a public website is often better documented by its own network tab than the sanctioned API is by its docs. Just treat it with respect: pace your requests, filter client-side when the server lies to you, and never trust a sort order you have not verified.

If you watch patents for competitive intelligence and there is a field or filter you miss, tell me in the comments. The last feature request I got on one of these feeds turned into a shipped field plus a bug fix within a day, so they do not land in a void.

Catawiki Publishes Its Expert Estimates in the Page JSON. I Built a Deal Finder on Top

Juan de Pablos — Wed, 08 Jul 2026 08:00:20 +0000

Catawiki is an auction site where every lot gets reviewed by a category expert who writes an estimated value range. That estimate is the single most useful number on the page, and it turns out Catawiki ships it to your browser in structured JSON, for every lot, before you even bid.

Compare the current bid against the expert estimate across a whole category and you have a shopping list of undervalued lots. I built a bot that does exactly that. Here is what I learned about the site on the way.

The data is in NEXT_DATA

Catawiki is a Next.js app, so each page embeds its state in a <script id="__NEXT_DATA__"> block. No hidden API, no reverse engineering, just parse the JSON out of the HTML.

Two levels matter:

The search page carries pageProps.searchLots.lots, 24 lots per page. Handy for discovery, but there is a catch: search results do not include prices or estimates. Just IDs, titles and images.

The lot page is where everything lives: current bid, the expert's low and high estimate, whether the reserve price has been met, bid history, closing time. One fetch per lot, so a category scan is one search request plus N lot requests. I run them at concurrency 4 and the site has never complained.

The pivot I didn't plan

My original idea was to analyze completed auctions and build price history. Dead end: Catawiki's public search only exposes live lots. Once a lot closes it disappears from search, and there is no public archive to walk.

So the product pivoted from "price history" to "live deal finder": scan open lots, compute how far the current bid sits below the expert estimate, and surface the gap. The site constraint decided the product, which happens more often than product plans admit.

Fresh auctions look absurd, and that's fine

First real run: Game Boy lots showing bids 85 to 99 percent below the expert estimate. Too good to be true? Sort of. Auctions that just opened have starting bids of one euro against estimates of hundreds. The gap is real but everyone bidding knows it will close.

The actionable filter is time: a lot ending within 24 hours with the bid still 40 percent under estimate is a genuinely interesting signal. A lot with six days left is noise. So the bot takes endingWithinHours as its main input, and that single filter separates a gimmick from a tool.

One practical note: fetching through catawiki.com/es/ pins all amounts to EUR, which saves you currency detection entirely.

Try it

Packaged as an Apify Actor, catawiki-deal-finder:

curl -s -X POST \
  "https://api.apify.com/v2/acts/jdepablos~catawiki-deal-finder/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query": "game boy", "endingWithinHours": 24, "minDiscountPct": 30}'

You get each lot with its current bid, the expert estimate range, the discount percentage, reserve status and closing time. Pricing is per auction scan, and a scan that returns nothing is never charged.

For AI agents there is an MCP route too:

https://mcp.apify.com?tools=jdepablos/catawiki-deal-finder

"Any watches closing today at half the expert estimate?" becomes a tool call your agent can run every morning.

Takeaways

Next.js sites are gifts: the state you want is usually sitting in __NEXT_DATA__, typed and complete, no DOM parsing needed. But check what the public surface actually exposes before you promise a product. I planned an archive analysis and the archive did not exist. The expert estimate data was better anyway: it is an opinion you can trade against, published in JSON.

If you hunt auctions and want a filter or a category I have not covered, comments are open.

One API Call to Price Any Luxury Watch (Median, Percentiles, Market Depth)

Juan de Pablos — Sun, 05 Jul 2026 23:13:12 +0000

Every watch-price tool I found gives you the same thing: a pile of raw listings. Scrape Chrono24, get 60 JSON objects, and now you write the median logic, you filter out the "strap only" listings poisoning your stats, you figure out that the first cards on the page are promoted ads for a different model entirely.

I wanted the opposite: one call in, one market analysis out. So I built it.

What "analyzed" means

Ask for "Rolex Submariner 16610" and you get back a single structured answer:

{
  "analysis": {
    "n": 46,
    "currency": "EUR",
    "median": 9150,
    "p10": 8000, "p25": 8690, "p75": 11400, "p90": 13465,
    "priceBasis": "asking",
    "marketDepth": 1181,
    "certifiedSharePct": 21.7,
    "dealerCountryMix": [
      { "country": "IT", "count": 10 },
      { "country": "ES", "count": 7 }
    ],
    "confidence": "high"
  }
}

Plus the cleaned comps behind every number, so you can audit the math.

The cleaning is where the real work happens. Before any statistics run, the pipeline drops:

Promoted cards. On chrono24.com they can be up to two thirds of the page, and they are often a different model than what you searched.
Accessories. A "strap for Submariner" contains every token of your query and costs 95% less. Keyword matching fails here: in watch land, "Blue Dial" and "Box & Papers" are descriptors of a complete watch, not accessory listings. The filter is positional instead: a title that starts with "strap", or matches "bezel for ...", is an accessory; a title that merely mentions a bezel is not.
Parts, defective, replica listings and duplicates.

Skip the cleaning and your median moves by hundreds of euros. I measured it.

Honest data: asks, not sales

Every price here is a live asking price (priceBasis: "asking" is stamped on every response). Asking prices tell you where the market is priced today: what you'd pay right now, how deep supply is (marketDepth), how spread the market is (p10 to p90).

What they don't tell you is what buyers actually pay. For that you want confirmed sales, like eBay sold listings. Comparing the two gives you the ask-vs-sold gap, which is where negotiation lives. (I run a sold-side appraiser for exactly that; point both at the same model and you have both sides of the market.)

Tools that blur this distinction produce numbers that look precise and mean nothing. Label your price basis.

Try it

The analyzer runs on Apify as chrono24-watch-analyzer. One HTTP call:

curl -s -X POST \
  "https://api.apify.com/v2/acts/jdepablos~chrono24-watch-analyzer/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query": "Omega Speedmaster Professional", "domain": "chrono24.com"}'

Batch mode takes up to 25 models per run. Six Chrono24 domains are supported (.com/.es/.de/.fr/.it/.co.uk), which sets the currency.

Pricing is per analysis ($0.02, comps included), and an analysis that returns no data is never charged. That last part took real plumbing: the actor distinguishes "this query genuinely has zero listings" from "the site soft-blocked this request", retries the latter with a different proxy, and only bills when you get numbers.

Wiring it into an AI agent (MCP)

If you're building agents, this is one line of config via Apify's MCP server:

https://mcp.apify.com?tools=jdepablos/chrono24-watch-analyzer

Your agent gets a chrono24-watch-analyzer tool that answers "what's this watch worth on the open market?" with structured, self-describing data. No pagination, no post-processing, no HTML. The output schema ships with the actor, so the agent knows what marketDepth and certifiedSharePct mean without you writing glue.

Agent use cases I'm already seeing work:

Deal triage: agent receives a watch listing (marketplace, auction, classifieds), pulls the ask distribution, flags anything under p25 for human review.
Collection valuation: batch-run 25 references, sum medians, track weekly.
Negotiation prep: median and p10 in hand before you make an offer.

What I learned building it

Two bugs worth sharing so you don't repeat them:

Regex money parsing: my price regex \d{1,3}(?:[.,\s]\d{3})* silently truncated "5590 €" to "559". The first alternative matched a prefix and stopped. Chrono24 omits decimals and thousands separators under 10k; eBay always emits decimals, so the same bug sat latent in another parser for weeks. Make the thousands group mandatory (+) and fall through to plain \d+.
Keyword-based accessory filters break on watches: "Box & Papers" is a good sign in a watch listing, not an accessory. Domain vocabulary matters; test your filters against real cards before trusting your medians.

Questions, feature requests (multi-page sampling and reference-number input are next), weird edge cases: comments are open.

I Built a Bot That Finds Underpriced Game Consoles on Vinted (Using Real eBay Sold Prices)

Juan de Pablos — Sun, 05 Jul 2026 08:00:34 +0000

Every reseller knows the feeling: you spot a Game Boy Micro for €100 on Vinted, you're pretty sure it flips for more on eBay, but by the time you've checked sold listings, filtered out the broken ones and done the fee math, someone else bought it.

The problem isn't finding listings — it's valuing them fast. So I automated the whole loop: scan live listings on Vinted, Wallapop and Milanuncios, value each item against confirmed eBay sold prices (not asking prices!), subtract every fee, and return only the listings with real profit in them. This tutorial shows you how to run it yourself in 5 minutes, across Spain, France, Germany, Italy or the UK.

Why most "deal finding" fails

Three traps kill naive flipping bots:

Asking prices lie. eBay is full of Game Boy Micros listed at €400 that never sell. The only truth is Sold listings — what buyers actually paid. Any valuation built on active listings is fantasy.
The junk poisons the math. "GBA cartridge lot", "Game Boy Micro stand", "for parts, not working" — if these enter your average, a €2.50 accessory looks like a 2,000% margin deal. (Ask me how I know.)
Fees eat the spread. Buyer protection (~5% + €0.70 on Vinted), shipping both ways, ~13% marketplace fees when you resell. A "€50 profit" is often €12 after honest math.

The Second-hand Deal Scanner EU Actor handles all three: sold-only valuations, aggressive junk/accessory/price-sanity filters (every rejection is counted so you can audit it), and a fee model you can override.

Run your first scan

Grab a free Apify API token, then:

const res = await fetch(
  'https://api.apify.com/v2/acts/jdepablos~second-hand-deal-scanner-es/run-sync-get-dataset-items?timeout=300',
  {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${process.env.APIFY_TOKEN}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      query: 'Game Boy Micro',
      country: 'ES',        // or FR, DE, IT, UK
      marginMinPct: 25,     // only show ≥25% estimated margin
    }),
  },
);
console.log(await res.json());

A real result from my test runs:

{
  "query": "Game Boy Micro",
  "valuation": { "median": 217.47, "n": 184, "confidence": "high", "currency": "EUR" },
  "deals": [
    {
      "source": "vinted",
      "title": "Game Boy micro en perfecto estado",
      "price": 100,
      "url": "https://www.vinted.es/items/...",
      "economics": {
        "estBuyTotal": 108.70,
        "estResaleNet": 162.92,
        "estProfit": 54.22,
        "estMarginPct": 49.65,
        "verdict": "deal"
      }
    }
  ],
  "meta": {
    "listingsScanned": 89,
    "filteredOut": { "titleMismatch": 21, "accessory": 3, "junk": 2, "suspiciouslyCheap": 5 }
  }
}

Read that valuation line: 184 confirmed sales, median €217.47, high confidence. The console at €100 has ~€54 of profit after buyer protection, shipping and resale fees. And look at filteredOut — 31 listings were rejected (wrong product, accessories, too-cheap-to-be-real). That's the boring machinery that makes the number trustworthy.

Build a daily watchlist

The real power move: a watchlist of items you know well, scanned every morning.

{
  "items": [
    { "query": "Game Boy Micro" },
    { "query": "PlayStation Vita OLED", "priceMax": 150 },
    { "query": "iPhone 13 128gb", "marginMinPct": 20 },
    { "query": "Lego 75192", "priceMax": 500 }
  ],
  "country": "ES"
}

Schedule it in the Apify Console (Actor → Schedule → daily 8:00), wire the webhook to Telegram or Slack, and you'll wake up to a ranked list of real opportunities. Each item scan costs $0.04 flat — a 10-item daily watchlist is ~$12/month, or roughly the profit of one decent flip.

Cross-border mode

Because valuation and scanning are decoupled, you can hunt arbitrage between markets: scan French Vinted but value against German eBay (country: "FR", marketplaceForComps: "ebay.de"). Retro games routinely price 15-30% apart between EU markets — that spread is the whole business model for some resellers.

Let your AI agent do the shopping research

The Actor doubles as an MCP tool:

{
  "mcpServers": {
    "deal-scanner": {
      "url": "https://mcp.apify.com?tools=jdepablos/second-hand-deal-scanner-es",
      "headers": { "Authorization": "Bearer <APIFY_TOKEN>" }
    }
  }
}

Then: "Is there any Game Boy Micro under €120 in Spain right now that flips for 30%+? Show the math." The agent runs the scan and reasons over economics — no parsing on your side.

Honest limitations

Estimates are estimates: condition, completeness (box/charger) and haggling move the final number. The valuation includes the comp URLs so you can verify before buying.
Wallapop and Milanuncios are Spain-only; outside Spain the scanner uses the local Vinted.
Zero deals on most days for most items is correct behavior — markets are mostly efficient, and the scanner is honest about it. The profit is in the exceptions, which is why you automate the watching.

Wrap-up

Stop valuing by gut feel. Try the Deal Scanner with a query you know well — the first scan costs 4 cents and will probably teach you something about your market. More copy-paste examples (including the standalone eBay Sold Price Appraiser that powers the valuations) live in the cookbook repo.

How to Track US Startup Funding Rounds in Real Time (Before TechCrunch Writes About Them)

Juan de Pablos — Fri, 03 Jul 2026 12:39:30 +0000

Every week, over 1,300 US companies file a funding round with the SEC — and most of them never appear in the tech press. If your job involves selling to funded startups, tracking competitors' war chests, or spotting investment trends, you're probably relying on funding newsletters and databases that are days late and hundreds of dollars per seat.

There's a better way: go to the primary source. In this tutorial you'll build a real-time startup funding feed from SEC Form D filings — the regulatory document every US company must file when it raises private capital. You'll get exact amounts, industries, locations and investor counts, as clean JSON, for a fraction of a cent per round.

Why Form D beats funding news

When a startup raises money under Regulation D (the exemption used by virtually all US venture rounds), it must file Form D with the SEC within 15 days of the first sale. That filing includes:

The exact amount sold so far — not a journalist's "sources say" estimate
The total offering size (or whether it's open-ended)
Industry group, city and state
Number of investors who participated
Date of first sale and year of incorporation

Compare that to funding news: TechCrunch covers a tiny, PR-driven slice. Databases like Crunchbase aggregate press and manual research — comprehensive over time, but late and expensive. Form D is the ground truth both of them chase.

The catch? EDGAR (the SEC's database) is built for lawyers, not for automation. The filings are XML documents scattered across an archive, discoverable only through a quirky full-text search API with hidden rate limits. That's the part we'll automate.

The 5-minute setup

We'll use the Startup Funding Feed Actor — it handles EDGAR's discovery API, XML parsing, rate limits and pagination, and returns one JSON record per filing. It's pay-per-event: $0.002 per filing returned (a full weekly sweep of all US rounds costs ~$2.60; a filtered slice costs cents). Failed fetches are never charged.

Create a free Apify account and grab your API token from Settings → API & Integrations.
That's it. No servers, no cron, no XML parsing.

Your first funding sweep

Let's get every round in California or New York from the last 7 days that already raised at least $1M:

const res = await fetch(
  'https://api.apify.com/v2/acts/jdepablos~startup-funding-feed/run-sync-get-dataset-items?timeout=300',
  {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${process.env.APIFY_TOKEN}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      daysBack: 7,
      states: ['CA', 'NY'],
      minAmountSold: 1000000,
      maxFilings: 100,
    }),
  },
);
const rounds = await res.json();

Each record looks like this:

{
  "company": "Acme Robotics, Inc.",
  "filedAt": "2026-07-02",
  "industry": "Other Technology",
  "city": "SAN FRANCISCO",
  "stateDescription": "CALIFORNIA",
  "totalOfferingUsd": 15000000,
  "totalSoldUsd": 12500000,
  "investorsCount": 14,
  "minInvestmentUsd": 25000,
  "yearOfIncorporation": "2024",
  "url": "https://www.sec.gov/Archives/edgar/data/..."
}

Every record links back to the official SEC document, so anything surprising is one click from verifiable.

Recipe 1: A weekly leads feed for your CRM

Companies that just raised have two things your sales team loves: budget and urgency. A weekly filtered sweep, piped into your CRM, is one of the highest-signal lead sources that exists:

// Monday morning: fintech + healthtech rounds over $2M, nationwide
const input = {
  daysBack: 7,
  industries: ['banking', 'insurance', 'health'],
  minAmountSold: 2000000,
  includeAmendments: false,   // only NEW rounds, not updates
};

Filter, dedupe against your CRM by company name, and enrich with your usual tools. The investorsCount and minInvestmentUsd fields are great qualifiers: 40 investors at $25k minimum is a party round; 2 investors at $2M is institutional conviction.

Recipe 2: Watch your competitive landscape

VCs and corp-dev teams: replace "check Crunchbase weekly" with a filtered feed of your thesis area. Because Form D arrives before or without press coverage, you'll regularly see rounds nobody has written about — which is exactly when a cold email is most welcome.

Recipe 3: Ask your AI agent instead

The Actor is exposed as an MCP tool, so agents like Claude can call it directly:

{
  "mcpServers": {
    "funding-feed": {
      "url": "https://mcp.apify.com?tools=jdepablos/startup-funding-feed",
      "headers": { "Authorization": "Bearer <APIFY_TOKEN>" }
    }
  }
}

Then just ask: "Which biotech companies in Massachusetts raised over $5M this month? Give me the SEC links." The agent picks the filters, runs the feed, and reads the structured output — no glue code.

Scheduling it (the set-and-forget part)

In the Apify Console, open the Actor → Schedule, pick weekly (Mondays 8:00 work well — Form D filings cluster early in the week), and connect the run to wherever you want the data: a webhook to your CRM, a Google Sheet via the integrations tab, or Slack.

Total cost of a serious weekly setup: filtered feeds usually return 20-80 filings → under $0.20 per week.

What you can't get from Form D (honesty section)

Valuations — Form D discloses amounts raised, not share prices.
Investor names — the filing lists officers/directors/promoters, not fund names. (The linked document sometimes reveals more.)
Non-US rounds — this is SEC data; it covers US offerings, including foreign issuers that file with the SEC.

For most sales, research and monitoring workflows, none of that matters: the who/where/how much/when is the signal.

Wrap-up

You now have a primary-source funding feed that runs itself: official data, exact amounts, days ahead of the press, at pay-per-use prices. Try the Startup Funding Feed on Apify — the first runs cost pennies, and the cookbook repo has copy-paste examples for this and seven other official-source data APIs (insider trading, trademarks, clinical trials and more).

Questions or feature requests? The Issues tab on the Actor page gets answered within a day.