DEV Community

Daniel Ainsworth
Daniel Ainsworth

Posted on

How I built an FCA Register scraper on Apify (and why it's the B2B data gap nobody talks about)

#fca #compliance #kyc #uk

When most people think about UK data sources to scrape, they go straight to Companies House. And they should — it's an excellent dataset. But there's a more valuable register that compliance teams, fintech sales teams, and KYC workflows desperately need, and it has zero scrapers on the Apify marketplace: the FCA Financial Services Register.

The Financial Conduct Authority maintains a live register of every firm authorised to provide financial services in the UK — ~50,000 firms ranging from Barclays Bank to a one-person IFA. It includes their regulated permissions (what they're actually allowed to do), their principal address, trading names, and enforcement history. It's the data source used to verify "is this firm actually regulated?" — a check every fintech, insurance company, and compliance team runs regularly.

The opportunity

The FCA register has a free official REST API at register.fca.org.uk/Developer. No paid tier, no scraping needed — just register an account, get a key, and start making authenticated requests. Despite this, there was literally no Apify actor for it. The US equivalent — FINRA BrokerCheck — already has two actors with thousands of users. The UK gap was sitting there, unoccupied.

The architecture

This is one of the simplest actor architectures I've built — no Playwright, no Cheerio, no browser automation. Pure HTTP requests with fetch.

The FCA API has a few key endpoints:

  • GET /V0.1/Search?q={query}&type=firm — search for firms by name or keyword, returns FRNs
  • GET /V0.1/Firm/{FRN} — fetch base details for a specific firm
  • GET /V0.1/Firm/{FRN}/Address — registered address + phone + website
  • GET /V0.1/Firm/{FRN}/Names — trading names and historical names
  • GET /V0.1/Firm/{FRN}/Permissions — the full list of FCA-regulated activities

All endpoints require X-Auth-Email and X-Auth-Key headers. The rate limit is ~100 requests per 60 seconds, which works out to about 25 fully-enriched firms per minute at 4 API calls each.

async function fcaFetch<T>(path: string, email: string, apiKey: string): Promise<FcaApiResponse<T>> {
  await rateLimit(); // 700ms minimum gap between calls
  const res = await fetch(`${BASE_URL}${path}`, {
    headers: {
      'X-Auth-Email': email,
      'X-Auth-Key': apiKey,
    },
  });
  // ... error handling, retry logic
  return res.json() as FcaApiResponse<T>;
}
Enter fullscreen mode Exit fullscreen mode

The main challenge was normalising the permissions endpoint — the API returns an object where each key is the permission name (e.g. "Accepting deposits", "Dealing in investments as principal"), with values being arrays of limitations. Not a standard format, but easy to handle once you know to expect it.

Two modes

searchFirms: Give it a query like "payment institution" or "consumer credit", get back a paginated list of matching firms — each enriched with the full profile. Good for building prospect lists.

lookupFirms: Provide a list of FRNs directly. Good for KYC workflows where you already have a firm's reference number and just need to verify their status and permissions.

The result

Output looks like this for each firm:

{
  "frn": "730427",
  "organisationName": "Monzo Bank Limited",
  "status": "Authorised",
  "businessType": "UK Authorised Bank",
  "companiesHouseNumber": "09446231",
  "tradingNames": ["Monzo"],
  "address": {
    "town": "London",
    "postcode": "EC2A 2DA",
    "website": "https://monzo.com",
    "phone": "0800 802 1281"
  },
  "permissions": ["Accepting deposits", "Dealing in investments as agent", "Issuing electronic money"],
  "registerUrl": "https://register.fca.org.uk/s/firm?id=730427"
}
Enter fullscreen mode Exit fullscreen mode

Available on Apify

The actor is live on the Apify Store: apify.com/danielainsworth/fca-register

PPE pricing at $0.10/firm — a 100-firm compliance check costs $11 total. The API credentials are free to get, and the data is completely legal to access (it's a statutory public register designed for programmatic access).

If you're building KYC automation, a fintech CRM, or sales tooling for the regulated sector, this fills a genuine gap in the available UK data.

Top comments (0)