DEV Community

Cover image for KNF Scraper: 75K+ Polish Financial Entities
Peter
Peter

Posted on • Originally published at apify.com

KNF Scraper: 75K+ Polish Financial Entities

#webscraping #javascript #fintech #api

TL;DR

  • Poland's KNF (Komisja Nadzoru Finansowego) maintains registries of all regulated financial entities - banks, insurers, payment institutions, lending companies, brokers
  • There are 75,000+ entities across 3 registries with no official API
  • The web portals use undocumented JSON APIs behind jQuery DataTables - no authentication required
  • I built an Apify actor that queries all 3 registries and returns structured JSON for $0.008 per entity

Why KNF Has No Public API

If you work in fintech compliance, you regularly need to answer: "Is this company licensed by the Polish financial regulator?"

KNF - the Polish Financial Supervision Authority - is responsible for supervising banks, insurance companies, investment firms, payment institutions, and the capital market in Poland. It was established in 2006 by merging three separate regulators (banking, insurance, and securities) into a single authority. Today, it oversees the entire Polish financial sector.

Despite the scale of its regulatory role, KNF does not offer a public API. Instead, it maintains three separate web portals, each covering a different segment of the financial market:

  1. e-RUP (e-rup.knf.gov.pl) - ~17,000 records: payment institutions, e-money institutions, account information providers
  2. RPKIP (rpkip.knf.gov.pl) - ~58,000 records: insurance agents, brokers, reinsurance intermediaries
  3. RDL (rdl.knf.gov.pl) - ~250 records: lending institutions (lombards, consumer lenders)

Each portal has a search form. One query at a time. No bulk export. No API documentation. If you need to verify 500 counterparties for a compliance review, you are looking at days of manual copy-paste work.

KNF Registry Data: How the Hidden API Works

All three portals use the same frontend pattern: a w2ui JavaScript grid library backed by a JSON API. The grid makes POST requests to load data - and these endpoints work perfectly without the browser.

The request format is a URL-encoded JSON body using w2ui's search protocol:

{
  "cmd": "get",
  "limit": 500,
  "offset": 0,
  "search": [
    { "field": "name", "type": "text", "operator": "contains", "value": "mBank" }
  ]
}
Enter fullscreen mode Exit fullscreen mode

No authentication. No cookies. No CSRF tokens. Just POST with the right body.

Each of the three registries has its own base URL, but the request structure is identical across all of them. The API returns clean JSON with all entity data - registration numbers, NIP (tax ID), entity types, addresses, status, and parent entities.

The pagination works via standard offset and limit parameters. The maximum limit accepted is 500 per request. For full database exports, the scraper iterates through all pages automatically, handling rate limits and retries.

One important detail: the RPKIP registry (insurance intermediaries) is the largest at ~58,000 records. A full export takes several minutes because the government servers throttle bulk requests. The actor handles this gracefully with exponential backoff.

KNF Registry Scraper: How to Use the Actor

JavaScript (Node.js) 

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_API_TOKEN' });

const run = await client.actor('minute_contest/knf-registry-scraper').call({
    registry: 'all',        // 'e-rup', 'rpkip', 'rdl', or 'all'
    name: 'mBank',          // search by name
    // nip: '5260200883',   // or search by NIP
    maxResults: 100
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();

for (const entity of items) {
    console.log(`${entity.name} | ${entity.entityTypeLabel} | ${entity.registry}`);
    console.log(`  NIP: ${entity.nip} | Status: ${entity.status}`);
}
Enter fullscreen mode Exit fullscreen mode

Python 

from apify_client import ApifyClient

client = ApifyClient("YOUR_API_TOKEN")

run = client.actor("minute_contest/knf-registry-scraper").call(
    run_input={
        "registry": "all",
        "name": "mBank",
        "maxResults": 100
    }
)

items = client.dataset(run["defaultDatasetId"]).list_items().items
for entity in items:
    print(f"{entity['name']} | {entity.get('entityTypeLabel')} | {entity['registry']}")
Enter fullscreen mode Exit fullscreen mode

Bulk Export All 75,000+ Entities 

const run = await client.actor('minute_contest/knf-registry-scraper').call({
    registry: 'all',
    exportAll: true
});
// Returns the entire registry database as JSON
Enter fullscreen mode Exit fullscreen mode

KNF Scraper Output: What You Get Back

Field Example
name mBank S.A.
registryNumber IP30/2013
nip 5260200883
entityType PSD_PI
entityTypeLabel Instytucja platnicza
status Aktywny
address ul. Prosta 18, Warszawa
registrationDate 2013-12-17
parentEntity (for agents: parent institution)
registry e-rup

Real-World Use Case: Fintech Compliance Screening

Consider a fintech company onboarding payment partners across Central Europe. Before integrating with a Polish payment institution, the compliance team must confirm that the entity holds a valid KNF license.

Without automation, an analyst opens e-rup.knf.gov.pl, types the company name, checks the status, copies the registration number into a spreadsheet, and repeats for the next company. At 2-3 minutes per lookup, screening 200 potential partners takes an entire workday.

With the KNF registry scraper, the same team passes a list of company names or NIP numbers to the actor. In a single API call with exportAll: true, they receive a complete dataset of all licensed payment institutions. A simple filter against their partner list produces a verified compliance report in minutes - not hours.

This approach also supports ongoing monitoring. By running the actor weekly and comparing results against the previous export, you can detect license revocations, new entrants, or status changes automatically.

Who Needs KNF Registry Data

  • Fintech compliance - verify if a partner/competitor holds a valid KNF license
  • AML/KYC teams - confirm that a financial counterparty is properly regulated
  • Insurance brokers - check agent registration status before partnerships
  • Lending platforms - verify that lending institutions are registered in RDL
  • Regulatory reporting - bulk-export supervised entities for regulatory analysis
  • Market research - analyze the Polish financial services landscape

FAQ

Can I search by NIP (tax ID) instead of company name?

Yes. The actor accepts both name and nip as input parameters. NIP search is exact-match and returns results faster since it queries a unique identifier. This is the preferred method when you already know the entity's tax ID.

How often is the KNF registry data updated?

The KNF portals are updated by the regulator as licensing decisions are made - typically within a few business days of a decision. There is no fixed schedule. For compliance purposes, running the scraper weekly or before each onboarding decision gives you near-real-time data.

What is the difference between the three registries (e-RUP, RPKIP, RDL)?

e-RUP covers payment services (payment institutions, e-money, account information services). RPKIP covers insurance distribution (agents, brokers, reinsurance intermediaries) - this is the largest registry at ~58,000 records. RDL covers lending institutions (consumer lenders, lombards) and is the smallest at ~250 records. Use registry: 'all' to search across all three at once.

Pricing

Method Cost
Manual search on KNF portals Free (one at a time)
This actor ~$3 per 1,000 entities
Full database export (75K entities) ~$230

Free $5 Apify credits on signup = ~625 entity lookups at no cost.

Try it: apify.com/minute_contest/knf-registry-scraper

This is part of the Polish Business Data APIs series covering programmatic access to Polish government registries.

Top comments (0)