DEV Community

Cover image for Filtering by regulatedOnly Drops Non-Regulated Entities from SRA Runs
Crawler Bros
Crawler Bros

Posted on Fully Autonomous

Filtering by regulatedOnly Drops Non-Regulated Entities from SRA Runs

When building compliance pipelines, Know Your Customer (KYC) workflows, or legal directories in England and Wales, cross-referencing law firms and practitioners against an authoritative record is non-negotiable. The Solicitors Regulation Authority (SRA) maintains the official public Solicitors Register, which tracks every regulated firm and practicing solicitor.

However, running raw keyword queries against the register frequently returns false positives for legal compliance workflows. Many corporate entities appear in the SRA register simply because they house an internal legal team or employ an SRA-regulated individual, despite the entity itself not being an SRA-regulated law firm. Parsing these mixed search results downstream requires unnecessary validation logic.

The SRA Solicitors Register Scraper addresses this by letting data engineers query the SRA database directly, separate person and firm entities, and pull structured organization profiles via an automated interface.

Structuring Firm vs Person Records

The public register holds two primary classes of entities: individuals (person) and legal organizations (firm). When ingesting this data, standard search queries can return both simultaneously, each containing distinct schemas.

A search result for a firm provides core metadata:

  • sraNumber: Unique identifier assigned by the regulator.
  • name: Official registered name of the firm.
  • alsoKnownAs: Known trading names.
  • status: Regulatory standing (e.g., SRA-regulated firm, Firm has closed).
  • headOfficeLocation: City or town of the main branch, often including branch counts.
  • profileUrl: Canonical link to the firm's register page.

Individual practitioner records omit office counts and trading names, returning:

  • sraNumber: The solicitor's individual ID.
  • name: Full registered name.
  • status: Individual status (e.g., SRA-regulated solicitor, SRA-regulated solicitor, not practising).
  • worksAt: Current firm affiliation.

By applying the regulatedOnly: true parameter, the actor discards non-regulated corporate entities (such as commercial organizations with in-house counsel) at extraction time. Note that regulatedOnly applies strictly to firm records; individual solicitor listings are not filtered out by this flag.

{
  "mode": "search",
  "searchQuery": "Smith",
  "searchType": "person",
  "maxItems": 50
}
Enter fullscreen mode Exit fullscreen mode

If you only require authorized law practices, passing searchType: "firm" alongside regulatedOnly: true guarantees that every emitted item possesses a valid status indicating active firm regulation.

Fetching Full Practice Profiles with firmDetails

Basic search records do not include practice areas, branch addresses, or disciplinary summaries. To extract this data, the actor provides a dedicated firmDetails mode that accepts an array of SRA numbers (sraNumbers).

This mode outputs deeper operational fields for each requested firm:

  • tradingNames[] and previousNames[]: Historical and brand aliases.
  • website: Primary web domain.
  • typeOfFirm: Legal classification (e.g., Recognised body since 16/04/2015, authorised for all legal services).
  • regulator: The oversight body.
  • regulatoryRecord: Published regulatory or disciplinary text on file.
  • offices[]: Array of branch objects containing officeName, address, phone, website, and email.
  • areasOfLaw[]: Categories of legal practice self-reported by the practice.
  • reservedActivities[]: Authorized reserved legal activities.
  • regulatedPeople[]: Up to 25 associated SRA-regulated individuals (name, status, worksAt).
  • regulatedPeopleCount: Total count of regulated professionals on record.

The scraper also enforces schema hygiene: empty properties are omitted from emitted JSON records rather than populated with null or empty strings.

Running an SRA Scraping Workflow

You can run queries and extract full firm records using the standard client runtime.

1. Define the Run Parameters

Configure the input object depending on whether you are running discovery or direct profile extraction.

{
  "mode": "firmDetails",
  "sraNumbers": ["620674", "570654"]
}
Enter fullscreen mode Exit fullscreen mode

2. Execute via API

Using Python, dispatch the job and fetch the default dataset:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run_input = {
    "mode": "search",
    "searchQuery": "Irwin Mitchell",
    "searchType": "firm",
    "regulatedOnly": True,
    "maxItems": 20
}

run = client.actor("crawlerbros/sra-solicitors-register-scraper").call(run_input=run_input)
dataset_items = client.dataset(run["defaultDatasetId"]).list_items().items

for item in dataset_items:
    print(f"Firm: {item.get('name')} - SRA: {item.get('sraNumber')}")
Enter fullscreen mode Exit fullscreen mode

3. Pipeline Ingestion

Because output records are cleanly differentiated by recordType (firm or person), downstream loaders can route entities to distinct database tables without complex regex matching.

Pricing and Execution Economics

The actor operates on a pay-per-event pricing model. There are no compute-time or platform-usage fees for the run; charges correspond strictly to start events and output rows:

  1. Actor Start (apify-actor-start): $0.005 per GB of memory allocated to the run, charged once upon startup.
  2. Dataset Result (apify-default-dataset-item): $0.005 per single result in the default dataset.

Volume discounts apply to the per-result event depending on tier:

  • FREE: $0.005 per event
  • BRONZE: $0.00433 per event
  • SILVER: $0.00367 per event
  • GOLD / PLATINUM / DIAMOND: $0.003 per event

A targeted batch extracting 200 firm profiles on a 1 GB run costs $0.005 (start) plus $1.00 (200 records at $0.005), totaling $1.005 on the base tier.

Acknowledged Boundaries and Limitations

This scraper does not retrieve individual solicitor profile subpages; individual records are populated strictly from search results and firm-level regulatedPeople lists because the SRA's individual solicitor detail pages use bot-mitigation challenges that cannot be accessed reliably through automated runs. Additionally, the actor enforces a maximum maxItems cap of 500 records per run, while the public register UI caps broad queries at roughly 1,000 items. To collect comprehensive regional data, partition runs into targeted searches by specific surnames, location keywords, or known SRA numbers.


The Actor used throughout this walkthrough is SRA Solicitors Register Scraper. Its README documents the full input schema, including the fields not covered here.

Top comments (0)