Court records look like a scraping problem for about a week.
The federal docket system is reachable, the pages are simple, the structure is consistent enough to parse, and a competent engineer can pull dockets on day two. Every legal data project we have seen starts with that early win and the confidence it produces.
The confidence is misplaced, and not because access gets harder. Retrieval is the cheapest part of legal data work. What follows retrieval is a chain of problems that are genuinely difficult, largely undiscussed, and where every one of these projects actually spends its time.
The document is not the data
A docket is a list of events pointing at documents, and the useful information is almost entirely inside the documents rather than in the list.
That distinction creates four separate difficulties.
Docket text is written by humans under no schema. Entries are free text, conventions vary by district and by clerk, and the same event is described several ways across courts. Extracting event type, filing party and relief sought from that text is an information extraction task, not a parse.
Documents are scanned, often badly. A meaningful share of what you retrieve is an image of a page that was printed, signed, scanned and filed. Recognition handles most of it. What it does not handle is structure: exhibit boundaries, signature blocks, stamped modifications, handwritten annotations on a scanned order.
Structure spans documents. A motion, its exhibits, the opposition, the reply and the order are five documents that constitute one event, related by references in text rather than by any field. Reconstructing that chain is the difference between a document archive and a litigation dataset.
Sealed and redacted content is invisible but consequential. Portions of the record are not available, and a dataset that does not model absence will present an incomplete docket as a complete one. Modelling absence explicitly is one of the first things that separates a working legal document processing pipeline from a document dump.
None of these are retrieval problems. They are legal document processing problems, and the tooling for them is entirely different from the tooling for fetching pages.
Expert Insight: Sample fifty dockets and hand-label the event type on every entry before you build anything. Two numbers come out of it and both change the plan: how many entries are unambiguous from the text alone, and how many require opening the document. The second number is usually far higher than teams expect, and it is the number that determines whether your project is an extraction project or a document understanding project.
Entity resolution in a domain built to defeat it
The second wall is identity, and legal data is close to a worst case for it.
Party names arrive as free text entered by filers, which produces the full range: abbreviations, punctuation variants, trailing corporate suffixes, misspellings, DBA names, and the fact that the same corporate group litigates under a dozen subsidiary names that share no tokens with the parent.
Three specific traps sit on top of that.
- Corporate structure is the point. Parties are frequently subsidiaries chosen deliberately, and rolling them up to a parent requires an external corporate hierarchy that the court record does not contain and will not help you build.
- Law firms and attorneys are their own resolution problem. Attorneys move firms mid-case, appear under several bar registrations, and are listed inconsistently. Firm-level analytics depend on getting this right and most datasets quietly do not.
- Same-name individuals are common and unmarked. There is no identifier for a natural person in the record, and matching on name alone across jurisdictions produces confident nonsense.
The resolution layer, not the scraper, is what separates a usable legal dataset from a pile of dockets. It also has to be temporal: a subsidiary acquired mid-case belongs to one parent before the acquisition and another after, and a dataset that resolves to today's structure silently rewrites history. This is the layer that most differentiates legal data providers from each other, and it is invisible in any feature comparison.
Expert Insight: Resolve parties with validity dates from the first day rather than retrofitting them. Retrofitting temporal validity into a resolution layer that assumed the present tense is one of the more painful migrations we have watched teams undertake, because every derived analytic silently changes when it lands and nobody can say which version was right.
Coverage, currency and the honest scope conversation
The third wall is the one that reframes the project, and it is worth confronting during scoping rather than in month four.
The federal system is the easy tier. State courts are the majority of litigation and they are a different world: dozens of systems, each with its own interface, its own access rules, its own coverage gaps, some of them not electronic at all. A dataset that covers federal matters and describes itself as covering litigation is describing a slice.
Currency is a per-court property. Filing-to-availability lag varies by court and by document type. A pipeline that treats all courts as equally current produces analytics with a systematic bias toward the fast ones, which is invisible unless you measure it per court.
Access terms vary and change. Every court system has its own rules about automated access, and they are not uniform, not permanent, and not always where you would expect to find them. This is a scoping question with a legal answer, not an engineering one, and it should be settled before the crawler is written rather than after.
The practical consequence is that legal data is an operations commitment rather than an acquisition project. Coverage expands court by court, each with its own access pattern and its own document quirks, and currency has to be monitored per court forever.
At Forage AI the legal work we run is mostly the three layers above rather than the retrieval one: document understanding across scanned filings, temporal party resolution, and per-court coverage and currency monitoring. The retrieval is the part that gets demoed and the part nobody remembers building.
Before scoping one of these, hand-label fifty dockets and count how many courts you actually need. Those two numbers will tell you more about the timeline than any technical spike will.
About the author: This piece was written by the data team at Forage AI, which extracts and structures legal and regulatory documents at scale across 10M+ documents. Learn more about Forage AI's work in intelligent document processing at forage.ai.
Top comments (0)