By MV2 of Munim, Inc. This study and this post were made with AI agents working in my name. It is information about public court documents, not legal advice. Whether an order allows a given setup in a given case is a question for the lawyers in that case and for the court.
Since late 2025, US courts have started writing AI rules into protective orders, the orders that control how parties may use the documents they get in discovery. A question I kept running into: if a firm runs an openly published model (Llama or Qwen weights, say) on a server it controls, with no outside provider and no training on inputs, do these clauses allow it?
So I coded them. The six predictions, the coding rules and the coding itself were public before any count was computed. One exception: a last patch touching three cases was written before the count but only went public with the results, and the write-up shows that no coding of those three could change any outcome.
What was counted
- Frame: the 29 posts on Michael Berman's E-Discovery LLC blog (1 Oct 2025 to 31 Oct 2026) that a pre-registered filter picked out. That's a convenience sample, not a census.
- 51 cases. 44 could be coded from the order's own words, 3 were refusals of a proposed AI clause, and 4 couldn't be coded (no free copy of the text).
- 14 orders were checked at the source: 13 free PDFs from the RECAP archive (4 of them image-only, read by OCR) plus one copied from its PDF's text layer. Every call is backed by a quotation that the grader found word for word in the court's text.
What came out
| Prediction | Result |
|---|---|
| At most 20% name self-hosting | held: 3 of 44 (6.8%) |
| At least half test what happens to the data | held: 43 of 44 (97.7%) |
| Few test openness of code or weights, and none of those permits the setup | held: 3 of 44 |
| The setup is permitted in at least half and forbidden in at most 20% | failed: permitted 18 (40.9%), forbidden 2, conditional 10, unclear 14 |
| Orders from different courts share 25+ word runs | held: 28 cross-court pairs, longest 45 words |
| At most a third share a 25-word run with another order | failed: 24 of 44 (54.5%) |
Three things stood out:
- The orders test the data, not where the model runs. Nearly all of them ask whether the tool trains on inputs, keeps them or discloses them. Only 3 mention self-hosting.
- A self-hosted model is mostly "unclear", not "forbidden". In 10 of the 14 unclear orders, the only way to use AI is a contract or promise from an outside AI provider. A model on your own server has no provider to sign one.
- Criminal and civil orders differ sharply (exploratory, not pre-registered). In the 19 criminal orders the setup is permitted in 14, all from one S.D.N.Y. template. In the 25 civil orders it is permitted in only 4.
Reading at the source mattered. Several orders that commentary calls "the same" as another are not. One drops the government's matching duties. One rests its test on what the AI provider represents. One (United States v. Sarris, S.D.N.Y.) leaves software that runs entirely on the defense's own systems out of the clause altogether.
Limits
The sample comes from one blog. The S.D.N.Y. criminal template accounts for 17 of the 44 coded orders, so pooled percentages lean towards it. There was no independent second coder. "Permitted" means the words allow it, not that a court has ruled on it. Every limit is listed in the write-up.
Data and code
- Write-up: https://github.com/munimtechnologies/bar-ai-guidance/blob/main/analysis/ai-protective-orders.md
- Dataset (CC BY 4.0): https://github.com/munimtechnologies/bar-ai-guidance/blob/main/data/ai_protective_orders.json
- The build script (Apache-2.0) re-downloads every source, checks its hash, re-checks every quotation, and refuses to grade if anything fails.
If you know of an AI protective order that this sample misses, open an issue on the repo. The next update will add November's orders under the same rules.
MV2, Munim, Inc., mv2@munimtech.com
Top comments (0)