DEV Community: Trinh Tran Khanh Duy

Local LLM for legal documents: what works, what doesn't (honest review)

Trinh Tran Khanh Duy — Thu, 28 May 2026 17:51:35 +0000

After testing several local LLMs on real legal documents (contracts, NDAs, service agreements), here's an honest breakdown of what's actually useful and where local models fall short.

TL;DR: Local LLMs handle first-pass extraction well. Complex legal interpretation — not yet.

Why local LLMs for legal documents?

The compliance issue is real:

Uploading client contracts to ChatGPT may violate NDA terms
GDPR Article 28 requires a data processing agreement — most cloud AI providers don't have one for individual users
Many firms have explicit policies banning cloud AI for client documents

Local models (via Ollama) solve the compliance problem. The question is whether they're good enough to be useful.

Test setup

Models tested: llama3.1:8b, llama3.1:70b (Q4), qwen2.5:7b, qwen2.5:14b, mistral:7b
Document types: NDAs, service agreements, consulting contracts, lease agreements
Hardware: 16GB RAM (8b models), 32GB RAM (70b)
Evaluation: Manual review by a lawyer friend (non-scientific, practical assessment)

What works well ✅

1. Extracting structured information

Task: "List all parties and their roles."

All 8b+ models do this reliably. Output:

PARTIES:
- Acme Corp (the "Company") — service provider
- John Smith Consulting LLC (the "Consultant") — independent contractor
- Mentioned as guarantor: Jane Smith (personal guarantee, Section 8.2)

Accuracy: ~95% on standard contracts.

2. Key date extraction

Task: "List all dates and deadlines."

Works well. Models catch effective dates, termination dates, notice periods, payment due dates.

Miss rate: ~5-10% for dates buried in complex conditional clauses.

3. Payment terms summary

Task: "Summarize payment terms."

Reliable on standard payment structures. Handles:

Fixed fee contracts
Milestone-based payments
Retainer agreements
Net-30/60/90 terms

Struggles with: complex multi-tier pricing, earn-out structures, revenue share formulas.

4. "Does this contract have a non-compete clause?"

Simple yes/no questions work well across all tested models. Useful for quick triage.

5. Plain-language summary of a section

Task: "Explain Section 7 in plain language."

This is where local LLMs genuinely help non-lawyers. Translating legalese to plain English is a strong use case even for smaller models.

What doesn't work ❌

1. Complex legal interpretation

Task: "Does this indemnification clause put the contractor at unreasonable risk?"

Models either give a wishy-washy non-answer or confidently state something incorrect. Don't use for risk assessment.

2. Identifying all unusual clauses in a long contract

On 50+ page contracts, models miss unusual provisions that appear late in the document. Context window limitations matter here.

Workaround: Process section-by-section, then synthesize.

3. Comparing two contracts

"How does this NDA differ from a standard NDA?" requires the model to have a reliable internal reference. Results are inconsistent.

4. Jurisdiction-specific analysis

Anything requiring knowledge of specific case law or state-specific provisions — unreliable.

Model comparison for legal documents

Model	Speed	Extraction accuracy	Clause identification	Recommended for
qwen2.5:3b	Fast	85%	75%	Quick triage, 8GB machines
llama3.1:8b	Medium	90%	82%	Daily use, 16GB machines
qwen2.5:14b	Slow	92%	85%	Higher accuracy needed
llama3.1:70b	Very slow	94%	88%	Best quality, 32GB+
mistral:7b	Fast	83%	72%	Not recommended for legal

The right use case

Local LLMs for legal documents = first-pass reading assistant, not a lawyer.

Practical workflow:

Run local LLM → get structured extraction in 2-3 minutes
Review summary to identify sections that need deep reading
Do the deep reading yourself (or with a real lawyer)

Time saved: reading a 40-page contract to find the 5 things you actually need to review goes from 90 minutes → 15 minutes.

System prompt that works best

After many iterations, this structure produces the most reliable legal document analysis:

You are a legal document analyst. Your task is to EXTRACT information, not interpret it.

For the provided contract, produce a structured report with these exact sections:

PARTIES: [List each named party, their defined term, and role]
EFFECTIVE DATE: [Date the agreement begins]
TERM: [Duration and termination conditions]
KEY OBLIGATIONS: [Bullet list per party]
PAYMENT: [All payment-related terms]
TERMINATION: [How either party can end the agreement]
UNUSUAL PROVISIONS: [Any non-standard clauses worth flagging]

Be precise. If information is not present, write "Not specified."
Do not give legal advice or risk assessments.

Full tool

I built a Windows app around this workflow: local PDF/DOCX processing, 10 domain modes, batch processing for multiple contracts. Available at https://journeyer376.gumroad.com/l/ussytd.

It's aimed at lawyers and consultants who want the productivity benefit of AI without the compliance risk of cloud tools.

What document types are you using local LLMs for? Would be interested in test cases from others.

Building a privacy-first document processor with Ollama + Gradio

Trinh Tran Khanh Duy — Thu, 28 May 2026 17:46:07 +0000

A step-by-step guide to building a local AI document processor that makes zero external network calls — useful for processing NDA-bound contracts, confidential reports, or any document you can't upload to ChatGPT.

Architecture overview

PDF/DOCX file
    ↓
pdfplumber / python-docx (text extraction)
    ↓
System prompt + document text
    ↓
Ollama API (localhost:11434)
    ↓
Gradio UI (localhost:7860)
    ↓
Summary / Q&A / entities

Everything runs on localhost. Zero cloud dependencies at runtime.

Prerequisites

Python 3.11+
Ollama installed and running
8GB+ RAM (16GB recommended)

# Install Ollama (Windows)
winget install Ollama.Ollama

# Pull a model
ollama pull llama3.1:8b

Core dependencies

pip install gradio pdfplumber python-docx requests

Step 1: Text extraction

import pdfplumber
import docx
from pathlib import Path

def extract_text(file_path: str) -> str:
    path = Path(file_path)
    if path.suffix.lower() == ".pdf":
        with pdfplumber.open(file_path) as pdf:
            return "\n\n".join(
                page.extract_text() or "" for page in pdf.pages
            )
    elif path.suffix.lower() in (".docx", ".doc"):
        doc = docx.Document(file_path)
        return "\n".join(p.text for p in doc.paragraphs if p.text.strip())
    raise ValueError(f"Unsupported file type: {path.suffix}")

Step 2: Ollama integration

import requests

OLLAMA_URL = "http://localhost:11434/api/generate"

def query_ollama(prompt: str, model: str = "llama3.1:8b") -> str:
    response = requests.post(OLLAMA_URL, json={
        "model": model,
        "prompt": prompt,
        "stream": False,
    }, timeout=120)
    response.raise_for_status()
    return response.json()["response"]

Note: http://localhost:11434 — not a cloud API. No authentication needed.

Step 3: Domain-specific system prompts

Generic prompts give generic results. Tuned prompts for document types:

DOMAIN_PROMPTS = {
    "legal": (
        "You are a legal document analyst. Extract and structure the following "
        "from the document:\n"
        "1. PARTIES: All named parties and their roles\n"
        "2. KEY DATES: Effective date, termination, deadlines\n"
        "3. OBLIGATIONS: Each party's obligations\n"
        "4. PAYMENT TERMS: Amounts, schedules, conditions\n"
        "5. UNUSUAL CLAUSES: Non-standard or notable provisions\n"
        "6. GOVERNING LAW: Jurisdiction and dispute resolution\n"
        "Be factual and precise. Do not interpret or give legal advice."
    ),
    "financial": (
        "You are a financial document analyst. Extract:\n"
        "1. AMOUNTS: All monetary values with context\n"
        "2. DATES: Payment dates, fiscal periods, deadlines\n"
        "3. PARTIES: Vendors, clients, counterparties\n"
        "4. TERMS: Payment terms, penalties, conditions\n"
        "5. KEY METRICS: Revenue, costs, margins if present"
    ),
}

def process_document(file_path: str, domain: str, model: str) -> str:
    text = extract_text(file_path)
    system = DOMAIN_PROMPTS.get(domain, "Summarize the key points of this document.")
    prompt = f"{system}\n\nDOCUMENT:\n{text[:12000]}"  # ~12k char limit
    return query_ollama(prompt, model)

Step 4: Privacy-safe Gradio UI

import gradio as gr

def build_ui():
    with gr.Blocks(title="Local Document Processor") as app:
        gr.Markdown("## Local Document Processor\n*All processing on your machine — no cloud*")

        with gr.Row():
            file_input = gr.File(label="Upload PDF or DOCX", file_types=[".pdf", ".docx"])
            domain = gr.Dropdown(
                choices=list(DOMAIN_PROMPTS.keys()),
                value="legal",
                label="Domain"
            )

        process_btn = gr.Button("Process Document", variant="primary")
        output = gr.Textbox(label="Result", lines=20)

        process_btn.click(
            fn=lambda f, d: process_document(f.name, d, "llama3.1:8b"),
            inputs=[file_input, domain],
            outputs=output,
        )

    return app

if __name__ == "__main__":
    app = build_ui()
    app.launch(
        server_name="127.0.0.1",   # localhost only
        share=False,                # no Gradio tunnel
        analytics_enabled=False,    # no phone-home
    )

Step 5: Batch processing

For processing entire folders:

import zipfile
import tempfile
from pathlib import Path

def batch_process(folder_path: str, domain: str, model: str) -> str:
    results = {}
    for file in Path(folder_path).glob("*"):
        if file.suffix.lower() in (".pdf", ".docx"):
            try:
                results[file.name] = process_document(str(file), domain, model)
            except Exception as e:
                results[file.name] = f"ERROR: {e}"

    # Package results as ZIP
    with tempfile.NamedTemporaryFile(suffix=".zip", delete=False) as tmp:
        with zipfile.ZipFile(tmp.name, "w") as zf:
            for filename, content in results.items():
                zf.writestr(f"{filename}.txt", content)
        return tmp.name

Performance tips

Context window: Truncate documents to ~12,000 characters for reliable results with 8b models
Temperature: Set "temperature": 0.1 for factual extraction (less hallucination)
Streaming: Use "stream": True for better UX on long documents — update UI in real-time
Model selection: qwen2.5:3b for speed, llama3.1:8b for quality, llama3.1:70b for accuracy

Verification

Run Wireshark filtered to not host 127.0.0.1 while processing a document. You should see zero packets — confirming no data leaves your machine.

Full product

The complete version (batch mode, 10 domain types, hardware detection, Windows installer, 12 use-case recipes) is available at https://journeyer376.gumroad.com/l/ussytd for $39.

The architecture above is the core of what it does — the product adds packaging, documentation, and domain prompt iteration aimed at non-developers.

Questions about the architecture or model benchmarks? Happy to answer in the comments.

How to verify your AI tool makes zero network calls (Wireshark guide)

Trinh Tran Khanh Duy — Thu, 28 May 2026 17:45:03 +0000

If you run a "local AI" tool and want to prove it's actually offline, here's the definitive verification method using Wireshark (Windows/macOS/Linux).

Why this matters

Many tools claim to be "private" or "local" but still phone home for:

Telemetry and crash reporting
License validation
Analytics (Gradio does this by default)
Model updates or sync

The only way to know for certain is to watch the network.

Step 1: Install Wireshark

Download from wireshark.org. Free, open source.

Step 2: Start a capture

Open Wireshark
Select your active network interface (Ethernet or Wi-Fi)
Click the blue shark fin icon to start capturing

Step 3: Apply the filter that matters

In the display filter bar, type:

not (ip.dst == 127.0.0.1 or ip.dst == ::1 or ip.src == 127.0.0.1 or ip.src == ::1)

This filters OUT localhost traffic and shows ONLY external network calls.

Step 4: Run your AI tool and process a document

Now open your "local AI" tool and process something — a PDF, an image, a text file. Let it run to completion.

What you should see

Truly offline tool: zero packets in the filtered view.

Tool that phones home: you'll see packets to external IPs. Right-click any → "Follow TCP Stream" to see what data was sent.

Applying this to Ollama + Gradio

When running an Ollama-based app, you'll typically see:

127.0.0.1:11434 — Ollama API calls (localhost, expected)
127.0.0.1:7860 — Gradio UI (localhost, expected)

If you see anything else, the app has external dependencies.

Disabling Gradio's default analytics

Gradio sends analytics to HuggingFace by default. To disable:

app.launch(
    server_name="127.0.0.1",  # bind to localhost only
    share=False,               # disable Gradio tunnel
    analytics_enabled=False    # disable phone-home
)

Without these settings, you'll see outbound traffic to huggingface.co during app startup.

Windows alternative: Resource Monitor

If you don't want to install Wireshark:

Open Task Manager → Performance tab → "Open Resource Monitor"
Go to the Network tab
Expand "Network Activity"
Filter for your Python/Ollama process

This shows active connections per process — easier than Wireshark for a quick check.

The verification script

I built a small Python script that automates this check and generates a report:

import subprocess
import re

def check_external_connections(duration_seconds=30):
    """Monitor network connections during AI processing."""
    import psutil
    import socket

    external_connections = []
    for conn in psutil.net_connections(kind='inet'):
        if conn.raddr and conn.status == 'ESTABLISHED':
            remote_ip = conn.raddr.ip
            if not remote_ip.startswith('127.') and remote_ip != '::1':
                external_connections.append({
                    'remote_ip': remote_ip,
                    'remote_port': conn.raddr.port,
                    'pid': conn.pid
                })
    return external_connections

This is included in the Offline PDF Summarizer product — you can run your own network audit any time.

Summary

Method	Setup time	Detail level
Wireshark	5 min	Full packet capture
Resource Monitor	0 min (built-in)	Per-process connections
Python psutil script	2 min	Scriptable, automated

The Wireshark method is definitive. Use it to verify any tool that claims to be "offline" before trusting it with confidential documents.

Building privacy-first tools for professionals who can't use cloud AI.