DEV Community: Muhammad Monjurul Karim

Redacting PII 100% Offline: A Technical Deep Dive

Muhammad Monjurul Karim — Mon, 10 Aug 2026 16:04:49 +0000

Cross-posted from Medium. Originally published at https://medium.com/data-science-collective/redacting-pii-100-offline-a-technical-deep-dive-a326f9fed3b8

If you handle contracts, medical records, legal discovery, or even just research data, redacting PII is not optional. HIPAA, GDPR, PCI, FOIA — they all want it gone. And yet the tools we reach for are stuck in two failure modes.

The first is the cloud route: you upload a document full of the exact personal data you're trying to protect, and a server sends back a "redacted" copy. That's a privacy paradox — you solve the privacy problem by first exposing the document. For anything sensitive, that's a non-starter.

The second is the manual route: draw black rectangles over every SSN by hand. Slow, error-prone, and because most viewers black the rendering rather than the text, the layer underneath can often be selected or deleted to reveal what was "redacted." Or the tool auto-blacks whole lines and you get a wall of censor bars.

Neither is good enough. I wanted accurate, automatic, value-only redaction that runs on my own machine. So I built one. (If you want the tool and not the story, it's here: github.com/monjurulkarim/privateredact.) This post walks through how it works — extraction, hybrid regex + local-LLM detection, the part nobody gets right (drawing the box), and proving the output is actually clean.

The "why"

The goal wasn't just to scribble black over a PDF. It was to build something I'd trust with a real medical record or contract:

Read anything — text PDFs, scanned/image PDFs (OCR), DOCX, TXT, images.
Find PII semantically, not just by format. Regex catches a phone number. It does not catch "patient presented with Type 2 diabetes."
Redact the value, not the label — 555-1234, not Phone: 555-1234. Never the whole line.
Preserve the document — DOCX formatting intact.
Prove it worked — a post-export scan that re-OCRs the output and reports what survived.
100% local — no file, hash, or byte leaves the machine.

The stack

Everything below runs on the user's machine. No backend.

Electron — bundles the renderer, the PDF/OCR engines, and the LLM into one installable app.
pdf.js (Mozilla) — reads born-digital PDFs, and crucially exposes each text item's bounding box.
Tesseract (v6) — OCR for scanned PDFs and images; v6 returns word-level boxes.
Ollama — local LLM runtime for semantic detection.
pdf-lib (encrypted-aware fork) — output assembly + password-protected I/O.
sharp + node-canvas — rasterizing pages and painting redaction boxes.
jszip — DOCX files are zip-of-XML; this lets you edit OOXML in place.

Extraction: born-digital vs. scanned

A PDF isn't a PDF. Some have a real text layer; others are wrapped images of scanned paper. Redacting text you never read is how leaks happen — so you fork on whether the page actually has a text layer:

# Conceptual — two extraction paths
if page.has_text_layer:
    # born-digital: pdf.js gives each text item + its bounding box
    words = [{'text': item.text, 'box': item.bbox} for item in page.items]
else:
    # scanned / image: OCR it, and we need WORD-level boxes, not just text
    ocr = tesseract.recognize(page.render_to_image(), blocks=True)
    words = [{'text': w.text, 'box': w.bbox} for w in ocr.words]

Non-obvious bit: when you OCR, you must ask for word-level boxes (Tesseract's { text: true, blocks: true }). Otherwise the engine hands you whole paragraphs as one box, and you're back to blacking entire lines.

Detection: regex first, LLM second

Two layers, because neither is sufficient alone.

The regex layer is fast and precise for well-formatted PII. The trick is value-only matching — you don't want to black SSN:, you want to black the number after it:

# Redact the VALUE (capture group 2), not the label (group 1)
pattern = r'(SSN|Social Security)[:\s]+([0-9X\-]{9,})'
for m in re.finditer(pattern, text):
    value_start, value_end = m.span(2)   # the number's offsets only
    boxes = offsets_to_boxes(value_start, value_end, page)

Real gotcha: ID patterns need a digit lookahead. Without one, they false-positive on prose — "passport control", "case study", "member number two."

The LLM layer catches what patterns structurally cannot: a diagnosis, a codename, an address buried in a sentence. The page text is chunked and sent to the local Ollama model with a strict JSON schema, and it returns { type, text, start, end, confidence }:

detections = regex_scan(page.text)              # fast, known formats
detections += llm_scan(page.text, model=ollama) # context-only PII
for d in detections:
    d.boxes = resolve_span(d.text, page)        # tolerant of case/whitespace

The hard part isn't calling the model — it's trusting its output enough to draw a box. You have to actually find that string in the page text and map it to a real bounding box; if the match rate is low, you drop the detection rather than risk a misplaced box.

The part nobody gets right: drawing the box

This is where most redaction tools fall apart.

pdf.js doesn't return one box per word. It returns runs of text, and a single run can span an entire line — sometimes 80% of the page width. If you naively black the run's box for any entity inside it, you black the label, the surrounding words, everything.

The fix is proportional sub-boxes. Inside a run, you know the character offset where the entity starts and ends. Assuming roughly uniform char width within that run, black only that horizontal slice:

# Don't black the whole run. Black the SLICE the entity occupies.
def proportional_box(run_box, ent_start_char, ent_end_char, run_len):
    width = run_box.x1 - run_box.x0
    pad   = 0.4 * (width / run_len)          # a little breathing room
    x0 = run_box.x0 + (ent_start_char / run_len) * width - pad
    x1 = run_box.x0 + (ent_end_char   / run_len) * width + pad
    return Box(x0, run_box.y0, x1, run_box.y1)

So Phone: ████-████ redacts just the bold part. The label survives. The document stays readable.

Two more rules, learned the hard way:

One union function, used twice. The boxes shown in the review overlay must be exactly the boxes painted on export. If they differ, what the user approves is not what they get. Both paths call the same routine.
Reject pathological boxes. If a computed box covers 95%+ of the page, something went wrong. Reject that detection outright rather than black the page.

Export and proof

For PDF, each page is rasterized, the unioned boxes are painted solid black, and the result is embedded into a fresh PDF. Rasterized output is the point: no recoverable text layer. You cannot select-and-delete the black box to reveal the original.

# Rasterize -> paint -> embed. No text layer survives.
for page in document:
    raster = render(page)
    raster = paint_black(raster, page.unioned_boxes)
    output_pdf.embed(raster)

DOCX is trickier — you want to keep the formatting. A DOCX is a zip of XML, so redaction is in-place surgery on the XML text nodes: find the runs, replace the PII characters inside them, leave every other node untouched.

The last stage is the one I care most about: the leak scan. After the file is written, the output is OCR'd again and re-scanned. It runs on the exact bytes written, and it never claims "clean" if the scan was skipped or failed:

output_text = ocr(exported_pdf_bytes)
remaining   = scan_for_pii(output_text)
if scan_failed_or_skipped:
    report("could not verify — do not ship")
elif remaining:
    report(remaining)          # exactly what survived
else:
    report("0 PII remaining")

"0 PII remaining" isn't a button label — it's the result of actually re-reading the redacted file and checking.

Frontend: approve what you export

A backend this autonomous is also dangerous — you don't hand a model the keys to black out a contract unsupervised. The frontend is a review surface: a React overlay on the page image showing every proposed redaction as a box, with a toggle between image and raw text. Remove a box the model got wrong, or draw one it missed. Because the overlay uses the same union-boxes logic as the exporter, what you see is byte-for-byte what gets painted.

Want to try it yourself?

I packaged the whole thing — extraction, hybrid detection, the review UI, the leak-scan proof — into a desktop app. It runs entirely on your machine: no account, no upload, nothing leaving your disk.

The fastest way to see it work on your own documents is the free build on GitHub. You can load, detect, and review a real file end to end — exporting the redacted output is the only thing behind a license.

→ Try it: github.com/monjurulkarim/privateredact
→ Or grab a license to export: in this link

Over to you: what's the hardest PII you've had to redact from a document — and did the tool you used actually get it right? I'm genuinely curious what detection gaps people hit in practice. Drop it in the comments 👇

How I Built a Private, Multi-User “Chat with Your Documents” App That Runs 100% Offline

Muhammad Monjurul Karim — Tue, 30 Sep 2025 18:33:31 +0000

A quick note: This article was originally published on Medium. You can read it there by clicking here.

Last month, I wrote an article on how I built a self-hosted AI Meeting Note Taker. The response was fantastic and showed a clear demand for private, offline AI tools that put users back in control of their data.

Whether you’re a small business uploading sensitive client data, a developer working on proprietary code, or just someone who wants to organize personal files without sending them to a third-party server, the problem is the same. You’re forced to choose between powerful AI tools and data privacy.

Building on that momentum, I decided to create the solution I wanted to see: a full-featured, multi-user “Chat with Your Documents” application that runs entirely on your own hardware. This is the second part of my self-hosted AI series — a technical deep-dive into a production-ready RAG application for everyone.

The Mission: A Private AI for Everyone

The core technical goals for this project were:

100% Local Processing: All data, from your company’s strategy documents to your personal financial records, is processed locally. Nothing ever leaves your server.
Team & Family Ready: The system is designed for multiple users, with isolated knowledge bases and a central admin dashboard to manage everything.
Efficient & Smart: The RAG pipeline intelligently syncs only new or modified documents, saving time and computational resources.
Incredibly Simple to Deploy: The entire application is packaged into a single executable with easy-to-use installer scripts. No complex setup required.

The Tech Stack: The Powerhouses

This application is built on a foundation of powerful open-source tools:

Backend: Flask serves as the lightweight web server, with Gunicorn for threaded performance to handle concurrent users.
AI & Embeddings: Ollama runs open-source LLMs (like Llama 3, Gemma) and embedding models locally.
Vector Store: ChromaDB provides a persistent, on-disk vector database for storing document embeddings efficiently.
RAG Orchestration: LangChain glues the components together, managing the flow from document loading to question-answering.
User Management: A custom authentication layer using SQLite for persistence and PyJWT for secure sessions.

Hardware Requirements & Performance

A common question with self-hosted AI is about the hardware required. The application is designed to be flexible, and performance will scale with your machine’s capabilities. The main requirement is sufficient RAM to load the language models.

To give you a real-world idea, here are a few setups I’ve tested:

Good (Accessible Start): For users with more modest hardware (e.g., a laptop with 8–16GB of RAM), the application runs well with a smaller, efficient model like gemma3:4b. The results are surprisingly good for most document Q&A tasks, making this a great starting point.
Better (Smooth Personal Use): On my MacBook with 24GB of RAM, it handles a 12-billion parameter model (eg.gemma3:12b) very smoothly for all my personal documents and projects. This offers a noticeable boost in the quality of the AI’s responses.
Best (Team Performance): At my office, we have it on a server with an NVIDIA A6000 GPU. Our whole team of 6 uses it with the much larger gemma3:27b model without any issues.

This demonstrates that the application can effectively scale from a standard laptop for individual use to a dedicated server for team collaboration, depending on your needs.

Deep Dive: The System Architecture

The application is broken down into three main Python components: the web server (main_app.py), the authentication system (auth.py), and the RAG core (local_rag_chroma.py).

The application includes a secure login portal and a dashboard for user administration. These features allow for multi-user support, making the app suitable for shared use by a project team, a small business, or a family, with each user having their own secure access. (image by author)

1. The RAG Core: `local_rag_chroma.py`

This is the heart of the application. I designed a KnowledgeBaseManager class to handle the entire lifecycle of document processing and retrieval.

Intelligent Document Synchronization

To avoid redundant processing, I implemented a synchronization function that compares the state of files on disk with the metadata stored in ChromaDB.

The _synchronize_documents method works in a few steps:

It scans the document directories and creates a dictionary of all current files and their last-modified timestamps.
It queries the ChromaDB collection to get a list of already-indexed files and their modification times stored in the metadata.
By comparing these two lists, it identifies:

New files to be added.
Modified files that need to be deleted and re-indexed.
Deleted files whose chunks must be removed from the database.

This ensures that only necessary changes are processed, making startup and resyncing incredibly fast.

Document Loading and Splitting

The system uses LangChain’s document loaders to handle various file types (.pdf, .docx, .md, .txt, .csv,etc.). Each document is then split into manageable chunks using the RecursiveCharacterTextSplitter.

# A snippet from _load_and_split_document in local_rag_chroma.py
def _load_and_split_document(self, file_path):
    # ... logic to select the correct loader based on file extension ...
    loader = PyPDFLoader(file_path) # Example for PDF
    documents = loader.load()    # Add file metadata for synchronization
    file_mod_time = os.path.getmtime(file_path)
    for doc in documents:
        doc.metadata['source'] = file_path
        doc.metadata['last_modified'] = file_mod_time
        text_splitter = RecursiveCharacterTextSplitter(
                chunk_size=CHUNK_SIZE, 
                chunk_overlap=CHUNK_OVERLAP
            )
    split_docs = text_splitter.split_documents(documents)
    return split_docs

Embedding and Storage

The chunks are then passed to a locally running embedding model via OllamaEmbeddings. Each resulting vector is stored in a persistent ChromaDB collection, creating a searchable index for that specific knowledge base. The application creates separate, isolated collections for each team's knowledge base and each user's personal documents, ensuring data segregation.

Here, the RAG system is demonstrated on a private, personal note. The model is able to extract and synthesize information directly from the user’s content. Because all processing is handled locally, it’s possible to run queries on sensitive information without the data ever leaving the machine. (image by author)

This example demonstrates the system’s ability to query complex, structured documents. The model extracts specific data points from a financial report in response to a direct question. This showcases its utility for analyzing dense information, whether it’s technical documentation, financial data, or academic papers. (image by author)

2. The Multi-User System: `auth.py`

A robust application needs a solid authentication and authorization layer. I built one from scratch using a simple SQLite database to store user and session information.

Database and Authentication

The DatabaseManager class sets up tables for users, sessions, and analytics. User passwords are never stored in plain text; instead, I use werkzeug.security to store salted and hashed passwords.

When a user logs in, the system verifies their credentials and generates a JSON Web Token (JWT) that is used to authenticate subsequent API requests.

Securing Endpoints with Decorators

To protect the API, I implemented custom decorators. The @require_auth decorator checks for a valid session or JWT, while @require_role('admin') restricts access to admin-only endpoints, like the analytics dashboard.

# A snippet from auth.py
def require_role(required_role):
    """Decorator to require specific user role."""
    def decorator(f):
        @wraps(f)
        @require_auth
        def decorated_function(*args, **kwargs):
            user_role = request.current_user.get('role', 'user')
            # ... logic to check if user has sufficient privileges ...
            return f(*args, **kwargs)
        return decorated_function
    return decorator

This design makes it easy to secure new API routes as the application grows.

User management using the admin dashboard (image by author)

3. The Web Server: `main_app.py`

The Flask application ties everything together. It defines the API endpoints for the frontend to interact with. The most important one is /api/ask.

# A simplified view of the /api/ask endpoint in main_app.py
@app.route('/api/ask', methods=['POST'])
@require_auth
def ask_question_api():
    session_id = get_or_create_session_id()
    data = request.get_json()
    question = data['question']
    kb_id = data.get('knowledge_base_id', 'user_personal')
    # Get or create a RAG instance for this session and knowledge base
    rag_chain, retriever, memory = get_or_create_rag_instance(session_id, kb_id)

    if not rag_chain:
        return jsonify({"error": "RAG service not ready."}), 503
    # Invoke the RAG chain and return the response
    response = rag_chain.invoke({"query": question})

    # ... format and return the answer and sources ...

The application maintains a thread-safe dictionary (rag_instances) to manage separate RAG chains and conversation histories for each user session and selected knowledge base, preventing memory leaks and ensuring that conversations are isolated.

The Result: Your Private AI Knowledge Base for Work and Life

The final result is a powerful, production-ready application that turns any collection of documents into an interactive AI assistant. It’s the perfect tool for:

Small Businesses wanting a secure, internal knowledge base for their team without the high cost and privacy risks of SaaS products.
Developers & Freelancers who need a powerful, private RAG system for their own documents or as a foundation for client projects.
Personal Users & Families who want to securely organize and ask questions about their private files, from financial records to research papers.

It’s the best of both worlds: the power of modern AI without sacrificing control over your data.

Want to Run It Yourself?

This project takes the core principles of local processing from the AI Meeting Note Taker and expands them into a robust, multi-user platform ready for any use case.

I’ve packaged the entire source code and made it available for a one-time purchase. It’s a fantastic way to get a powerful, private AI tool up and running in minutes and serves as a solid foundation for your own customizations.

For a limited time, use the code MEDIUM25 for a 25% discount.

You can get the complete RAG Application project by clicking here

Thanks for reading! I hope this deep-dive inspires you to take control of your data and explore the incredible potential of self-hosted AI.