DEV Community

Cover image for wardcat: On-Premise Data Privacy Architecture for RAG and LLM Projects
Oğuzhan TOPCU
Oğuzhan TOPCU

Posted on Originally published at oguzhantopcu0.github.io

wardcat: On-Premise Data Privacy Architecture for RAG and LLM Projects

#ai #security #privacy #python

Hey folks!

I developed and published wardcat on PyPI to solve the security risks of sending enterprise data to third-party LLM APIs and to prevent data leaks in RAG pipelines completely on-premise. Going beyond simple pattern matching, I built a hybrid layered architecture that integrates Regex, SpaCy NER, and local open-weights LLMs (such as Qwen3:14b). Thanks to this architecture, sensitive data in texts is not only detected with high accuracy; texts anonymized using the reversible masking feature can be restored to their original form after LLM processing, and with the is_sensitive() function, prompts are passed through a contextual security filter before being processed.

To install directly from PyPI:

pip install wardcat
Enter fullscreen mode Exit fullscreen mode

To validate the architecture, here is a concise summary of the benchmark results compared against Microsoft Presidio, which is widely considered the industry standard:

Test Scenario / Corpus Presidio wardcat (Regex + NER) wardcat (LLM Layer)
Presidio Corpus (1,500 samples - F1) 0.703 0.759 —
Independent Corpus (Gretel Finance - F1) 0.462 0.504 —
Challenging Scenarios (100 cases - F1) 0.838 0.875 0.920
Sensitivity Classification (100 texts - Accuracy) 63% 72% 88%

As clearly shown in the table, while the rule-based (Regex) and linguistic (NER) layers alone outperform Presidio, an F1 score of 0.920 is achieved in challenging cases with the wardcat LLM layer. Especially when understanding the overall sensitivity of a text, the is_sensitive() filter performs far beyond traditional rule engines with 88% accuracy. These results prove how critical it is to use a hybrid, layered architecture to eliminate data leakage rather than relying on a single detection method.

If you'd like to secure your own on-premise pipelines, feel free to check out the project. Please feel free to reach out to me to contribute, share new ideas, or ask any questions!

github: https://github.com/oguzhantopcu0/wardcat
docs: https://docs.wardcat.com/
pypi: https://pypi.org/project/wardcat/

Top comments (0)