This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle
What I Built
Over the past year, many freelance developers, small agencies, and independent builders have integrated LLM capabilities into their client work. Whether creating customer intake assistants, contract summarizers, or invoice extractors, clients increasingly expect AI-driven workflows.
However, deploying generative models in client-facing environments introduces several practical operational and compliance challenges. Clients and end users frequently paste sensitive personal data - including customer contact information, phone numbers, and production API credentials - directly into prompt inputs. Input fields are also vulnerable to adversarial prompt injections designed to bypass system instructions. In addition, document analysis workflows can be targeted with hidden or low-contrast text embedded within PDFs and images, potentially hijacking downstream reasoning.
While enterprise AI security platforms exist, they are typically aimed at large organizations with corresponding price points and complex onboarding processes. For independent developers and small software studios delivering custom applications on modest budgets, these enterprise solutions are often impractical.
I built AEGIS as a lightweight, drop-in security proxy that developers can place in front of their LLM endpoints. It acts as an intermediary gateway that sanitizes personal data in real time, inspects and OCRs uploaded documents, evaluates prompt safety using an open-source fine-tuned Gemma 3 model, and provides sanitized web retrieval through SerpApi. For agencies and independent contractors, this provides a straightforward security layer that can be offered to clients as a paid compliance add-on or hosted as a managed micro-service.
Incoming Untrusted Payload (Text / PDF / Image / Web Search)
│
▼
[ Layer 1: Magika Deep MIME & Polyglot Verification ]
│
▼
[ Layer 2: Dual-Stream OCR & Hidden Text Extractor ]
│
▼
[ Layer 3: Context-Aware Presidio + Regex PII Scrubber ]
(Transforms PII -> <EMAIL>, <PHONE>, <API_KEY>)
│
▼
[ Layer 4: Fine-Tuned Gemma 3 / LoRA Safety Classifier ]
(Detects Prompt Injections & Malicious Payloads)
│
▼
[ Layer 5: Sanitized Web Retrieval via SerpApi ]
(Filters Poisoned Knowledge Blocks & Injections)
│
▼
Clean, Verified Context -> Target LLM -> User
The gateway handles inbound requests across multiple input types. Plain text prompts are stripped of credentials and personal identifiers before being evaluated by the safety classifier. File uploads undergo deep MIME verification and visual OCR to detect obscured text before text extraction proceeds. When web search augmentation is enabled, SerpApi results are retrieved and scrubbed before being appended to the model context. A web dashboard provides real-time visibility into each verification step, showing exactly what data was modified and why.
Demo
The application provides an interactive interface built to demonstrate each stage of the proxy pipeline. Users can enter raw prompts, upload documents, or toggle web augmentation while observing how the system handles the input.
When a request is submitted, the interface displays the immediate verdict from the fine-tuned Gemma 3 classifier alongside a detailed breakdown of the sanitization process. Original text and redacted text are presented side by side, highlighting replaced entity tags such as email addresses, phone numbers, and API tokens. For file submissions, the system displays the extracted text alongside confidence scores from the OCR and file analysis stages. The dashboard also includes reference summaries of recent real-world security incidents, such as source code leaks and exposed credentials, to provide practical context for why these protective layers are necessary in production software.
Code
The complete source code is available on GitHub at
AEGIS 🛡️
Guardian for your LLMs - A secure, production-ready AI gateway with multi-layer defensive sanitization.
An open-weight, privacy-first proxy that intercepts sensitive PII, neutralizes prompt injections, de-weaponizes uploaded documents, and filters web context using a fine-tuned Gemma 3 model.
Overview • Architecture • Key Features • Tech Stack • Quickstart • Verification
Overview
As generative AI transitions into production workflows, organizations face several critical operational risks: accidental exposure of personal identifiable information (PII), adversarial prompt injections designed to hijack model instructions, and malicious payloads hidden within uploaded files or untrusted web context.
AEGIS provides an automated security proxy that sits between client applications and downstream models. It cleans inbound prompts, redacts sensitive credentials and contact data, inspects multi-format documents using dual-stream OCR, classifies injection threats using an open-source fine-tuned Gemma 3 model, and sanitizes web knowledge retrieved through SerpApi.
Zero Trust AI Gateway: Raw client prompts, uploaded documents…
backend.py, the training and inference notebook for fine-tuning Gemma 3 in backend notebook/aegis-server-wow-verse-mmcoe.ipynb, and the frontend interface in the src/ directory.
How I Built It
AEGIS is structured around an asynchronous multi-layer pipeline combining open-source models, parsing utilities, and structured validation routines.
At the core of the system is a fine-tuned Google Gemma 3 open-weight model (gemma-safety-classifier). Using open weights was central to this project because safety classification needs to be fast, deterministic, and free of per-token API charges. I fine-tuned Gemma 3 using Low-Rank Adaptation (LoRA) through the PEFT library, training it specifically to evaluate user prompts and classify them into discrete categories including safe prompts, prompt injections, cybercrime or malware payloads, and general policy violations. To keep memory utilization minimal while maintaining classification accuracy, the model runs with 4-bit NormalFloat quantization through BitsAndBytes:
# Quantized loading of the fine-tuned Gemma 3 safety classifier
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16
)
peft_config = PeftConfig.from_pretrained(model_path)
base_model = AutoModelForCausalLM.from_pretrained(
peft_config.base_model_name_or_path,
quantization_config=bnb_config,
torch_dtype=torch.bfloat16,
trust_remote_code=True,
low_cpu_mem_usage=True
)
self.model = PeftModel.from_pretrained(base_model, model_path)
By deploying the fine-tuned Gemma 3 model directly within the proxy, the system evaluates inputs deterministically without routing them to third-party providers. The model is paired with a lightweight heuristic check for common benign drafting requests, ensuring that standard creative writing tasks pass through without unnecessary latency.
File uploads introduce an attack surface where malicious instructions can be hidden inside documents. To handle this, AEGIS uses Google Magika to inspect byte headers and confirm that files match their claimed extensions. When processing PDFs, the system uses a dual-stream parsing approach: pdfplumber extracts standard textual content, while pdf2image renders each page into an image for OCR evaluation using Tesseract and EasyOCR. This allows the system to catch adversarial techniques such as zero-opacity fonts, off-canvas text, or white-on-white text that bypass typical text parsers.
Personal data sanitization runs through Microsoft Presidio combined with targeted regular expressions. The pipeline identifies emails, phone numbers, credit card numbers, US Social Security numbers, IBANs, and common API key patterns, replacing them with standardized placeholders such as <EMAIL> or <API_KEY> before text reaches any downstream LLM:
def sanitize_input(user_text: str) -> Dict[str, Any]:
regex_cleaned = regex_sanitize(user_text)
HIGH_RISK = {"EMAIL_ADDRESS", "PHONE_NUMBER", "CREDIT_CARD", "API_KEY", "IBAN", "US_SSN"}
final_text, presidio_findings = presidio_redact(regex_cleaned, redact_types=HIGH_RISK)
return {
"original": user_text,
"final_sanitized": final_text,
"presidio_findings": presidio_findings
}
For applications requiring external knowledge retrieval, AEGIS integrates SerpApi to query Google Search. Rather than passing raw search results directly into the prompt context, AEGIS extracts organic search snippets, question summaries, and overview blocks, running each text fragment through the fine-tuned Gemma 3 classifier and PII sanitizer before assembling the final reference block. This mitigates indirect prompt injection from untrusted external websites:
def secure_serp_knowledge(results: Dict) -> List[str]:
raw_blocks = extract_knowledge_from_serp(results)
safe_blocks = []
for text in raw_blocks:
threat = detect_prompt_injection(text)
if threat != "SAFE":
continue
if len(text.strip()) >= 20:
safe_blocks.append(text)
return safe_blocks
Why Does Open Innovation Matter?
For a developer tool designed as a sustainable side project, using open-source AI is what makes the architecture viable both technically and commercially.
First, open weights make the unit economics predictable. If a security proxy had to invoke a closed commercial API to evaluate every incoming prompt, inference costs would scale directly with volume. A client sending hundreds of thousands of requests per month would generate significant API bills, eroding the margins of a flat-rate subscription. By running a fine-tuned Gemma 3 model in 4-bit precision, inference runs on accessible consumer or cloud GPU hardware with zero marginal cost per request.
Second, open models uphold genuine data privacy. Routing unvetted user prompts containing confidential customer information to a third-party proprietary API simply to check for sensitive data creates the very exposure problem the tool is meant to solve. Running an open-source model locally or in a dedicated private environment ensures that raw customer data never leaves the developer's controlled infrastructure.
Finally, open innovation enables task-specific fine-tuning. General-purpose proprietary models are designed to balance broad conversational capabilities, making them prone to either over-refusing benign technical prompts or missing specialized adversarial injection techniques. Having direct access to Gemma 3 allowed me to fine-tune the weights on dedicated security and prompt injection datasets, resulting in a compact, highly focused classifier that performs its specific role reliably.
My First User
To test AEGIS in a real-world workflow, I shared an instance with an independent developer who builds intake chatbots and document summarization tools for local accounting clients. His primary concern was handling client tax documents where individuals frequently paste sensitive identity numbers or confidential notes into unstructured form fields.
We configured his development environment to route test requests through AEGIS. During testing, he submitted sample tax summaries containing mock Social Security numbers and phone numbers, as well as test documents with hidden prompt injection overrides. AEGIS redacted the personal data into standardized tokens before passing the text to the summarizer, and successfully flagged the injection attempts without disrupting normal queries.
Based on the testing, he noted that having an auditable sanitization layer reduced the compliance risk of offering AI features to his clients. He indicated he would pay a monthly fee of $45 to route his production client applications through a hosted instance of the proxy, and suggested adding webhook alerts to notify his team via Slack whenever a prompt injection attempt is intercepted.
My Agent Session
Hacktoberfest Dev Challenge: Touch Grass. Build an open-source AI project that gets people off the screen and into the real world, running open-weight models locally with privacy and grounded discovery.
Designing Feels: a local, privacy-first web application that turns recent Spotify listening history into curated outdoor nature destinations. Architecture uses local Qwen 2.5 Coder 7B via Ollama for mood synthesis, SerpApi for grounded Google Maps venue discovery, SQLite WAL mode for deterministic caching and rate limiting, and editorial neobrutalism UI.
Separate the Landing Page and the App. After connecting to Spotify, redirect to the main app with a 4-stage data processing indicator before beautifully revealing the outdoor recommendations. Rename the app to Feels.
Segregated editorial landing page at / and application dashboard at /app. Integrated real-time 4-stage processing progress bar (track reading, local Qwen inference, SerpApi discovery, and grounding), updated branding to Feels, migrated SQLite to feels.db, and verified 15/day quota enforcement.
Prize Categories
Best Use of Gemma
AEGIS uses a fine-tuned Google Gemma 3 open-weight model as its central security classifier. Trained with LoRA adapters and running with 4-bit quantization, Gemma 3 provides deterministic, low-latency classification for prompt injections, malware payloads, and unsafe requests without relying on closed cloud moderation APIs.
Top comments (0)