Generative AI has become part of everyday software development.
Developers use AI assistants to debug code, analyze logs, generate scripts, review configurations, summarize documents, and troubleshoot production issues.
But there is a security problem hiding inside this workflow.
A developer can unintentionally send sensitive information to an AI service simply by asking for help.
For example:
Why is this API request failing?
API_KEY=sk-xxxxxxxxxxxxxxxx
INTERNAL_HOST=prod-db.internal
customer_email=user@example.com
The developer's intention is completely legitimate.
The problem is the data contained in the prompt.
The new data-flow problem
Traditional application security generally focuses on protecting systems, endpoints, networks, identities, and data stores.
AI assistants introduce another data-flow:
User
|
v
Prompt / File
|
v
AI Application
|
v
AI Model
The security question becomes:
What is inside the prompt or file before it reaches the AI service?
That requires content-aware inspection.
What should happen before the request leaves the device?
A security layer can sit between the user's interaction and the external AI service.
Conceptually:
ON-DEVICE
|
User ---> Prompt ---> Inspection ---> Policy ---> AI Service
|
+-- Detect
+-- Warn
+-- Mask
+-- Block
This architecture allows the security decision to happen locally.
Instead of sending sensitive content somewhere else for inspection, the device can inspect the content before it is transmitted.
What can be detected?
An AI-focused DLP layer can look for several categories of sensitive information:
- Credentials and API keys
- Personally identifiable information
- Financial information
- Internal infrastructure information
- Organization-specific sensitive keywords
- Potential prompt-injection patterns
The important point is that detection is based on the content being sent to AI, rather than simply the destination.
Prompts aren't the only problem
AI applications increasingly accept files.
A user may upload:
- PDF documents
- Word documents
- screenshots
- images
- application logs
- scanned documents
A document can contain sensitive information even when the user never explicitly types that information into the prompt.
For this reason, AI data protection needs to consider both:
Text input
+
File upload
For scanned documents and images, OCR can make the content available for inspection before the file reaches the AI service.
Security policy needs to be flexible
A simple "block everything" approach creates another problem: users stop trusting the security control.
Different situations require different responses.
For example:
Low risk
|
+--> Allow
Potential risk
|
+--> Warn
Sensitive information that can be removed
|
+--> Mask
High-risk information
|
+--> Block
Exceptional business case
|
+--> Require approval
This is where policy becomes as important as detection.
The detection engine identifies the risk.
The policy engine decides what should happen.
Why on-device inspection?
Consider two architectures.
Cloud inspection
User
|
v
DLP Cloud Service
|
v
AI Provider
The organization now has another location where potentially sensitive prompts must be processed.
On-device inspection
User
|
v
Local Inspection
|
v
Policy Decision
|
v
AI Provider
The inspection occurs locally before the data leaves the endpoint.
This can be particularly useful for organizations with strict privacy, confidentiality, or data-residency requirements.
A practical implementation
The SafePrompt Agent project demonstrates this architecture with an on-device security agent, detection components, policy controls, local audit logging, and browser integration.
The agent is implemented in Rust and can work with a browser extension to inspect AI interactions locally.
The architecture can be viewed as:
Browser
|
v
Browser Extension
|
v
Local Agent / API
|
+----------+----------+
| | |
v v v
Detection Policy Audit
| |
+----------+
|
v
Allow / Warn /
Mask / Block
This approach is useful because the security control becomes part of the AI interaction rather than something that only monitors the network afterward.
The larger engineering challenge
AI security isn't only about securing models.
It is also about securing the information flowing into those models.
For developers, the question is becoming:
Can we use AI as freely as possible while preventing accidental exposure of secrets, PII, confidential documents, and internal information?
That requires a security layer capable of understanding both the AI interaction and the data being transmitted.
The future of AI security will likely involve a combination of:
Identity
+
Access Control
+
DLP
+
Prompt Security
+
Policy Enforcement
+
Local Inspection
+
AI Governance
The interesting engineering challenge isn't stopping people from using AI.
It is making AI usage safer without destroying productivity.
That is where an on-device AI data-security boundary becomes an interesting architectural pattern.
sg2technologies
/
safeprompt_agent
On-device DLP security for AI prompts and file uploads. Scan, mask, warn, or block sensitive data before it reaches AI tools.
SafePrompt Agent
Open-source, on-device data-loss-prevention scanning for AI chat tools.
SafePrompt inspects text and files before they reach ChatGPT, Claude, Gemini, Copilot, or any other AI tool — catching secrets, personal data, financial information, and prompt-injection attempts, and masking or blocking them as you configure. Nothing is scanned in the cloud; detection runs entirely on-device.
This repository is the Community edition: the Rust Agent (agent/ at the repo root) and the browser extension (browser-extension/) that actually intercepts and scans prompts and uploads on AI chat sites — the same components used across every SafePrompt edition, published together so the piece that sees your traffic is just as inspectable as the engine behind it. Both are licensed under Apache 2.0: use them, modify them, self-host them, or build on them, commercially or not — see License below.
Features
- On-device detection — API keys & credentials, PII (names…
Top comments (0)