MaskAgent is a privacy-first browser agent designed to perform
intelligent browser automation while keeping sensitive user data
protected on the device.
Smart India Hackathon 2026 · Problem Statement SIH26171
🔗 Project: https://bhuvanesh-m-dev.github.io/maskagent/
🔗 GitHub: https://github.com/bhuvanesh-m-dev/maskagent
▶️ Demo: https://youtube.com/watch?v=_rl1w6US6_U
The Problem: AI Browser Agents Can See Too Much
AI browser agents are becoming capable of understanding webpages and
performing tasks such as clicking buttons, entering information,
scrolling, and navigating websites.
But there is a privacy problem.
A webpage may contain:
- Names and addresses
- Email addresses and phone numbers
- Account information
- Financial information
- Personal identifiers
- Other confidential information
If an AI system receives the raw webpage or screen content, sensitive
information may cross the privacy boundary before the agent can reason
about the task.
MaskAgent approaches this problem differently:
Protect the sensitive information before AI reasoning begins.
What Is MaskAgent?
MaskAgent is a privacy-first browser agent that combines browser
automation, DOM analysis, visual perception, sensitive-data detection,
local masking, and AI reasoning.
Its core principle is:
Perceive Locally → Protect Locally → Reason Intelligently
Instead of directly sending raw browser content to an AI model,
MaskAgent first analyzes the webpage locally.
Browser Screen
↓
DOM + Visual Perception
↓
Sensitive Data Detection
↓
Local Masking / Redaction
↓
Sanitized Context
↓
AI / LLM / VLM Reasoning
↓
Browser Action
How MaskAgent Works
1. Perceive Locally
MaskAgent analyzes the browser environment using webpage structure and
visual information.
It can combine:
- DOM information
- Visual webpage information
- UI elements
- Text regions
- Form fields
- Page structure
2. Protect Locally
Sensitive information is detected and masked before it becomes part of
the AI reasoning context.
The goal is:
Raw sensitive data stays local.
3. Reason Intelligently
After sanitization, the AI receives a safer representation of the
webpage.
The agent can then reason about the task and determine the required
browser action.
4. Execute Locally
The browser agent can perform actions such as:
- Click
- Type
- Scroll
- Navigate
The Privacy Boundary
The most important architectural idea in MaskAgent is the privacy
boundary.
┌───────────────────────────────────────────────┐
│ LOCAL / PROTECTED │
│ │
│ Browser → DOM + Vision → Detection → Masking │
│ │
└──────────────────────┬────────────────────────┘
│
Sanitized Context
│
▼
┌───────────────────────────────────────────────┐
│ AI REASONING │
│ │
│ LLM / VLM → Browser Action │
│ │
└───────────────────────────────────────────────┘
The design principle is simple:
RAW SENSITIVE DATA STAYS LOCAL; ONLY SANITIZED CONTEXT REACHES AI.
DOM + Visual Perception
A major part of the planned MaskAgent architecture is combining
DOM-based understanding with visual perception.
DOM information can provide structured webpage information, while visual
perception can help understand what is actually displayed on the screen.
This combination can help MaskAgent detect sensitive regions more
precisely, especially on dynamic or visually complex webpages.
WEBPAGE
│
┌────────┴────────┐
↓ ↓
DOM Visual Perception
│ │
└────────┬────────┘
↓
Sensitive Data Detection
↓
Mask / Redact
Why On-Device Processing?
Privacy-sensitive information should ideally be protected as early as
possible.
MaskAgent therefore focuses on local processing for sensitive-data
detection and masking.
Potential benefits include:
- Reduced exposure of raw sensitive data
- Better privacy control
- Lower dependence on remote preprocessing
- Faster local decisions for supported workloads
- A clearer privacy boundary between perception and AI reasoning
The project also explores browser technologies such as WebGPU and
WebAssembly for lightweight local computation.
Technology Stack
Area Technology
Browser Extension Chrome / Firefox
Extension Standard Manifest V3
Languages JavaScript / TypeScript
Browser APIs Chrome Extension APIs
Local Acceleration WebGPU / WebAssembly
ML Runtime ONNX Runtime Web / Transformers.js
AI Lightweight Vision Models / LLM / VLM
Local AI Interface Ollama
Backend / Services FastAPI
Privacy Layer DOM + Visual PII Detection, Masking and Redaction
Current Prototype
MaskAgent is being developed as a browser extension prototype.
The current project explores:
- Browser-based AI interaction
- DOM analysis
- Sensitive-data detection
- Privacy masking
- Local AI integration
- Browser automation
- Click, Type, Scroll and Navigate actions
The project is being developed for Smart India Hackathon 2026, under
problem statement SIH26171 --- On-device Visual Perception for
Lightweight Browser Agents.
Example Privacy Flow
Imagine a webpage contains:
Name: Bhuvanesh M
Email: bhuvanesh@example.com
Account Number: 1234567890
Instead of directly giving the raw information to an AI reasoning
system, the privacy layer can transform sensitive fields into protected
representations.
For example:
Name: [PERSON]
Email: [EMAIL]
Account Number: [ACCOUNT_NUMBER]
The AI can reason about the structure and task while the original
sensitive values remain protected locally.
This is the core idea behind sanitized context.
Browser Automation
MaskAgent is designed around practical browser actions:
Click
↓
Type
↓
Scroll
↓
Navigate
The privacy layer sits before AI reasoning rather than being added after
the AI has already received the webpage.
Activity and Agent Flow
A browser agent needs to continuously understand what is happening on
the page and decide what to do next.
MaskAgent separates perception, protection, reasoning and execution.
┌──────────────┐
│ Browser │
└──────┬───────┘
↓
┌──────────────┐
│ Perception │
└──────┬───────┘
↓
┌──────────────┐
│ Privacy │
│ Protection │
└──────┬───────┘
↓
┌──────────────┐
│ AI Reasoning │
└──────┬───────┘
↓
┌──────────────┐
│ Browser │
│ Action │
└──────────────┘
Where Can This Be Useful?
A privacy-first browser agent can be relevant to workflows where
webpages contain confidential information.
Potential application areas include:
Banking and Finance
Protect account details, transaction information and other financial
data during browser automation.
Healthcare
Reduce exposure of sensitive patient or medical information during
automated workflows.
Government
Support browser automation for portals that contain citizen or
administrative information.
Enterprise
Provide privacy-aware automation for internal dashboards, business
systems and confidential workflows.
Education
Support automated browser tasks while reducing unnecessary exposure of
student or institutional information.
Challenges
Building a privacy-first browser agent introduces several technical
challenges.
Inference Latency
Local visual or AI inference can be computationally expensive.
Approach: lightweight models and browser acceleration technologies.
Detection Errors
Sensitive information may be missed or incorrectly detected.
Approach: combine DOM information with visual perception and improve
detection rules/models.
Over-Redaction
Masking too much information can reduce the usefulness of the AI
context.
Approach: precise region detection and semantic placeholders.
Dynamic Webpages
Webpages can change continuously.
Approach: continuous page analysis and efficient browser-side
processing.
Limited Resources
Browser-based AI must work within available CPU, memory and GPU
resources.
Approach: lightweight inference, WebGPU acceleration and efficient
processing pipelines.
Privacy by Design
MaskAgent is built around a privacy-by-design concept:
RAW PAGE
│
▼
┌─────────────────┐
│ Local Detection │
└────────┬────────┘
↓
┌─────────────────┐
│ Local Masking │
└────────┬────────┘
↓
SANITIZED DATA
│
▼
AI REASONING
The objective is not simply to tell an AI model to "ignore" private
information.
The objective is to reduce exposure before AI reasoning receives the
context.
What Makes MaskAgent Different?
The project focuses on the combination of:
- Browser automation
- On-device visual perception
- DOM understanding
- Sensitive-data detection
- Local privacy masking
- Sanitized AI context
- AI-based browser reasoning
The central design idea is:
Privacy should be part of the browser agent's perception pipeline,
not an afterthought.
Project Architecture
At a high level:
Browser
│
├── DOM
│
└── Visual Screen
│
▼
Perception Layer
│
▼
Sensitive Data Detection
│
▼
Privacy / Masking Layer
│
▼
Sanitized Context
│
▼
AI Reasoning
│
▼
Browser Automation
Repository Structure
maskagent/
├── README.md
├── CONTRIBUTING.md
├── code_files/
│ ├── README.md
│ ├── background.js
│ ├── content.js
│ ├── manifest.json
│ ├── popup.html
│ ├── popup.js
│ ├── styles.css
│ └── icons/
├── img/
├── papers/
├── ppt_files/
├── index.html
├── developers.html
└── team.html
Try the Prototype
GitHub
https://github.com/bhuvanesh-m-dev/maskagent
Project Website
https://bhuvanesh-m-dev.github.io/maskagent
Demo
The demo explains the motivation behind MaskAgent and shows the working
prototype.
Research and References
The project explores technologies and documentation related to:
- Chrome Extensions
- Browser Extension APIs
- WebGPU
- WebAssembly
- Ollama
- Transformers.js
- ONNX Runtime Web
- FastAPI
- Lightweight AI models
- Browser automation
- Privacy-preserving AI systems
Useful Documentation
- Chrome Extensions: https://developer.chrome.com/docs/extensions
- Chrome Extensions API: https://developer.chrome.com/docs/extensions/reference
- WebGPU API: https://developer.mozilla.org/en-US/docs/Web/API/WebGPU_API
- WebAssembly: https://webassembly.org/
- Ollama: https://docs.ollama.com/
- Transformers.js: https://huggingface.co/docs/transformers.js/en/index
- FastAPI: https://fastapi.tiangolo.com/
Smart India Hackathon 2026
Project: MaskAgent --- Privacy-First Browser Agent
Problem Statement: SIH26171
Problem: On-device Visual Perception for Lightweight Browser Agents
Core Approach:
Perceive Locally → Protect Locally → Reason Intelligently
MaskAgent is being developed as a solution concept for privacy-aware
browser automation where sensitive screen information should be
protected before AI reasoning.
Roadmap
Future development areas include:
- Improved visual PII detection
- More precise bounding-box detection
- DOM + vision fusion
- Lightweight on-device models
- Better dynamic webpage handling
- Improved privacy masking
- More browser actions
- Improved local AI integration
- Performance optimization using WebGPU
- Broader browser compatibility
Limitations
MaskAgent is an active prototype and research-oriented project.
Detection accuracy, browser compatibility, inference speed and model
capabilities may vary depending on the webpage, device and selected AI
model.
The project should not be treated as a guarantee that all sensitive
information will always be detected or protected.
Team
Team DataSync
MaskAgent is being developed as part of the Smart India Hackathon 2026
project work.
Keywords
AI browser agent · privacy-first AI · browser automation ·
PII detection · on-device AI · local AI · Chrome extension ·
Manifest V3 · WebGPU · WebAssembly · visual perception ·
LLM browser agent · privacy-preserving AI · AI automation ·
sensitive data masking · DOM analysis
Final Thought
AI browser agents should not have to choose between intelligence and
privacy.
MaskAgent explores another approach:
Let the browser understand sensitive information locally, protect it
locally, and give the AI only the context it needs to reason.
Perceive Locally → Protect Locally → Reason Intelligently.
Connect With the Project
🌐 Website: https://bhuvanesh-m-dev.github.io/maskagent/\
💻 GitHub: https://github.com/bhuvanesh-m-dev/maskagent\
▶️ Demo: https://youtube.com/watch?v=_rl1w6US6_U
Top comments (0)