Smashed a 2.9GB Memory Monster: On-Device AI Optimization to <400MB

#devchallenge #bugsmash #python #ai

When I started building Nirvana Browser, my goal was simple:

Create an offline, privacy-first AI browser where every security decision happens locally—without sending user data to the cloud.

The vision was exciting.

The implementation... wasn't.

The Problem

My first prototype relied on running PyTorch models directly inside the desktop application.

Although it worked, it introduced three major problems:

📦 Huge installer size — nearly 2.9GB
🧠 High RAM consumption during local inference
⚠️ Renderer freezes caused by unhandled exceptions during domain security evaluation

For a desktop browser, these issues were unacceptable.

Users expect applications to launch instantly, consume minimal resources, and remain responsive. Shipping a multi-gigabyte installer simply wasn't an option.

The Optimization Strategy

Rather than making incremental improvements, I redesigned the inference pipeline from the ground up

1. Replacing PyTorch with ONNX Runtime

The biggest optimization came from removing the heavy Python runtime from production.

Instead of bundling PyTorch, I converted the inference models to ONNX Runtime and optimized them for deployment.

import torch.onnx

def convert_to_onnx(model, dummy_input, export_path):
    torch.onnx.export(
        model,
        dummy_input,
        export_path,
        export_params=True,
        opset_version=14,
        do_constant_folding=True,
        input_names=["input"],
        output_names=["output"],
    )

After conversion, the models were further optimized through quantization to reduce memory usage and improve inference efficiency.

The result was a dramatically smaller deployment footprint and significantly faster startup time.

2. Replacing Large Domain Lists with a Smart Rule Engine

Initially, I experimented with maintaining a large database of malicious domains.

While functional, loading massive datasets into memory created unnecessary overhead.

Instead, I designed a lightweight pattern-based filtering engine inside safe.py.

Rather than storing every possible malicious domain, the engine evaluates:

Suspicious keyword patterns
High-risk domain structures
Content-based indicators

This allows the browser to identify potentially dangerous domains using constant-time lookup logic while keeping memory usage low.

Instead of scaling with dataset size, the filtering process remains lightweight and predictable.

3. Monitoring Runtime Stability with Sentry

Optimization isn't only about speed.

Reliability matters just as much.

To identify crashes and unexpected execution paths during testing, I integrated Sentry for runtime monitoring.

import sentry_sdk

sentry_sdk.init(
    dsn="YOUR_SENTRY_DSN",
    traces_sample_rate=1.0,
    profiles_sample_rate=1.0,
)

This helped surface latency spikes, renderer exceptions, and asynchronous failures that were difficult to reproduce locally.

Fixing these edge cases noticeably improved browser stability.

Key Takeaways

This project reinforced several engineering lessons.

Quantization Matters

Production applications don't always need heavyweight machine learning frameworks.

Optimized ONNX Runtime deployments can provide excellent performance while dramatically reducing application size.

Algorithms Beat Raw Data

A carefully designed rule engine can often outperform loading massive static datasets into memory.

Better algorithms frequently provide greater gains than simply adding more data.

Privacy Doesn't Have to Be Slow

Building an offline-first AI browser showed me that local inference can be both private and performant when the deployment architecture is designed carefully.

Final Thoughts

Performance optimization isn't about chasing benchmarks.

It's about removing unnecessary complexity until the software feels effortless to use.

Watching Nirvana Browser evolve from a bloated prototype into a lightweight desktop application has been one of the most rewarding engineering experiences of this project.

There are still many improvements ahead, but solving this optimization challenge fundamentally changed how I think about deploying AI on end-user devices.