DEV Community: Naing Oo

Gemma 4: What I Learned Running Google's Open AI Model on Real Hardware

Naing Oo — Sat, 23 May 2026 00:46:38 +0000

This is a submission for the Gemma 4 Challenge: Write About Gemma 4

Most AI tutorials show you how to call an API. You send text in, you get text back, and everything works perfectly in a Jupyter notebook.

Real deployments are messier. And the mess is where you actually learn something.

I've been running Gemma 4 on an HPC (High-Performance Computing) cluster for the past few weeks — the kind of environment where you submit jobs to a queue, share GPUs with other researchers, and debug library errors at 11pm. Here's what I wish someone had told me before I started.

First: What Even Is Gemma 4?

Gemma 4 is Google's latest family of open-weight language models. "Open-weight" means you can download and run the model yourself — no API key, no usage fees, no data leaving your machine.

The family includes several variants, but the two most interesting are:

Gemma 4 E4B — A "Mixture of Experts" model. Think of it as a large model that only activates a small part of itself for each word it generates. Clever architecture, ~15GB to load.
Gemma 4 27B — A traditional "dense" model. All 27 billion parameters work together every time. Much more memory-hungry, but predictable.

There are also smaller 4B and 12B dense versions. For most developers, these are the starting point.

The "Mixture of Experts" Thing, Explained Simply

You'll see MoE (Mixture of Experts) mentioned a lot with Gemma 4. Here's what it actually means.

A normal language model processes every word through all of its parameters, every single time. A MoE model has multiple "expert" sub-networks, and for each word, it only activates a few of them — the most relevant ones.

The promise: You get the capacity of a big model with the compute cost of a small one.

The catch: The entire model — all experts — still needs to fit in your GPU's memory, even though only some of them run at any given moment.

In practice, on a 20GB GPU slice, I measured:

Model	Speed
Gemma 4 E4B (MoE)	~3–4 words/second
Gemma 3 4B dense	~10–11 words/second

The dense model was nearly 3× faster in my constrained setup. The MoE model's routing overhead and larger memory footprint more than offset its theoretical efficiency gains when you're tight on VRAM.

This doesn't mean MoE is bad — it means it needs room to breathe. On a full A100 80GB GPU or multiple GPUs, the story flips.

Which Variant Should You Pick?

Here's a simple guide:

Gemma 4 4B dense — Start here. Runs on a gaming GPU (RTX 3080/4080), fast, capable, easy to experiment with.

Gemma 4 E4B (MoE) — Pick this when you have a large GPU (40GB+) and need multimodal support (text + images). Don't pick it just because "MoE" sounds exciting.

Gemma 4 27B — When quality matters more than speed and you have a serious GPU. Research, complex reasoning tasks, high-stakes extraction.

The 128K Context Window: Why It Actually Matters

Every model has a "context window" — the maximum amount of text it can read at once. Older models had 4,000–8,000 tokens (roughly 3,000–6,000 words). Gemma 4 supports 128,000 tokens.

That's not just a bigger number. It changes what you can build.

What becomes possible with 128K context:

Read whole documents. Instead of chopping a long PDF into chunks and hoping the right chunk gets retrieved, you can feed the entire document in. No information lost between sections.

Longer conversations with memory. You can keep the full history of a conversation in context instead of needing a database to "remember" what was said earlier.

Complex multi-step tasks. Automated agents that reason across many steps need space to store their thinking. At 4K context, they run out of room. At 128K, they can work much longer.

The practical limit is memory — filling 128K context uses a lot of GPU RAM. But even using 30–40K when you need it (rather than being capped at 4K) is a real quality-of-life improvement.

Images Too: Gemma 4's Multimodal Capability

The E4B and 27B variants can understand images as well as text. You send a photo alongside your question, and the model can describe it, extract information from it, answer questions about it.

What this looks like in code:

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}},
            {"type": "text", "text": "What does this document say about payment terms?"}
        ]
    }
]

Why this is a bigger deal than it sounds:

Before multimodal models were locally runnable, if you wanted to extract data from a scanned form, you needed: an OCR tool, a parser, and then a language model. That's three separate systems to maintain.

Now: one model, one call.

More importantly — it runs on your hardware. Documents that contain sensitive information (medical records, financial statements, legal contracts) never leave your control.

Running It on an HPC Cluster: The Honest Account

If you're deploying Gemma 4 in a university or corporate HPC environment, here are the things that will go wrong and how to fix them.

Problem 1: Missing CUDA library

The first error you'll almost certainly hit:

OSError: libcusparseLt.so.0: cannot open shared object file

PyTorch ships with this library under a versioned filename, but then looks for the un-versioned version. Create a symlink:

cd $CONDA_PREFIX/lib/python3.x/site-packages/torch/lib
ln -sf libcusparseLt-f80c68d1.so.0 libcusparseLt.so.0

And add to your SLURM job script:

export LD_LIBRARY_PATH=$CONDA_PREFIX/lib/python3.x/site-packages/torch/lib:$LD_LIBRARY_PATH

Do this once, before anything else.

Problem 2: Two requests hitting the model at the same time

When you serve a model via a web API, multiple users can send requests simultaneously. Two calls to model.generate() at the same time on a single GPU will either crash or corrupt each other.

The fix is a simple lock — one request runs at a time:

import threading

_MODEL_LOCK = threading.Lock()

def run_model(prompt):
    with _MODEL_LOCK:
        # Only one thread can be here at a time
        output = model.generate(...)
    return output

Not fancy. Not fast. But correct.

Problem 3: The system prompt being silently ignored

Gemma uses a specific message format. If you format your system message like this:

# This looks right but doesn't work properly
{"role": "system", "content": "You are a helpful assistant."}

The model may not follow it reliably. The correct format wraps the content in a list:

# This is what Gemma actually expects
{
    "role": "system",
    "content": [{"type": "text", "text": "You are a helpful assistant."}]
}

Use tokenizer.apply_chat_template() to avoid doing this manually:

formatted = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

Getting Gemma to Extract Structured Data Reliably

One of the most useful things you can do with any instruction-tuned model is ask it to read unstructured text and return structured JSON. Gemma 4 does this well — but the quality depends heavily on how you write your instructions.

Vague instruction → unreliable output:

Extract the person's work experience as JSON.

Specific instruction → reliable output:

Extract the object's attributes as JSON.
Rules:
- Only include attributes listed under "Colors"
- Do NOT include defects from the shape section
- Calculate total years: sum all months worked, divide by 12, round to 1 decimal
  Example: 6 + 24 + 18 = 48 months = 4.0 years
- Return ONLY valid JSON. No markdown fences, no explanation.

The two changes that matter most:

Show example arithmetic — models are bad at mental math. Showing the steps ("6 + 24 + 18 = 48") dramatically improves accuracy.
Name what NOT to do — counter-examples work better than positive examples alone. "Do NOT include" is often clearer than "Only include."

Why Run Locally at All?

Fair question. Calling GPT-4o or Claude via API is easier. Why go through all of this?

Your data stays yours. If you're processing CVs, medical notes, legal documents, or anything confidential — locally-run means zero data ever leaves your system. For regulated industries, this isn't optional.

No per-call costs at scale. API pricing adds up. Once your model is running, inference is just electricity.

No rate limits. No waiting in a shared queue. Your batch job runs when you tell it to.

The model doesn't change under you. API providers update their models silently. A locally-downloaded checkpoint stays exactly as it was.

You can customise it. Fine-tuning, adding adapters, changing the tokeniser — all possible locally. Most APIs don't expose this.

For a specific, well-defined task — document understanding, structured extraction, domain-specific Q&A — Gemma 4 running locally is genuinely competitive with frontier APIs. It won't win every benchmark. But for the use cases where data control and cost matter, the gap in quality is smaller than the gap in control.

Quick Reference: Things to Know Before You Start

Topic	Key Point
Which model to start with	4B dense — works on a single consumer GPU
MoE (E4B)	Needs 40GB+ GPU to outperform dense; multimodal support
27B dense	Best quality, needs serious hardware
Context window	128K tokens — use it for whole documents and long tasks
Images	E4B and 27B support image input out of the box
CUDA error on startup	Create symlink for `libcusparseLt.so.0`
Concurrent requests	Use a threading lock — one generate call at a time
System prompt format	Wrap content in a list: `[{"type": "text", "text": "..."}]`
Structured JSON extraction	Use explicit rules + counter-examples + show arithmetic

Gemma 4 is the clearest sign yet that frontier-quality open models are no longer a future promise. They're here, they run on hardware you can actually get, and the reasons to use them — privacy, cost, control — are only getting stronger.

The friction of local deployment is real. But it's mostly one-time setup friction, not ongoing complexity. Once the symlinks are in place, the lock is written, and the chat template is correct, it just runs.

I launched Two Mirrors on Playstore

Naing Oo — Sat, 25 Apr 2026 22:51:02 +0000

I wanted a tool that would allow me to fully utilize the cameras of my smartphones as mirrors. By utilizing Wifi direct as the underlying technology, I built Two Mirrors on Android. The way it works is, one smart phone acts as the host and the other connects to it. Once the pairing succeeds, the camera feed is reflected on both phones.

As a bonus, one can also use the Webcam feature. For that, tailscale can be used to put the PC and the smart phone on the same network and a Python script available on Github can be run to connect to the target smartphone's tailscale IP.

App Link Two Mirrors

Python script for webcam Two Mirrors

How many of you agree with this?

Naing Oo — Fri, 10 Apr 2026 05:38:53 +0000

7 days ago, Garrett Rose posted a video of his struggles in getting hired as a SWE. It is a really sincere video where he clearly articulated what many in the industry are facing currently.

The threat of being replaced by AI
Scam job posts
Companies bullying the candidate with at least 4 rounds of interviews followed by a ridiculous technical test.
Ghosting and much more.

Check it out and let us kick off this discussion because sooner or later, it is going to affect you and me.

Silent Flames

Naing Oo — Wed, 25 Mar 2026 00:44:12 +0000

This is a submission for the 2026 WeCoded Challenge: Frontend Art

Art

Silent Flames - For the voices behind the shadows

Inspiration

Silent Flames was inspired by the millions of women — mothers, wives, caregivers — whose daily sacrifices form the invisible foundation of the world around them. They iron the shirts behind the promotions, stay awake so others can rest, and choose silence over applause — not because they are lesser, but because they are the architecture of someone else's sky.

The meaning I hope to convey is simple: recognition. Not with grand gestures, but with the same quiet sincerity these women bring to everything they do. You are the silent flame — and the world is warm because of you.

My Code

Github Link

AnimateLI5: Animation Software for a 5 year old

Naing Oo — Tue, 24 Mar 2026 20:02:04 +0000

The Problem

My niece wanted to make animations. I tried a dozen apps — they all had tiny buttons, complex menus, text-heavy interfaces, and features designed for adults. A 5-year-old shouldn't need to read a tooltip to draw a star.

So I built one from scratch.

Meet AnimateLI5

AnimateLI5 (Animate Like I'm 5) is a free, open-source Windows desktop app that lets very young children create frame-by-frame animations.

The design philosophy is simple: if a child can't figure it out in 5 seconds, it's too complicated.

No Reading Required

Every single feature works through icons, colors, and spatial cues. There are no menus to navigate, no dialogs to read, no settings to configure. Just big colorful buttons that do obvious things.

What Can Kids Do With It?

Draw shapes — tap a shape button, tap the canvas. 10+ shapes including stars, triangles, and hexagons
Freehand drawing — brush tool with adjustable size and 9 color choices
Move, resize, rotate — drag handles on any element
Erase mistakes — circle eraser removes elements under the cursor
Build animations — add frames, duplicate frames, reorder them in a visual timeline strip
Watch it play — hit the big green play button, adjust speed from 1–30 FPS
Import stickers — drag & drop any image as a stamp
Undo everything — every action is undoable, because kids experiment fearlessly
Export to video — render to MP4 with a progress bar

The Tech Stack


Framework	WPF on .NET 8
Language	C# 12
Architecture	MVVM with CommunityToolkit.Mvvm
DI	Microsoft.Extensions.DependencyInjection
Video Export	ffmpeg (external)
Tests	xUnit

Architecture Decisions

Why WPF in 2026?

I know, I know. But for a Windows-only kids tool:

Hardware-accelerated vector rendering out of the box
Mature data binding (MVVM just works)
Rich input handling (mouse, touch, stylus)
Single self-contained EXE — no browser, no Electron, no 200MB runtime

Why MVVM with Source Generators?

CommunityToolkit.Mvvm's source generators eliminate boilerplate. Instead of 15 lines of INotifyPropertyChanged plumbing:

[ObservableProperty]
private string _selectedColor = "#FF4081";

[RelayCommand]
private void SelectColor(string color) => SelectedColor = color;

Two attributes. Done. The generator handles the rest.

Plugin System

The solution is split into 3 projects:

SimpleAnimate.Core    → Models, interfaces, services (zero WPF refs)
SimpleAnimate.Plugins → IPlugin contract + loader
SimpleAnimate         → Thin WPF shell

Plugins reference only Core and implement IPlugin.Configure(IServiceCollection). Drop a DLL in the plugins/ folder and it's loaded at startup. This keeps the door open for community extensions without touching the main app.

Undo Everything

Every canvas mutation creates an IUndoableAction:

var action = new MoveAction(element, oldX, oldY, newX, newY);
_undoService.Execute(action);
// Ctrl+Z → action.Undo() restores the old position

Classic dual-stack (undo/redo). Kids mash buttons — undo must always be there.

UX Decisions Worth Noting

Color, not text — the toolbar uses colored backgrounds to group tools. Shapes are on colored buttons matching the app's palette. No labels.

Frame timeline is visual — frame thumbnails at the bottom, not a number list. Tap to edit, big + button to add.

FPS control is just ➖ and ➕ — no slider, no textbox. Minus and plus buttons with a number between them.

Video Export

Export renders each frame to PNG via WPF's RenderTargetBitmap, then pipes them through ffmpeg to produce an MP4. A progress window shows real-time status by parsing ffmpeg's frame=N output with regex.

The app auto-discovers ffmpeg from PATH or common install locations. No ffmpeg? Everything else still works — you just can't export video.

Try It

Download: AnimateLI5 v1.0.0 (Windows, 64-bit, self-contained — no .NET install needed)

Source: github.com/n-92/AnimateLI5

Demo Video: Watch on YouTube

What I Learned

Designing for kids is harder than designing for developers. You can't fall back on text, tooltips, or documentation. Every affordance must be visual and immediate.
Undo isn't optional — it's the most important feature. Kids explore by clicking everything. Without undo, one wrong click means tears.
WPF is still perfectly fine for desktop tools. The ecosystem is mature, the tooling works, and a self-contained single-file publish produces a clean distributable.
Source generators changed C# MVVM. CommunityToolkit.Mvvm eliminates the tedium that made MVVM feel heavy. Two attributes replace dozens of boilerplate lines.

AnimateLI5 is MIT licensed. Contributions welcome — especially if you have ideas for making it even simpler.

If you've got a young kid who likes to draw and animate, give it a try and let me know how it goes. 🎨

Nanoshot has officially replaced Greenshot on my PC

Naing Oo — Thu, 19 Mar 2026 02:22:50 +0000

This is a submission for the 2026 WeCoded Challenge: Echoes of Experience

🚀 Why I Finally Ditched Greenshot for a Tool I Built Myself: Meet NanoShot!

For years, Picpick and Greenshot were my go-to screenshot tools. They were reliable, familiar, and got the job done. But as developers, we constantly crave tools that are faster, leaner, and perfectly tailored to our specific workflows. I wanted something that felt invisible until the exact moment I needed it—without the bloat.

So, I decided to stop searching and start coding.

Today, I’m thrilled to announce that my own creation, NanoShot, has officially replaced Greenshot as my primary daily driver! 🎉

🛠️ What is NanoShot?

NanoShot is a blazing-fast, system-tray-resident screenshot and editor tool for Windows. I wrote it in pure C (C99) utilizing the Win32 API.

The result? An incredibly lightweight executable that clocks in at ~50 KB with zero heavy external dependencies. It’s lightning fast, brutally efficient, and permanently ready.

Samples

✨ Why It Won Me Over (The Unique Features)

I didn't just want to clone existing tools; I wanted to optimize my specific workflow and add a few superpowers along the way:

🪶 Unbelievably Lightweight: At roughly 50 KB, it uses practically zero system resources while running silently in your system tray.
🕵️ Built-in Privacy Protection: Capturing screenshots often means accidentally grabbing sensitive data. NanoShot features a built-in Editor Canvas that instantly lets you draw rectangles to pixelate/blur credentials, faces, or sensitive text before copying.
🖼️ Sobel Edge Detection: This is my favorite unique feature! By hitting Ctrl + Shift + PrintScreen, you can capture a screen region with real-time Sobel edge detection applied. Perfect for extracting UI layouts, highlighting borders, or just getting a creative, stylized structural view of your screen.
⚡ Frictionless Workflow: Hitting Ctrl + PrintScreen freezes the screen for capture. Once you're done editing/blurring, hitting Enter, Space, or simply double-clicking the image instantly sends it to your clipboard.
⚙️ Zero-Config Auto-Start: Run it once, and it automatically registers itself in the Windows startup registry. It’s always there, always ready, and requires absolutely zero maintenance.

💡 The Takeaway

Building NanoShot reminded me of why I love software engineering. Sometimes the best way to solve a minor daily annoyance is to engineer a bespoke solution from the ground up. It’s an incredibly rewarding feeling to use a tool hundreds of times a day and know exactly how every single byte of it works under the hood.

If you love minimalist, high-performance utilities and want an insanely fast screenshot tool for Windows, I invite you to give it a spin!

👉 Check out the NanoShot Repository on GitHub

Have you ever replaced a daily-driver application with something you built yourself? Let me know in the comments!

Meet NanoPdf Editor

Naing Oo — Thu, 19 Mar 2026 02:04:01 +0000

Do you still remember the good old days around the year 2007 when you didn't have to worry about subscription and cloud mess for desktop based applications? Those good old days where desktop based applications just acted like they were meant to work without the internet. I still remember being able to edit PDF files directly using the free version of Adobe. Since then, things have changed and it is all about subscriptions and dependencies towards the ecosystems of the giants.

I had been sitting down writing my own little pdf editor for Windows which I humbly call NanoPDF editor. It does one thing only and it does it really well -to edit PDFs. It has worked well for me so far with a wide range of PDF files that I have tested and I would humbly like to sell this software for just 10 USD for a life time license. You make the payment and I will send you the download link within 24hrs. It is that simple.

A demo can be found on youtube of how it works.

NanoPdf Editor

Purchase Link

NanoShot

Naing Oo — Sat, 07 Mar 2026 14:55:17 +0000

A lightweight, system-tray-resident screenshot & editor tool for Windows, written in pure C with the Win32 API.

Samples

v1.0.0 Download

Disk Octopus: AI-Powered Terminal Storage Analytics Built with GitHub Copilot CLI

Naing Oo — Tue, 03 Feb 2026 19:03:30 +0000

This is a submission for the GitHub Copilot CLI Challenge

What I Built

Disk Octopus - Intelligent Storage Analytics powered by AI

I built a professional-grade terminal application that transforms how users analyze and understand their disk storage. Disk Octopus combines sophisticated file system visualization with AI-powered analysis to provide deep insights into storage usage patterns.

Key Features Built:

🖥️ Interactive Terminal UI

Split-panel interface with resizable sections using Textual framework
Beautiful braille octopus ASCII art for visual appeal
Real-time file tree with intelligent lazy-loading
Status bar with progress tracking

📊 Advanced File Analysis

Recognition and categorization of 40+ file types (code, documents, media, config, specialized formats)
Real-time file type statistics and storage distribution analysis
File path indexing and searchable results
Intelligent file complexity classification

🤖 AI-Powered Deep Analysis

Integration with GitHub Copilot CLI for enhanced file analysis
Intelligent fallback analyzer using pattern-based analysis when Copilot unavailable
Analysis of file purpose, usage context, creator information, and practical assessments
Support for files up to 5MB with streaming analysis

⚡ Performance Optimizations

Lazy-loading of directory trees for instant responsiveness
Intelligent caching of analysis results
Efficient terminal rendering with minimal CPU usage
Async file scanning with real-time progress

Technical Architecture:

Core: Python 3.10+ with Textual TUI framework
UI: Rich terminal formatting with CSS-based styling
Analysis: Pattern-based intelligent analyzer + Copilot CLI integration
File System: Recursive directory scanning with size tracking
Data Processing: Real-time statistics aggregation

Why I Built This:

Traditional disk analysis tools like WizTree are GUI-only and resource-intensive. Disk Octopus brings powerful storage analytics directly to the terminal - where developers work. The AI integration transforms generic file analysis into actionable insights about file purposes, storage patterns, and disk organization opportunities.

Demo

Live Repository: https://github.com/n-92/Disk-Octopus

Screenshots:

Splash Screen - Welcoming interface with octopus branding

Main Analysis Interface - Comprehensive disk visualization

File Statistics - Real-time storage distribution analysis

Copilot-Powered Analysis - AI-generated file insights

Deep Analysis Results - Intelligent file categorization and assessment

File Paths Display - Complete file indexing and organization

Quick Start:

# Clone and setup
git clone https://github.com/n-92/Disk-Octopus.git
cd Disk-Octopus
pip install -r requirements.txt

# Run
python main.py

# Navigate with arrow keys, press 'd' for deep analysis, 'q' to quit

Key Keyboard Shortcuts:

q - Quit
h - Help menu
d - Deep file analysis with Copilot
a - Quick file type analysis
s - Show statistics
↑/↓ - Navigate

My Experience with GitHub Copilot CLI

How I Used Copilot CLI:

1. Deep File Analysis Integration
I integrated GitHub Copilot CLI to analyze file contents and generate intelligent commentary. The challenge was that subprocess calls with complex prompts would hang on Windows. I solved this by implementing an intelligent fallback analyzer that uses pattern-based analysis, while still leveraging Copilot when available.

2. Problem-Solving for Complex Issues
When deep analysis wasn't working (files showing "Uploading... Please Wait" indefinitely), I used Copilot to help debug the subprocess hanging issue. This led me to discover PowerShell argument parsing limitations and implement a robust fallback mechanism.

3. Development Acceleration
Copilot CLI helped me:

Generate analysis frameworks for file categorization
Create intelligent fallback logic for when Copilot is unavailable
Develop comprehensive file type recognition across 40+ formats
Debug threading and async issues with terminal UI updates

Impact on Development Experience:

Positive Impacts:

✅ Reduced debugging time for complex subprocess issues
✅ Accelerated implementation of file type detection system
✅ Helped design the intelligent fallback analyzer architecture
✅ Provided pattern suggestions for file analysis frameworks
✅ Enhanced code quality through intelligent suggestions

Technical Achievements:

Built a production-ready terminal application with advanced features
Implemented robust error handling and fallback mechanisms
Created 40+ file type recognition system
Achieved seamless Copilot integration with graceful degradation
Delivered comprehensive documentation and user guides

Key Learning:
The biggest insight was designing systems that gracefully degrade when external services (like Copilot CLI) are unavailable. This approach made the application more robust and user-friendly, as it works perfectly even without Copilot while providing enhanced features when available.

Copilot CLI Integration Details:

The application automatically:

Detects if Copilot CLI is installed
Attempts analysis via Copilot when available
Falls back to intelligent pattern-based analysis if not
Shows users which analysis method was used
Provides meaningful insights either way

This design philosophy made Disk Octopus inclusive for all users, regardless of their GitHub Copilot subscription status.

Project Highlights

Professional Quality: Production-ready code with clean architecture
Cross-Platform: Works on Windows, macOS, and Linux
Well-Documented: Comprehensive README with screenshots and setup guides
User-Friendly: Intuitive keyboard navigation and clear help system
Open Source: MIT licensed, ready for community contributions
Robust: Handles large directories and files gracefully

Version: 1.0.0

Author: N-92

Technologies: Python 3.10+, Textual, Rich, GitHub Copilot CLI

Repository: https://github.com/n-92/Disk-Octopus

Special Thanks: GitHub Copilot CLI for intelligent code assistance and analysis acceleration.