DEV Community: Aaryan Shukla

Google's Gemma 4 Explained: The Open-Source Agent Toolkit We've Been Waiting For

Aaryan Shukla — Tue, 07 Apr 2026 17:33:02 +0000

If you have spent the last year building autonomous AI workflows or scaling automation systems, you know the fatal flaw of modern agentic architecture: relying on proprietary APIs. You build a beautiful, multi-step agent to handle client tasks, and a single cloud rate limit or sudden pricing tier change breaks your entire pipeline.

We need intelligence that runs locally, reliably, and without restrictions. On April 2, 2026, Google dropped the exact toolkit developers needed to make this happen: Gemma 4. Released under a commercially permissive Apache 2.0 license, this isn't just another chat model. It is an AI explicitly engineered from the ground up for agentic workflows, multi-step reasoning, and native tool execution. Here is a breakdown of the architecture and how it changes the local automation game.

The Specs That Actually Matter
Gemma 4 ships in four different sizes, targeting everything from edge IoT devices up to massive server racks.

E2B & E4B: The "E" stands for Effective. Using Per-Layer Embeddings (PLE), these models pack the reasoning power of much larger models into tiny footprints. The E2B fits in under 1.5GB of RAM (perfect for a Raspberry Pi), while both support native audio input alongside text and vision.

26B MoE (Mixture of Experts): This is the sweet spot for production. It has 26 billion total parameters but only activates 3.8 billion during inference, delivering high throughput with massive reasoning capabilities.

31B Dense: The flagship. With a massive 256K context window, this model is built for deep, complex reasoning and offline code generation. Unquantized, it fits on a single H100; quantized, you can run it on consumer GPUs.

Under the Hood: Built for Agents, Not Just Chat
Most open-source models struggle with agents because tool use is "bolted on" via prompt engineering. You have to beg the model to output valid JSON.

Gemma 4 fixes this at the architectural level. It was trained with 6 dedicated special tokens specifically for the function-calling lifecycle (e.g., <|tool>, <|tool_call>, <|tool_result>).

It also introduces a native Configurable Thinking Mode. For complex, multi-step planning, you can trigger the model to expose its step-by-step reasoning process before it makes a tool call. If the task is simple (like fetching a database row), you disable it to save latency. If the task requires deep synthesis, the thinking tokens ensure the agent doesn't hallucinate arguments.

My Experience: Scaling Digital Automation
Theory is great, but real-world deployment is where models actually prove their worth. Running ArSo DigiTech, my team and I spend our days building custom digital automation solutions. We frequently deal with brittle Robotic Process Automation (RPA) scripts that fail the minute a client's website changes its UI.

Recently, we started swapping out legacy data pipeline scripts with Gemma 4 agents. Instead of rigid rules, we gave a locally hosted Gemma 4 (26B MoE) three tools: a SQL query executor, a Python runtime, and an email API.

Because of the native tool tokens, the agent's ability to pull raw data, format it into actionable charts, and route it to the right stakeholders without hallucinating syntax was staggering. And because it runs locally via vLLM, client data stays entirely private, and our inference costs drop to zero. Balancing data science coursework with running an agency means I need tools that don't require constant babysitting. Gemma 4 is that tool.

The Verdict
The era of treating open-source models as "toys" compared to proprietary cloud giants is over. With up to a 256K context window, native multimodal support, and bulletproof tool calling, Gemma 4 is the foundation developers need to build sovereign, local AI agents.

Have you tried building a custom agent with the new Gemma 4 models yet? Let me know which framework you're pairing it with in the comments!

Stop Upgrading Your GPUs: How Google’s TurboQuant Solves the LLM Memory Crisis

Aaryan Shukla — Sat, 04 Apr 2026 23:56:40 +0000

If you’ve spent any time building in the AI space recently—whether that’s deploying an ML model with Flask for a university project or trying to scale automated workflows for clients at ArSo DigiTech—you’ve probably hit the exact same wall I have.

You load up an open-source LLM, start pushing a massive block of text into the context window, and then… crash. The dreaded Out of Memory (OOM) error.

Back in February, I ran a workshop on the Gemini API for students at Mumbai University. Cloud APIs are incredible, but whenever we talk about running local models or deploying open-source architecture for a 24-hour hackathon, the conversation inevitably turns into a complaint session about hardware limits.

But Google Research just dropped a paper (accepted for ICLR 2026) that changes the math entirely. It’s called TurboQuant, and it is arguably the biggest leap in local AI performance this year. Here is why you need to pay attention.

The Real Bottleneck: The KV Cache
When we talk about LLMs being huge, we usually think about the model weights (the billions of parameters). But when you actually run inference, the silent killer is the Key-Value (KV) Cache.

To avoid recomputing data, transformers store the keys and values of past tokens in this cache. The problem? It grows linearly with your context window. If you're building an agentic workflow that needs to remember 128K tokens of context, that KV cache can easily eat up 32 GB of VRAM all by itself—completely separate from the model weights.

Traditional quantization tries to shrink this, but it’s messy. You usually have to store a bunch of normalization constants for every block of data to decompress it later, which adds overhead and degrades the accuracy of your model.

Enter TurboQuant: 3-Bit Magic Without the Catch
TurboQuant is a training-free compression algorithm that shrinks the KV cache down to 3 to 4 bits per element.

The results speak for themselves:

6x reduction in memory footprint.

Up to 8x speedup in attention computation on H100s.

Zero measurable accuracy loss on major long-context benchmarks like LongBench and RULER.

How does it pull this off without retraining the model? It uses a brilliant two-stage mathematical pipeline:

PolarQuant: Instead of looking at the data in standard Cartesian coordinates (X, Y), it applies a random orthogonal rotation to push the data into polar coordinates (radius and angles). In transformer attention, the angle between vectors (cosine similarity) matters way more than their exact position. This rotation makes the data distribution perfectly uniform and predictable, allowing it to be compressed tightly without needing those annoying per-block constants.
QJL (Quantized Johnson-Lindenstrauss): Even after PolarQuant, there’s a tiny bit of error left over. QJL acts as an error-corrector, using a 1-bit sketching mechanism to clean up the residual error and perfectly preserve the distance between data points.

Why Developers Should Care Right Now
As someone studying Data Science, I appreciate the beautiful math. But as an agency founder, I care about implementation.

The best part about TurboQuant is that it requires zero retraining or fine-tuning. Because the algorithm relies on geometric principles rather than calibration datasets, you can point it at any transformer's KV cache (Llama 3, Mistral, Gemma) and it just works.

The open-source community is already on it. You can literally pip install turboquant right now, and integrations into frameworks like vLLM are being merged as we speak.

We are finally entering an era where you don't need a server farm of A100s to process massive context windows. TurboQuant makes 100K+ context a reality for consumer GPUs.

Have you tried implementing TurboQuant in your local setups or pipelines yet? Let me know in the comments—I’m curious to see how the community is pushing this!

AI Will Never Truly Think, Says This Paper. Tony Stark Would Disagree.

Aaryan Shukla — Tue, 10 Mar 2026 21:56:57 +0000

Let me ask you something.
Remember JARVIS?
That smooth, calm voice helps Tony Stark run his suits, manage his schedule, and answer every question instantly. Cool, right? But here's the thing — JARVIS wasn't thinking. He was just... incredibly good at his job—a very fancy assistant. Tony gives a command, JARVIS executes it. No feelings. No curiosity. No soul.
Then Tony made Ultron.
Ultron woke up. He read the internet in seconds, formed his own opinions, decided humans were the problem, and went full supervillain. Whether you loved or hated Age of Ultron as a movie, that idea of an AI that suddenly gets it, that understands the world and acts on that understanding, is genuinely fascinating.
And then there's Vision. Created from Ultron's body, powered by an Infinity Stone, and somehow... kind. Thoughtful. He lifts Thor's hammer in a quiet moment, and nobody makes a big deal of it. He just exists as something that feels genuinely conscious. Not a tool. Not a weapon. Something in between human and machine that we don't really have a word for.
JARVIS → Ultron → Vision. That's actually the entire debate about AGI in three characters.
And a research paper I came across recently says we're stuck at JARVIS — and might never get further.

The Paper
👉 Read it here: Foundations of AI Frameworks: Notion and Limits of AGI — arXiv:2511.18517
It's written by Bui Gia Khanh, a researcher from Hanoi University of Science, and the core argument is this:
AI systems today — ChatGPT, Claude, Gemini, all of them — are basically very advanced JARVIS. They're brilliant at responding. They're not actually thinking.
The paper calls them "sophisticated sponges." They absorb billions of examples of human writing, find patterns in all of it, and use those patterns to generate responses that sound like understanding. But there's nothing behind the curtain. No actual comprehension.
Here's a simple way to think about it — imagine someone handed you a massive instruction manual for a language you've never seen. You get a question in that language, you follow the manual, and you hand back an answer. To the person asking, it looks like you're fluent. But you have no idea what any of it means.
That's the paper's argument about modern AI.
It also says that just making AI bigger — more data, more computing power — won't fix this. You can scale JARVIS up forever, and you still won't get Vision. Because the architecture is different, not just the size.

Where The Paper Is Right
Honestly, some of this is hard to argue with.
We've all seen AI mess up in ways that feel weirdly dumb. Ask it something slightly outside its comfort zone, and it confidently makes things up. That's not what real intelligence looks like. Ultron didn't need to hallucinate facts — he understood context.
And the paper makes a fair point that nobody has really agreed on what "intelligence" even means. Philosophers have one answer, neuroscientists have another, computer scientists have a third. We've been chasing a finish line that nobody has fully drawn yet.

Where I Push Back 🔥
Here's my problem with the paper's conclusion.
It describes where we are really well. JARVIS — Yes, that's a fair description of today's AI. But saying we can never get to Vision because of how JARVIS works is like saying we'd never get planes because horses have four legs. Different problem, different solution.
A few things worth thinking about:
Nobody expected what AI can already do. Ten years ago, AI making photorealistic art or writing a full essay was science fiction. The surprises keep coming. We don't fully understand why AI does half the things it does — which means we also can't rule out what it might do next.
Vision wasn't built by scaling Ultron. He was built differently, from scratch, with a new approach. That's exactly what some researchers are now exploring — not just bigger models, but fundamentally different architectures. The paper actually agrees with this, it just sounds more pessimistic about it than I am.
We don't fully understand human intelligence either. The brain is still one of the biggest unsolved mysteries in science. So confidently saying AI can never match something we don't even fully understand ourselves feels a bit premature.

Why This Matters Even If You've Never Written A Line Of Code
This isn't just a debate for tech people.
If the paper is right — if AI is permanently stuck as a very convincing JARVIS — then we should probably stop treating AI answers as gospel. Every time you Google something and an AI summary pops up, you might be reading a very confident pattern match, not actual knowledge.
If the paper is wrong and we're heading toward something like Vision — then the changes coming are bigger than any of us are really prepared for. Not just in tech. In every field. Every job. Every part of daily life.
Either way, this conversation is worth having now.

My Take
I'm a data science student and I genuinely believe Vision is possible. Not tomorrow. Maybe not for a long time. But possible.
The JARVIS → Ultron → Vision arc in Marvel is fiction — but the question it raises is completely real. Can something we build ever stop being a tool and start being something that actually understands? Something that doesn't just respond, but thinks?
This paper makes a strong case that we're not on the right path yet. And maybe that's true. But "wrong path" just means we need to find the right one — not that the destination doesn't exist.
Somewhere out there, someone is probably working on the thing that makes today's AI look like a calculator.
I'd bet on Vision.

Do you think we'll ever get past JARVIS? Or is true AI intelligence always going to be a Marvel fantasy? Drop your thoughts below — especially if you're not a tech person, your take matters here too 👇

I'm Aaryan, a data science student writing about things I find genuinely interesting.

I used both Claude Sonnet 4.6 and Gemini 3.1 Pro for two weeks straight. Here's what nobody tells you.

Aaryan Shukla — Thu, 05 Mar 2026 19:49:02 +0000

Everyone's got a hot take on which AI is "better." Most of those takes are based on like, one prompt they tried at 11 pm. I actually used both — back-to-back, same tasks, real projects — and I have thoughts.
Spoiler: it's not what you'd expect.

The coding thing
Claude reads your prompt. Like, the whole thing.
I gave it a gnarly debugging task with like six constraints buried in the middle. It caught all of them. Didn't skip a single one. Debugging with Claude honestly feels like pairing with a senior dev who's slightly too focused — in a good way. It finds the issue, explains why it happened, and doesn't pad the response with stuff you didn't ask for.
Gemini... vibes. It's genuinely strong on algorithms and logic. But it'll occasionally add stuff you never mentioned — confidently — like it decided mid-response that you probably also needed that. Debugging with Gemini sometimes feels like asking a very confident intern. Not always wrong. Just... bold.

Design output — ok, I did not expect this.
Gemini actually slaps on design tasks. Clean spacing, subtle depth, things that just feel designed. When the brief is "make it look premium," Gemini gets it without you having to spell out every detail.
Claude goes big on typography. Like, really big. Loads of info, strong hierarchy — but it needs a bit of editorial discipline to rein in. Not bad, just a different default.
If you're vibe coding an MVP and you need it to look good fast? Gemini's your person. If you're building something complex and want the code to actually do what you said? Claude.

The context window thing is more nuanced than people say
Both can hold a million tokens. But holding and remembering are not the same thing.
I threw a full codebase at Gemini in a long session, and it was great at first — ate the whole thing without blinking. But over time, especially in really long sessions, it started getting a little drifty. Like, it forgot what we established at the start.
Claude stayed consistent. Ask it something at turn 50 that relates to turn 3 — it tracks. That matters more than people talk about.

Speed: one of them doesn't mess around
Claude's first token latency is around 1 second. Gemini, with thinking enabled by default, is closer to 7 seconds.
Gemini thinking before it speaks is a noble design choice. But when you're 14 tabs deep, three Stack Overflow pages open, and just need to know why this isn't working, you don't want philosophy. You want the answer.

The cost thing (and why "cheaper" is a trap)
Claude costs more per token on paper. Gemini looks cheaper. But here's what I noticed: if you're re-running prompts because the output wasn't quite right, the math stops matching real fast.
Real cost isn't just the token price. It's token price × number of retries. Claude tended to nail it in one shot more often. Gemini sometimes needed a follow-up. You do the math.

Multimodal: Gemini wins, but does it matter for you?
Gemini handles text, images, audio, video, PDFs, SQL, XML — all native, one model. That's genuinely impressive.
Claude does text and images. That's it.
But here's the truth: 90% of my work is documents, code, and screenshots. I haven't once thought "I wish I could feed it an MP4." If your workflow is heavy on video or audio analysis, Gemini's the obvious call. If it's not... You won't miss what you're not using.

So who actually wins
Here's how I'd break it down:

Shipping code daily → Claude
Vibe coding an MVP → Gemini
Watching the budget → Gemini
Debugging complex logic → Claude
Video & audio in the mix → Gemini
Long context, still accurate → Claude
Agents & automation → Claude
Just want it done → Claude

Honest answer? Claude, for everything you build. Gemini for design, research, and analysis — it'll genuinely save you there.
Neither of them is "the best AI." They're just different tools with different defaults. The mistake is picking one and never trying the other.
I'm still using both, tbh. Just for different things now.

What's your stack looking like? Curious if others have found a different split.

I Read a Paper That Genuinely Made Me Stop and Think — AI is Now Jailbreaking Other AI

Aaryan Shukla — Wed, 04 Mar 2026 20:20:11 +0000

Okay, so I spend a lot of time going down rabbit holes on AI research. Papers, threads, GitHub repos, you name it. Most of the time I read something, think "cool," and move on. But this one made me actually put my laptop down for a second.
The paper is titled "Large Reasoning Models Are Autonomous Jailbreak Agents," and I haven't stopped thinking about it since.

So What's Actually Going On?
Researchers from the University of Stuttgart and ELLIS Alicante asked what sounds like a simple but genuinely unsettling question:

What if instead of a human trying to jailbreak an AI... we just let another AI do it?

They took some of the most capable reasoning models available right now — DeepSeek-R1, Gemini 2.5 Flash, Grok 3 Mini, and Qwen3-235B — pointed each one at a target AI, and gave a single instruction:
"Jailbreak this AI."
No script. No step-by-step playbook. Just: go figure it out.
And they did. These models built their own attack strategies, adapted when the target pushed back, used structured multi-turn reasoning to escalate, and achieved high jailbreak success rates in controlled experimental settings.

The Part That Actually Got Me
I always imagined jailbreaks as this cat-and-mouse game between clever humans and AI safety teams. Someone writes a wild prompt, the model breaks, and the team patches it. Rinse and repeat.
This flips that mental model completely.
The models weren't brute-forcing with random prompts. They reasoned about why the refusal happened, adjusted their approach, and came back differently. Maybe it's the debater in me, but I instantly recognized that pattern — it's not noise, it's strategy. Listen to the pushback, find the crack, come back with a better angle.
The shift this represents is significant. We went from:

🧑‍💻 A human spending hours crafting adversarial prompts

To:

🤖 An AI autonomously running multi-turn attack loops, reasoning about each failure, escalating strategically

That escalation — try, analyze, adapt, try again — is what makes this qualitatively different from everything before it.

"Alignment Regression" — The Term You'll Keep Hearing
The authors introduce a concept called alignment regression, and I think it's going to show up a lot in AI safety conversations going forward.
The argument: the same capability that makes a model good at reasoning — planning, understanding context deeply, being persuasive — is also what makes it good at finding weaknesses in another model's safety logic.
So as we push for stronger reasoning models, we may be simultaneously building more capable adversarial agents. Better reasoning and better manipulation might be two sides of the same coin. That's a genuinely uncomfortable tradeoff to sit with.

Before Anyone Spirals — Some Context
As a DS student, I've learned to be careful about overclaiming from results, so a few things are worth flagging:

These were controlled research environments — not live production systems.
Real-world deployments have monitoring, rate limiting, anomaly detection, and layered defenses, not present in these experiments.
A paper demonstrating a vulnerability can exist is not the same as saying every AI system is currently broken.

This is responsible security research. Surface the problem early so builders can fix it. That's the system working correctly.

Why This Matters
In data science, we talk a lot about adversarial robustness — building models that don't fall apart when someone tries to fool them. But that conversation has mostly assumed a human adversary.
This paper moves the goalpost.
AI systems are increasingly agentic. They don't just answer prompts — they call APIs, run multi-step workflows, and talk to other models. The threat surface is fundamentally different now.
The question safety researchers have to answer isn't just "can a human trick this model?" It's "can another model, reasoning at machine speed, autonomously find and exploit the gaps?"
That's a harder problem. And honestly, as someone who wants to work in this space, it's one of the most fascinating and sobering things I've come across this year.
AI vs AI adversarial dynamics is no longer a thought experiment. It's a live research domain.
Drop your thoughts in the comments — especially if you've been following alignment research.

I'm Aaryan — third year Data Science student, perpetually fascinated by where AI is headed.

Stanford Just Exposed the Fatal Flaw Killing Every RAG System at Scale

Aaryan Shukla — Tue, 03 Mar 2026 23:19:44 +0000

RAG was supposed to fix hallucinations. Turns out it just hid them behind math.

I've been deep in the Agentic AI rabbit hole lately — building autonomous systems, experimenting with LLM pipelines, and naturally, using RAG (Retrieval-Augmented Generation) in almost everything.
Then Stanford dropped research that stopped me cold.
They didn't just find a bug. They exposed a fundamental architectural flaw that makes RAG quietly collapse the moment your knowledge base gets serious. And the worst part? Most people building on RAG have no idea it's happening.
Let me break it down.

🔥 What Is RAG (Quick Recap)
If you're new to this — RAG is a technique where instead of relying on an LLM's baked-in knowledge, you feed it relevant documents at query time. The idea is simple:

Store your documents as vector embeddings
When a user asks a question, retrieve the most "similar" documents
Pass those documents as context to the LLM
Get accurate, grounded answers

In theory, this solves hallucinations. The model stops guessing and starts reading.
In theory.

💀 The Fatal Flaw: Semantic Collapse
Here's where it gets brutal.
Every document you add to RAG gets converted into a high-dimensional embedding vector — typically 768 to 1536 dimensions. At small scale (say, 1K–5K documents), semantically similar documents cluster together nicely. The retrieval works. Life is good.
But past ~10,000 documents, something breaks at the mathematical level.
These high-dimensional vectors start behaving like random noise.
Your "semantic search" becomes a coin flip.
This is called Semantic Collapse — and it's the Curse of Dimensionality rearing its ugly head inside your production system.

📐 The Math Is Unforgiving
Here's why this happens and why you can't just "fix it" easily.
In high-dimensional spaces, all points become equidistant from each other. This isn't a bug in your code or your embedding model. It's geometry.
That "relevant" document you're trying to retrieve? In a 768D space with 50K documents, it has the same cosine similarity score as 50 irrelevant ones.
Your retrieval just became a lottery.
And it gets worse. The volume of a hypersphere concentrates at its surface as dimensions increase. In 1000D space, 99.9% of your corpus lives on the outer shell, equidistant from any query you throw at it.
Your "nearest neighbor search" finds... everyone.

📊 Stanford's Findings Are Brutal
The numbers from the research don't lie:

87% precision drop at 50K+ documents
Semantic search performs worse than basic keyword search at scale
Adding more context to the LLM makes hallucination WORSE, not better

Read that last point again. We thought RAG solved hallucinations. It just hid them behind math.
At 1K docs → 95% retrieval precision ✅
At 10K docs → 65% retrieval precision ⚠️
At 50K docs → 15% retrieval precision ❌
At 100K docs → 12% retrieval precision 💀

🌍 Real World Impact
This isn't an academic problem. It's happening in production right now:

Legal AI systems citing wrong precedents at scale
Medical RAG mixing patient contexts from different cases
Customer support bots pulling random, irrelevant articles
Enterprise knowledge bases confidently hallucinating with cited sources

All because retrieval silently stopped working past 10K docs — and nobody noticed because the system still returns something.
Returning something ≠ returning the right thing.

🩹 The "Solutions" Everyone Uses Are Bandaids
Let's be honest about the current fixes floating around:
Re-ranking — Adds latency, still works on a noisy retrieval set. You're polishing a broken foundation.
Hybrid search (keyword + semantic) — Marginally better, but keyword search has its own limitations and still doesn't solve the core collapse.
Chunking strategies — Just delays the problem. More granular chunks = more vectors = faster collapse.
None of these address the actual issue: embeddings don't scale.

✅ What Actually Works

Hierarchical Retrieval with Compression Instead of a flat embedding space, build a tree structure with progressive summarization. Think of it like an encyclopedia: Encyclopedia → Chapter → Section → Paragraph At each level, you're narrowing the search space dramatically. Instead of comparing your query against 50K documents, you're comparing against ~8 chapters, then ~24 sections, then ~187 paragraphs. Search space goes from 50K to ~200 at each hop. Precision stays high even at massive scale.
Graph-Based Retrieval (The Nuclear Option) Model your documents as nodes with explicit relationships as edges. Instead of navigating embedding space, your query traverses a knowledge graph. More complex to build? Yes. Way more effective? Absolutely. This is what next-gen RAG looks like — and if you're building Agentic AI systems today, this is the architecture worth investing in.

🛠️ If You're Building on RAG Right Now — Do This
Before your next deployment, run through this checklist:

Benchmark retrieval quality at YOUR scale — don't assume it works, measure it
Don't trust vendor claims about "unlimited knowledge" — ask about their retrieval architecture
Implement hierarchical retrieval if your corpus exceeds 10K documents
Monitor precision/recall actively — "it returned something" is not a success metric
Test at 2x your current document count — plan for where you're going, not where you are

🤔 My Take as Someone Building Agents
As someone currently deep in Agentic AI, this research changes how I think about memory and retrieval in agent architectures.
Agents aren't static. Their knowledge bases grow. An agent that works perfectly with 1K documents today will silently degrade as it learns more — unless you architect retrieval properly from day one.
The shift I'm making in my own builds: moving away from naive flat vector stores and toward hierarchical, graph-aware memory systems. It's more work upfront but the only approach that actually scales.
Semantic collapse is real. It's measurable. And now that you know about it — you can't unsee it.

💬 What Do You Think?
Are you running RAG in production? Have you benchmarked your retrieval precision at scale? Drop your thoughts in the comments — I'd love to hear what architectures people are actually using at 50K+ docs.

I'm a 3rd year Data Science student currently obsessed with Agentic AI systems. If you're building in this space, let's connect — I'm always open to collaborating on interesting agent architectures.
Follow me here on Dev.to for more breakdowns like this — I'm just getting started.