Gemma 4: Google’s new open AI model family
Gemma 4 is Google DeepMind’s latest family of open-weight AI models, purpose-built for advanced reasoning, long context, and multimodal “agentic” workflows. Built on technology from Gemini 3, these models offer unprecedented “intelligence-per-parameter” and are released under a permissive Apache 2.0 license. In practice, Gemma 4 supports text, images, and even audio input (on its smaller models) and excels at tasks like math reasoning, planning, code generation, and tool use. Crucially, Google provides open weights and code: Gemma 4 models can be downloaded from Hugging Face or Kaggle for local use. This makes them freely tunable and deployable in your own applications (even for commercial use), from on-device agents to cloud services.
Open and accessible: All Gemma 4 models have open weights and an Apache 2.0 license, meaning developers can use, modify, and commercialize them freely.
Multimodal and multilingual: They natively handle text, images, and video (all models) and audio (E2B/E4B models). They support over 140 languages and can process large documents (up to 128K–256K tokens in one prompt).
Agentic and tool use: Gemma 4 has built-in support for function calling and structured JSON output, enabling it to be the brain of autonomous agents that call APIs, fill forms, or control applications.
Advanced reasoning: These models outperform previous open models on logic and coding benchmarks; for example, the 31B version ranks #3 on Arena’s global Open LLM leaderboard, and the 26B Mixture-of-Experts (MoE) ranks #6, beating models 20× larger.
Long context: The two smaller models (Effective 2B and 4B) offer a 128K token context window, while the larger 26B and 31B models handle up to 256K tokens. This lets the model “see” entire books, codebases, or video scripts in one pass.
Speed and efficiency: Every Gemma 4 model includes a special multi-token draft feature for speculative decoding, which boosts inference speed without losing accuracy. The MoE design on the 26B model activates only 3.8B of its 26B total parameters at a time, giving near-4B speed with 26B-level smarts. Meanwhile, the E2B/E4B models are optimized to run offline on phones and edge devices with near-zero latency.
Model sizes and architectures
Gemma 4 comes in four sizes tailored to different hardware and uses:
1. E2B (effective 2B): ~2.3B effective parameters (5.1B including large embeddings), 128K context, image+audio+text support. Ultra-efficient for phones and IoT (e.g. Android devices, Raspberry Pi, Jetson Nano).
2. E4B (effective 4B): ~4.5B effective (8B with embeddings), 128K context, image+audio+text. Also mobile/edge-optimized with a bit more capacity.
3. 26B A4B (26B MoE): Mixture-of-Experts model with 26B total parameters but only ~4B active per token, 256K context. Offers very fast generation (comparable to 4B models) with high reasoning power.
4. 31B (31B Dense): 31B fully-dense parameters, 256K context. Highest raw performance for coding and complex tasks; fits on a single high-end GPU (e.g. 80GB H100) in 16-bit form.
All models use an efficient transformer architecture: local sliding-window attention (512 tokens for small models, 1024 for large) combined with some global context layers. They incorporate tricks like dual RoPE positional encodings, per-layer embeddings (PLE) in the small models, and a shared KV cache to save memory. In short, Gemma 4 is designed to maximize the “bang for your buck” for each model size.
Performance and benchmarks
Despite being open and relatively compact, Gemma 4 sets new performance records for open models. In Google’s testing, the 31B Gemma 4 is the third-highest-scoring open model in Arena’s “chat” benchmark (text-only, April 2026), and the 26B A4B is #6. These models beat or match much larger closed models on reasoning, math, science QA, coding, and multilingual tasks. For example, Gemma 4 scores above 88% on scientific Q&A and 89% on math benchmarks (AIME 2026), vastly outperforming older models of similar size.
Efficiency is also state-of-the-art. Google reports that a 31B Gemma 4 in full 16-bit precision runs on one NVIDIA H100 GPU, while quantized versions (4–8 bit) can run on standard consumer GPUs or even on-device. The 26B MoE notably only activates ~3.8B parameters at once, giving very high tokens-per-second. The small E2B/E4B models run entirely offline on phones, Raspberry Pi, and even Jetson devices, processing vision and audio in real time. In practice, this means developers can get “frontier-class reasoning” on a laptop or phone, and can scale to huge servers or cloud instances if needed.
Using Gemma 4: Tools and examples
Because Gemma 4 is open-source and widely supported, you can integrate it into many environments:
Python/Transformers: Hugging Face Transformers and TRL support Gemma 4 out of the box. For example, in Python you might do:
from transformers import pipeline pipe = pipeline("any-to-any", model="google/gemma-4-e2b-it") result = pipe("Explain the water cycle in simple terms") print(result["generated_text"])
This lets you feed either text or images through pipeline, and returns rich responses. You can also use AutoModelForMultimodalLM and AutoProcessor to handle images and audio explicitly.
Llama.cpp:The open-source llama.cpp framework now supports Gemma 4 (including images+text). You can download GGUF checkpoints (e.g. via Hugging Face) and serve them locally. For instance, after installing llama.cpp, you could start a local Hugging Face API-compatible server that runs Gemma 4. This makes it easy to connect Gemma 4 to local apps (coding assistants, GUI agents, etc.) without cloud.
Web/JS: The transformers.js library lets you run Gemma 4 in-browser (WebGPU/WebAssembly). Hugging Face’s examples show calling Gemma models via JS for text, image+text, and even audio+text chats.
Other frameworks: There is wide support across the ecosystem. For example, you can use vLLM or NVIDIA Triton Inference (NIM) for high-throughput serving, ONNX or TensorRT for acceleration, and Google Colab or Vertex AI for scaling. Google even provides a specialized Gemma library (gemma-llm) with Colab tutorials. The readthedocs page and model card contain conversion guides for Keras, JAX/Flax, LangChain, and more.
Download and formats: You can get Gemma 4 weights from Hugging Face, Kaggle, Ollama, Docker Hub, etc. LM Studio also offers GGUF versions (with 4-bits quantization) for CPU/Apple Silicon. In LM Studio’s docs, they note that the largest 31B model takes ~19GB RAM to run, while 2B/4B models need ~4–6GB. (Indeed, the 4-bit quantized version of 31B is only ~17GB.)
Example: Fine-tuning with Cloud Run
Gemma 4 can also be fine-tuned on your own data. For example, Google Cloud advocates have published guides showing how to fine-tune Gemma 4 on serverless GPUs. In one blog, a developer used NVIDIA RTX 6000 GPUs on Cloud Run Jobs to fine-tune a 31B Gemma model (with LoRA and 4-bit quantization) for pet-breed classification. They explain how to adapt prompts for multimodal inputs (placing image tokens before text) and how to load the correct AutoModelForMultimodalLM class. (Crucially, Gemma 4’s multimodal models expect image tensors injected via a special pipeline.) Cloud Run now supports such ML jobs with GPUs, so you can train or batch-infer Gemma models at cloud scale without managing servers. Google’s Cloud Run guide also shows how to deploy a Gemma 4 model behind a REST API (using vLLM for serving) and call it with simple curl requests. This means you can prototype an agent (e.g. with Google’s Agent DevKit) using Gemma 4 locally or in Cloud Run and then productionize it on Google Cloud or on-device.
In summary
Gemma 4 represents a major step in open AI. By combining high reasoning ability, true multimodal input (text+image+audio+video), and huge context windows, it lets developers build sophisticated agents and assistants that run on-device or on affordable hardware. It’s not just a research demo — it’s production-ready and widely supported. Whether you want to do everything on your laptop (the 31B model on a PC GPU) or on your phone (the E2B/E4B models on Android), Gemma 4 provides state-of-the-art performance. You can get started today via Hugging Face Transformers or your favorite LLM framework, and even use Google Cloud tools for scaling up. The open-source community is already contributing many tutorials and demos (see Hugging Face’s Gemma 4 blog). In short, Gemma 4 is now one of the most capable open models available, and it’s designed to be easy to use and extend in real projects.
Top comments (0)