This is a submission for the Gemma 4 Challenge: Write About Gemma 4
Disclaimer: This is an individual submission for Francis Tran (@francistrdev)...
Some comments have been hidden by the post's author - find out more
For further actions, you may consider blocking this person and/or reporting abuse
Ahahah, I had the same issues running Gemma locally with Ollama – my computer slowly turned into a snail, everything felt super slow, and I had to close almost every app 😅 In the end, it completely froze anyway!
Good job with your multiversal analysis! That's a top example of collaboration!👏🏻
Hey Klaudia! Yea it is always a common issue when running any local model in particular. You just have to hope that it will be relatively fast as if you are using the Cloud Version (which I had to use for this case). I am surprised @codingwithjiro (Elmar) got his to run on a laptop. If I were to run on my laptop, it would be cooked.
Appreciate the comment! Glad you liked it :D
Glad it's not just me @klaudiagrz. Thanks for reading!
Have you tried the E4B and E2B models, they're quite fast and easy to run. I used them for my agentic browser swarm using a custom MCP (albeit it dropped token drain by 80%, so extremely lightweight), to run concurrent instances. I got to 4 concurrent E2B's on a 8gb gpu running at 100+ TPS each using an RX 9060 XT and LM Studio using Vulkan (trying to get lllama.cpp rocm working)
Even with lower models, it tends to be the same result (for me at least and I am using a Desktop). Maybe if you get lucky? Not sure if she tried it yet, but would assume she had?
Well that means probably running it wrong? Try LM Studio, then make sure you set the pipeline to use your native accelerator (Cuda/RoCm) if not supported, run Vulkan, turn on KV Cache quantization to Q8 and give that a try, if still not, turn on shared KV Cache, just be sure to scale your KV Cache accordingly for all your parallel runners)
The Strawberry test How many r’s are there in strawberry?” (There are three) is interesting. Why need 3 interpretations for that? That seems unnecessary.
I find that to be interesting as well since I ran it in Ollama with the latest Gemma 4 model and it gave me this:
@konark_13 How did you ran Gemma 4?
I ran Gemma4 on the terminal and played and tested it. Am I using temu version of Gemma4? I think I need to check my model and then try it. haha
Haha yeah it would be interesting to observe if you get a similar output again
Yea ran the same thing on terminal and same outcome. How did you install Gemma 4?
Really interesting breakdown — especially the way you frame Gemma 4 in terms of “when to use vs when not to use” across different application scales.
One thing I’ve noticed building with Gemma 4 (specifically e4b-it) is that the real advantage isn’t just capability, but deployability in constrained environments.
I’ve been working on an offline-first education system where Gemma 4 runs locally in classrooms (no cloud dependency). In that context, the “small but efficient model” argument becomes more important than raw benchmark performance.
For example, latency + affordability + offline inference matter more than peak reasoning ability when you’re trying to support real students in low-connectivity regions.
Curious if you think the tradeoff between “model power vs local deployability” will become a bigger deciding factor than benchmarks in the next wave of LLM adoption?
I agree, a local, small, and efficient AI like Gemma 4 is good for areas with low-connectivity. Personally, the first pro that comes in mind is its local capabilities not its model power. What's important is that I can use AI while offline and that is already a great feature in itself.
What's interesting would be the future local AI models that use less compute power. Imagine an efficient and reliable AI in a low-end device powered locally. This is perfect since not all people need big data centers from the cloud for everyday AI use.
Thanks Allan! Sorry for the late response!
I believe the "model power vs local deployability" is important and I believe it is already used as a deciding factor, which will receive more attention. Benchmarks isn't a good way to measure AI capabilities since there are cases where data are artificially modified to reach those requirements instead of relying on if the AI is accessible and powerful enough for others to use. Hope that makes sense! Thanks again Allan :D
What a wonderful article. The collaboration and teamup was awesome. Learned a lot while discussing ideas and distribution of ideas. We let loose on Gemma4 and tried everyway possible to check it's capabilities. If we missed any, next time we will bring something even better.
Awesome collaborating with you all. Thanks for the time and lessons @francistrdev, @codingwithjiro and @javz
Yea thanks for sharing with your experience on Gemma 4. It was fun to meet you guys on call (especially not knowing who is AI lol).
Tôi cũng là 1 dev nhỏ lập trình viên nhỏ và muốn được tham gia chung nghiên cứu 1 số dự án 🥰
Nghe tuyệt đấy, Thắng! Cụ thể là những chủ đề nào trong Nghiên cứu?
This is a great, well-rounded breakdown. The open-weight space is moving so fast that it's hard to know which model fits the local-dev workflow best. Highlighting Gemma 4's specific strengths—especially its coding and multimodal capabilities—against the hardware requirements makes the decision-making process much clearer.
Glad it helped one of your decisions Tahosin!
Indeed! It is always important to factor not only the hardware to run local LLMs, but to also determine which model suits your needs! Thanks Tahosin :D
This is a fun framing, but the practical question is exactly right.
Gemma 4 might be good for development, but I would not judge it only by benchmark numbers. For real dev work, the test is much messier: can it understand an existing codebase, follow project conventions, avoid over-editing, explain tradeoffs, and recover when the first attempt fails?
A model can look strong in isolated coding tasks and still struggle with repo-level context, dependency issues, tests, edge cases, and debugging across multiple files.
For me, the best use case for models like this is not “replace the developer.” It is fast scaffolding, code explanation, refactoring help, test generation, and catching obvious mistakes.
The real value depends on whether it reduces thinking friction without adding cleanup debt.
@sunnysingh1997 that's a really mature and practical take. Value is only ever served when it helps a developer's thinking and goals for a project. Because in reality, the bottleneck still lies on the developer's decisions for the project. I'd say, as long as the model keeps the developer sane and productive without mental overload, that model is valuable enough.
Hey Suny! The big thing is never judge an AI by the benchmark because there is a history of data being skewed to get the requirement.
I agree! This is quite common for those using AI in general and I think it's a good idea to determine if an AI can do that. Thanks Suny for sharing :D
This multi-perspective review is remarkably grounded. On-device LLMs are never built in a vacuum; they are strictly bound by hardware constraints. Balancing edge-cloud boundaries, managing token loops, and handling contextual freezing on standard hardware with limited RAM requires a deterministic architectural mindset. Navigating these constraints is the exact engineering literacy every developer needs in the era of local AI.
@lcmd007 I just really hope that local AIs will take way less compute power than what we currently have. That would be a complete game-changer for sure.
Thanks Andy! Adding to @codingwithjiro point, it would be neat to have less compute power. I believe Google is currently researching on how to maximize its potential without the need to build more data centers, which I think is why Gemma4 exist, though I might be wrong. Thanks again Andy!
SRE lens worth adding: model comparison without a golden eval suite and a drift monitor is theatre. We swapped Llama-3.3 for Gemma-3 on a classification surface and the win on benchmark turned into a 12% regression in production, because the training distribution differed from real traffic. Now we run a paired-comparison test: same 500 inputs on both models, scored against a human-labeled gold set, with a McNemar test on the disagreement vector. Plus an OTel recording rule that alerts on any model-swap-day classification distribution divergence. Without that, the benchmark numbers are just press releases.
Interesting share Jasmine!
Interesting observation about Gemma being more "careful" as an agent.
I noticed something similar while integrating multiple LLMs into an audio-first product. Once the summaries became "good enough", the main differences stopped being raw intelligence and became things like tone, density, pacing and reliability under load.
That was a weird moment because it made the model feel more like one component inside a media pipeline rather than "the product".
Hey Kirill! Hope you are well. I am curious on why you decide to use multiple LLMs into your audio product? Probably I am not understanding what your product does specifically with audio? Otherwise, thanks for sharing!! :D
The audio part is actually the core idea 🙂
I built a small audio-first system where you can dump long reads into a Telegram bot and get back short spoken summaries for passive listening while walking, commuting, cooking, etc. So I ended up testing multiple LLMs not because I wanted "the smartest model", but because different models create noticeably different listening experiences once converted to speech.
Some feel more like concise radio hosts
Some feel more chaotic
Some ramble
Some compress information better
At some point the model itself stopped feeling like the product and started feeling more like casting different voices into the same media pipeline.
Oh that makes sense! Have you figured out the solution or are you still trying to? I am curious to see if there is a way to do this without an LLM summarizing it (I know it's possible, but can't pinpoint it).
I suspect the funny part is that once summaries become "good enough", users stop caring how the summary was technically produced. At that point they care more about:
Does playback feel seamless?
Can I consume this while doing something else?
Does the pacing feel natural?
Does the voice become mentally tiring after 20 minutes?
Does the app interrupt my flow?
That was the weird realization for me - the product gradually became less about "AI summarization" and more about minimizing cognitive friction around information consumption
Yea fair enough. It's all about how the users are using the product and ensuring you have cases meet. Obviously, you can't pleased everyone but it's the reality of it.
local inference means you can commit your model config - quantization, context window - right into the repo. you lose that on cloud APIs; the vendor rolls config changes under you without notice.
That is true Mykola!
yeah - and once you've had a vendor silently change their default context window on you, you start treating the model spec like any other dependency.
My experience has been, if you could run Gemma 4, why not run Qwen 3.5/3.6 instead? While Gemma is quite capable, Qwen just performs faster and with less bugs for everything I threw at it.
Hey! Hope you are well. It is the reason why this Analysis exist! It accounts for other people's experience using Gemma 4 compare to what others originally use and see if it is right for them. Of course, there are other options like Qwen like you mention! I believe having different experiences from different people gets the reader a general idea of what you are dealing with when using Gemma 4 and having a local AI in general. Thanks for sharing :D
Really interesting breakdown of Gemma 4 and its real-world development use cases. The practical discussion around local performance, tooling, and developer workflow makes this much more useful than typical benchmark-only comparisons. At Asmorix, we also encourage students to test AI models hands-on to understand where they actually fit into modern development workflows. 🚀
Thanks asmorix for sharing :D
the looping bug is real, ran into that myself a few times. the to-do list before acting thing is actually something I've come to appreciate though, beats Copilot just going rogue on your codebase
I agree! Is always nice to have for Gemma 4 to do one task at a time in that list! Thanks Jaime :D
Focusing on the multiversal analysis of developer personas really highlights the main point: there is no single 'best' model anymore. Tailoring the choice based on hardware constraints and specific coding workflows—rather than just chasing raw parameter count—is how teams actually optimize their stack.
@syedahmershah True. When AI comes in all shapes and sizes, there is more freedom for engineers on choosing which model suits the problem best. Necessity will be one of the main game-changers when it comes to building newer models.
The breakdown of resource efficiency versus raw output size is spot on. For everyday development tasks, being able to run a highly capable model locally without burning through massive cloud compute credits is the exact kind of practical trade-off most developers are weighing right now.
Indeed Syed. Local AI Vs. the cloud is a big deciding factor at the moment and it's important to choose based on what you currently have! Thanks Syed!