What’s Inside?
This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend.
What I Built
I built WHAT’S INSIDE? because of a friend of mine, A.
She has always been quite particular about what she eats and what she uses on her skin and hair. If she is buying a shampoo, she will turn the bottle around and read the ingredients. If she is buying a snack, she will check the back of the packet. Sometimes she will even send me a picture of an ingredient and ask, “What is this supposed to be?”
The funny part is that I don't think she is overly paranoid about these things. She just genuinely wants to know what she is putting into her body and what she is putting on herself.
The problem is that most ingredient labels are almost impossible for a normal person to understand. You look at a shampoo bottle and see things like phenoxyethanol, sodium chloride, limonene, linalool, surfactants and citric acid, and unless you have a chemistry or biology background, most of those names don't really mean much.
So naturally, you search them online — and that creates an even bigger problem. One website says an ingredient is dangerous, another says it is perfectly fine, and somewhere on Instagram there is probably a reel claiming that the same ingredient is going to ruin your life.
After seeing this happen so many times, I thought there had to be a better way.
That was the starting point for WHAT’S INSIDE?
The idea is simple: take a picture of a product label and let the application break it down into something a normal person can actually understand.
The project uses Gemma 4:12B locally through Ollama to read the label, identify the product and extract its ingredients. Those ingredients are then matched against a local knowledge base containing information about what they do, why they are used, available scientific evidence, regulatory context, and things that are still uncertain.
One thing I was very particular about while building it was not turning the application into another “toxic chemicals” scanner.
I didn't want to build something that sees a complicated chemical name and immediately puts a big red warning sign next to it. An ingredient can have a particular hazard in one situation and still be reasonable in another. The amount used, how someone is exposed to it, how long it stays on the skin, whether it is washed off, and individual sensitivity can all change the picture.
And sometimes the ingredient list simply doesn't tell us enough.
In those cases, the application should say that instead of pretending it knows more than it actually does.
That idea ended up influencing the whole project.
The backend is built with Python and FastAPI, the frontend uses React, TypeScript, Vite and Tailwind, and SQLite stores the local ingredient and evidence data.
The application is also designed to work across many different types of products — not just shampoo and skincare. It can handle food, cosmetics, cleaning products, baby products, pet products, gardening products, automotive products, electronics, DIY products and several others.
I also added personal preferences, because A's concerns aren't necessarily the same as someone else's. One person might care mostly about fragrance, another might be concerned about allergens, another might want to avoid certain food additives, and someone else might care about environmental impact.
Instead of giving everybody exactly the same generic explanation, the application can bring the things that matter to that particular person to the front.
I also wanted the project to work locally rather than depending completely on a cloud API. That was one of the reasons I chose Ollama and Gemma 4:12B.
A photo of someone's products can reveal more about them than you might initially think, and I liked the idea that the label could be analysed on the user's own machine without having to send it somewhere else just to get an answer.
It also made the project more interesting to build because I had to think about the entire pipeline myself rather than simply sending an image to an API and displaying whatever came back.
Why the UI is intentionally simple
The UI was another decision I made quite deliberately.
I didn't want it to look like one of those generic AI websites with a dark background, glowing purple gradients, floating cards and a giant “AI” in the middle of the screen.
The whole purpose of this project is to make something complicated feel simpler, so the interface needed to reflect that.
I kept it clean and fairly minimal, taking inspiration from actual ingredient labels, scientific notes, packaging and editorial layouts. The idea was that when someone opens the application, it should feel calm and understandable instead of making them feel like they have opened some complicated piece of AI software.
Even the colours have a purpose. They are used to separate normal information, things worth paying attention to, evidence and uncertainty rather than simply being there because they look flashy.
In a way, the simple UI is part of the message of the project.
There is already so much noise around “chemicals” and product safety, and I didn't want the application itself to add to that noise. A chemical name isn't automatically something bad, just like something being called “natural” doesn't automatically make it good.
I wanted WHAT’S INSIDE? to sit somewhere in the middle and say:
Here is what we know. Here is why this ingredient is here. Here is what the evidence says. And here is what we still don't know.
At the end of the day, this project started with something very small: a friend standing in a store aisle, reading the back of a shampoo bottle and wondering what half of the words meant.
I built WHAT’S INSIDE? because I thought she shouldn't need to spend twenty minutes searching every ingredient on Google just to make a simple decision.
She should be able to look at a label and actually understand it.
And if I can make that experience a little easier for her, then I think the project has already done what I wanted it to do.
Demo
Try the project:
https://whatsinside-pi.vercel.app/
The public deployment currently demonstrates the application workflow. The full local AI experience uses Gemma 4:12B through Ollama.
Code
GitHub repository:
https://github.com/Crytic35/WhatsInside
The repository contains the frontend, FastAPI backend, local ingredient/evidence data, tests, and deployment configuration.
How I Built It
WHAT’S INSIDE? is built around Gemma 4:12B, an open-weight model running locally through Ollama. The goal was to make AI useful without making it the unquestioned source of truth.
When a user scans a product label, Gemma helps read the label, identify and structure the ingredients, classify the product, and turn technical ingredient names into explanations that a normal person can understand.
The extracted information is then checked against a local evidence database containing ingredient functions, available evidence, potential concerns and limitations.
This distinction matters. The AI isn't simply asked, “Is this ingredient dangerous?” Instead, it helps interpret available evidence and clearly separates what is known from what cannot be determined from an ingredient list alone.
The main stack is:
- Gemma 4:12B + Ollama — local AI inference
- React + TypeScript + Vite + Tailwind — frontend
- FastAPI + Python — backend
- SQLite — local ingredient and evidence database
The architecture is intentionally local-first, keeping the main AI workflow on the user's machine when running with Ollama.
Why Does Open Innovation Matter?
For me, open innovation was important because the problem itself is about transparency and trust.
A closed AI API could have made the project easier to build, but it would also mean relying on someone else's infrastructure for the core intelligence of the application.
With Gemma running locally through Ollama, I can control the model, prompts, data flow and application logic myself.
It also makes the idea of privacy much more meaningful. A user can scan something they eat, apply to their skin, give to their child, or use around their home without the core analysis automatically requiring their images to be sent to a third-party AI provider.
Open technology made it possible for me to build a product that explains information without turning the AI itself into another black box.
My Agent Session
This was a solo build, but I used AI-assisted development throughout the process.
I used Antigravity and DevRelay to help implement features, work through debugging, structure parts of the application and iterate quickly as the project evolved.
The important product decisions were still mine — especially the evidence-based analysis, privacy-first architecture, safety boundaries, UI direction and how the results should actually be communicated to users.
DevRelay Agent Session:
[PASTE YOUR DEVRELAY AGENT SESSION LINK HERE]
Prize Categories
Best Use of Gemma
I’m participating in the Best Use of Gemma category because Gemma is not just an API sitting behind my application — it is part of the core workflow I’m building.
For WHAT’S INSIDE?, I’m running Gemma 4:12B locally through Ollama to help read product labels, identify ingredients, structure information and turn complicated ingredient names into explanations that people can actually understand.
While building this, I’ve also started thinking beyond a simple “send a prompt, get an answer” approach. I’m learning how to make the model work as one part of a larger workflow — passing structured information between different stages, retrieving evidence, validating what the model produces, handling uncertainty and making sure the final answer is grounded in actual data instead of whatever the model happens to generate.
This is exactly the direction I want to explore further with agentic AI harnesses and workflows.
Building this project with Gemma is giving me hands-on experience with local inference, model orchestration, structured outputs, tool-based workflows and designing systems where an LLM is a component of the application rather than the entire application.
For me, this category is an opportunity to push that work further and learn how far I can take open-weight models when they are given the right tools, structure and workflow around them.
Team
This was a solo project — I designed, developed and worked on the project independently, so there are no teammates to credit for this submission.
Top comments (0)