Nutrilens

What We Built :

NutriLens is an open-source, edge-first multimodal AI agent that audits packaged food labels in real time. Powered by Google's Gemma 4 model and built strictly according to the Agent Skill Open Standard (agentskills.io), NutriLens reads nutrition panels, verifies packaging claims, identifies hidden glycemic sweeteners, and computes an automated Claim Integrity Score (0–100).

Unlike generic food-scanning apps, NutriLens combines multimodal vision perception with deterministic Python verification, ensuring that math and regulatory checks are 100% accurate without relying on LLM arithmetic.

How We Built It :
NutriLens uses a dual-engine hybrid architecture designed for zero-latency edge computing with cloud failover capability:

┌─────────────────────────────────────────────────────────┐
│ User Input (Packaging Photo) │
└────────────────────────────┬────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ Dual Inference Engine (harness/engine.py) │
│ • Edge Mode: Local RTX 3050 (6GB VRAM) via Ollama │
│ • Cloud Mode: Gemma 4 via Google Gemini API │
└────────────────────────────┬────────────────────────────┘
│ Extracted JSON
▼
┌─────────────────────────────────────────────────────────┐
│ Agent Skill Engine (skills/nutrilens-auditor/) │
│ • SKILL.md (Agent Skill Open Standard Spec) │
│ • macro_verifier.py (Atwater 4:4:9 factor math) │
│ • glycemic_index.json (Hidden sugar cross-reference) │
└────────────────────────────┬────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ Streamlit Dashboard & Compliance Scorecard │
└─────────────────────────────────────────────────────────┘

Technologies & Tools Used :

AI & Multimodal Intelligence: Gemma 4 (via Google Gemini API & local Ollama weights)

Agent Framework: Agent Skill Open Standard (agentskills.io)

Backend API: FastAPI & Uvicorn

Frontend UI: Streamlit

Local Hardware Optimization: NVIDIA RTX 3050 (6GB VRAM) running quantized weights

Quality Assurance & Testing: Pytest & GitHub Actions CI/CD

License: Apache License 2.0

youtube link

https://youtu.be/PngWvji0x_Q

github

https://github.com/venky829/hacktoberfest-hack-day-coimbatore-x-init-club-and-idea-club/tree/main#open-source-and-ai-usage