DEV Community

Cover image for 🍔 FoodSnap Tutor — Snap a meal, get a recipe (Gemini 2.5 Flash)
Long Phan
Long Phan

Posted on

🍔 FoodSnap Tutor — Snap a meal, get a recipe (Gemini 2.5 Flash)

#devchallenge #googleaichallenge #ai #gemini

This is a submission for the Google AI Studio Multimodal Challenge

What I Built

FoodSnap Tutor is a frontend-only app that turns any food photo into instant cooking guidance. Upload an image and the app:

  • Detects whether the image is food
  • Identifies the likely dish name (with alternatives when uncertain)
  • Generates a step-by-step recipe
  • Estimates nutrition per serving (calories, protein, carbs, fat)
  • Suggests a healthier variation and friendly moderation tips

Tech stack: React 19, TypeScript, Vite 6, Tailwind (CDN), and @google/genai calling Gemini 2.5 Flash. Everything runs in the browser — no backend required.

Demo

✨ Upload screen:
Upload screen

✨ Analysis screen:
Analysis screen

✨ Error screen:

Error screen

How I Used Google AI Studio

I leveraged the @google/genai SDK to call Gemini 2.5 Flash with:

  1. An inline image part (base64-encoded upload).
  2. A structured prompt telling the model to reply in JSON.
  3. A response schema so the SDK validates the output client-side. 
import { GoogleGenAI, Type } from "@google/genai";

// Strict JSON schema
const schema = {
  type: Type.OBJECT,
  properties: {
    isFood: { type: Type.BOOLEAN },
    dishName: { type: Type.STRING },
    recipe: {
      type: Type.OBJECT,
      properties: {
        ingredients: { type: Type.ARRAY, items: { type: Type.STRING } },
        steps: { type: Type.ARRAY, items: { type: Type.STRING } }
      },
      required: ["ingredients", "steps"]
    },
    nutrition: {
      type: Type.OBJECT,
      properties: {
        calories: { type: Type.STRING },
        protein: { type: Type.STRING },
        carbs:    { type: Type.STRING },
        fat:      { type: Type.STRING }
      },
      required: ["calories", "protein", "carbs", "fat"]
    },
    healthierVariation: { type: Type.STRING },
    friendlyAdvice:     { type: Type.STRING }
  },
  required: ["isFood", "dishName", "recipe", "nutrition", "healthierVariation"]
};

// Convert upload to an image part
const fileToPart = async (file: File) => {
  const dataUrl = await new Promise<string>((res, rej) => {
    const r = new FileReader();
    r.readAsDataURL(file);
    r.onload = () => res(r.result as string);
    r.onerror = rej;
  });
  const [meta, data] = dataUrl.split(",");
  const mime = meta.match(/:(.*?);/)?.[1] ?? "image/jpeg";
  return { inlineData: { mimeType: mime, data } };
};

export async function analyzeFoodImage(image: File) {
  const ai = new GoogleGenAI({ apiKey: process.env.API_KEY! });
  const imagePart = await fileToPart(image);
  const prompt = "You are FoodSnap Tutor, an expert AI chef and nutritionist...";
  const res = await ai.models.generateContent({
    model: "gemini-2.5-flash",
    contents: { parts: [imagePart, { text: prompt }] },
    config: {
      responseMimeType: "application/json",
      responseSchema: schema
    }
  });
  return JSON.parse(res.text);
}
Enter fullscreen mode Exit fullscreen mode

Environment wiring (Vite):

// vite.config.ts
import { defineConfig, loadEnv } from "vite";
export default defineConfig(({ mode }) => {
  const env = loadEnv(mode, ".", "");
  return {
    define: {
      "process.env.API_KEY": JSON.stringify(env.GEMINI_API_KEY)
    }
  };
});
Enter fullscreen mode Exit fullscreen mode 
# .env.local
GEMINI_API_KEY=ai-xxxxxxxxxxxxxxxx
Enter fullscreen mode Exit fullscreen mode

Multimodal Features

  • Vision input: Users upload a dish photo that the SDK sends as inline image data.
  • Structured output: Gemini returns validated JSON (recipe, nutrition, advice) for deterministic UI rendering.
  • Single multimodal call: Image + text prompt → cohesive culinary analysis.
  • UX touches: Drag-and-drop upload, instant preview, animated results, retry flow.
  • Robustness: UI handles blocked content or JSON parse errors gracefully with friendly messages.

Notes
• Frontend-only: In production restrict the API key to allowed origins or proxy requests through a lightweight backend.
• Built with React + Vite + Tailwind for fast iteration and static deployment.

Thanks for reading! If you’d like to try it out or peek at the code, check the demo and repo links above. 🚀

Top comments (0)