SlothChef: A Local AI Vision Agent That Turns Your Fridge Into a Meal

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

I built SlothChef, a local, zero-effort vision agent for anyone who stares blankly into their refrigerator hoping a meal will magically assemble itself.

Instead of manually typing ingredients, scrolling through endless recipe pages, or standing in front of the fridge trying to decide what to cook, SlothChef does the thinking locally:

1. Snap & Forget: Take a single photo of an open fridge or pantry.
2. Detect & Decide: SlothChef identifies visible ingredients and uses a local language model to reason about them.
3. Lazy Meal: It generates a quick, low-effort recipe designed around minimal cooking and minimal cleanup.
4. Use What You Have: The workflow prioritizes ingredients already available instead of requiring a shopping trip.

The goal is simple: reduce decision fatigue and turn "I don't know what to cook" into an actual meal.

Demo

Here is an example CLI trace of SlothChef processing a smartphone photo:

[SlothChef Intake Monitor Active]

[07:15:22 PM] Image Detected: "fridge_shelf_evening.jpg"
   ↳ Analyzing via SmolVLM-Instruct...
   ↳ Detected items: Eggs (3), half a block of cheddar,
     wilted spinach, corn tortillas, jar of salsa.

   ↳ Querying Qwen2.5-Coder-3B for lazy culinary logic...

[Recipe Generated: 10-Minute Lazy Skillet Migas]

- Effort Level: Low
- Dishes Used: 1 skillet
- Ingredients: Corn tortillas, 3 eggs, spinach,
  cheddar, salsa

- Instructions:
  1. Tear tortillas into the pan with a little oil.
  2. Add the eggs and spinach, then scramble.
  3. Turn off the heat, add cheese and salsa, and eat.

Code

The core Python workflow connects a local vision model with a local language model through Ollama:

import json
import ollama
from pathlib import Path


def analyze_fridge_image(image_path: Path) -> str:
    """Identify edible ingredients visible in a fridge image."""
    response = ollama.chat(
        model="smolvlm",
        messages=[
            {
                "role": "user",
                "content": (
                    "List only the edible ingredients, produce, "
                    "and condiments visible in this fridge photo "
                    "as comma-separated items."
                ),
                "images": [str(image_path)]
            }
        ]
    )

    return response["message"]["content"]


def generate_lazy_recipe(ingredients_list: str) -> dict:
    """Generate a minimal-effort recipe using local AI."""

    system_prompt = """
    You are SlothChef.

    The user is tired and wants to cook with minimal effort
    and as few dishes as possible.

    Based on the available ingredients:
    1. Pick 3-5 compatible ingredients.
    2. Prefer one-pan or one-pot meals.
    3. Keep preparation simple.
    4. Return valid JSON only.

    Format:
    {
        "recipe_name": "...",
        "prep_time_minutes": 10,
        "dishes_used": 1,
        "steps": ["...", "..."]
    }
    """

    response = ollama.chat(
        model="qwen2.5:3b",
        messages=[
            {"role": "system", "content": system_prompt},
            {
                "role": "user",
                "content": f"Available items: {ingredients_list}"
            }
        ],
        format="json"
    )

    return json.loads(response["message"]["content"])


if __name__ == "__main__":
    photo = Path("fridge_shelf_evening.jpg")

    if photo.exists():
        print("Inspecting fridge contents locally...")

        detected_items = analyze_fridge_image(photo)
        print(f"Found: {detected_items}")

        print("Designing low-effort dinner...")

        recipe = generate_lazy_recipe(detected_items)

        print(
            f"\nMeal: {recipe['recipe_name']} "
            f"({recipe['prep_time_minutes']} mins, "
            f"{recipe['dishes_used']} pan)"
        )

        for i, step in enumerate(recipe["steps"], 1):
            print(f"{i}. {step}")

How I Built It

SlothChef uses modular open-source AI components running locally:

• Vision: SmolVLM through Ollama for extracting ingredients from fridge/pantry images.
• Language Model: Qwen2.5-3B-Instruct for turning detected ingredients into a structured recipe.
• Workflow: A simple local Python interface that accepts an image, runs vision analysis, passes the detected ingredients to the language model, and prints the resulting meal.
• Output: Structured JSON containing the recipe name, preparation time, number of dishes, and cooking steps.

The important part is that the AI pipeline does not require a cloud API.

Why Open Innovation Matters

Open-weight models make small personal AI utilities practical without requiring every interaction to go through a commercial API.

1. Privacy

A fridge photo can reveal a surprising amount about someone's home and lifestyle. Local inference means the image can stay on the user's machine instead of being uploaded to a remote service.

2. Offline Independence

The application can continue working without an internet connection once the required models are available locally.

3. No Per-Request API Cost

There is no per-meal API charge. Users can experiment with the workflow as often as they want using their own hardware.

4. Developer Control

Running open models locally gives developers control over prompts, output formats, model selection, and inference configuration.

This makes it possible to build small, focused AI agents around very specific everyday problems.

My Agent Session

[DevRelay Session: slothchef-local-vlm-trace]

The Lazy Payoff

SlothChef is intentionally simple.

The problem isn't that people don't know how to cook. Sometimes the problem is simply not wanting to decide what to cook.

One photo of the fridge can become a concrete meal suggestion without typing ingredients, searching recipe websites, or opening another delivery app.

Less thinking. Less scrolling. Less cleanup. More eating.

That's the idea behind SlothChef.