DEV Community

Saquib Shahid
Saquib Shahid Subscriber

Posted on

LazyCart 🦥 — Zero-Effort Pantry Scanner & Autonomous Grocery Cart

#devchallenge #hf26challenge #gemma #browserbase

Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady Submission 🦥

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

Standing in front of the fridge with the door open, staring blankly while cold air pours out, is a universal human experience. You wonder what you can make with half a block of cheddar, four eggs, and hot sauce, and you dread making a mental shopping list for the rest.

LazyCart 🦥 is built for the unapologetic sloth.

Instead of meal planning or manually shopping:

  1. You snap one lazy photo of your fridge or pantry (or click a sample preset).
  2. Gemma 2 inspects the contents, inventories your staples, and invents "Sloth-Approved Recipes" engineered strictly for 1-pot/pan preparation and minimal dishwashing.
  3. It figures out what missing ingredients are required to unlock more zero-effort meals.
  4. With one click, an autonomous Browserbase cloud browser agent launches, searches online supermarket aisles, and adds the items directly into your grocery cart while you lie back on the couch.
  5. Every model reasoning step, DOM action, and agent tool invocation is instrumented and visualized with Sentry Agent Tracing.

Who gets to be lazier? Anyone who hates deciding what to eat, hates doing dishes, and hates typing items one-by-one into online grocery search bars.

Demo

  • Live Web App: http://localhost:3000 (Deployable to Render with 1-click)
  • Built-in Supermarket Demo: /mock-store (SlothMart interactive store)

Visual Walkthrough

  1. Sloth Presets & Upload: Instant one-click presets like "The Bachelor Pad Fridge" and "The Busy Family Pantry" or drag-and-drop custom camera photos.
  2. Pantry & Recipe Breakdown: Inventory chips with freshness indicators, paired with recipes rated by an "Effort Score" (1 to 5).
  3. Live Browserbase Terminal: Real-time Server-Sent Events (SSE) stream showing the cloud browser navigating store aisles, searching products, and adding items to the cart.
  4. Sentry Traces Drawer: Slide-out observability drawer showing spans, latency breakdown, and execution tree.

Code

The complete source code is open source under the MIT License:

📁 Repository: github.com/saquib-shahid/hacktoberfest-ai-challenge-week-3

(Complete codebase with vision engine, Browserbase agent, Sentry tracing, and SlothMart mock store)

How I Built It

LazyCart is built around a modern, resilient open-source AI and browser automation architecture:

graph TD
    User([User / Sloth]) -->|1. Takes Single Photo / Clicks Preset| UI[LazyCart Web App]
    UI -->|2. POST /api/analyze| Server[Express Node.js Backend]

    subgraph Open-Source AI Engine
        Server -->|3. Multimodal Vision Reasoning| Gemma[Gemma 2 Open-Weight Model]
        Gemma -->|4. Parsed Inventory & 1-Pot Sloth Recipes| Server
    end

    subgraph Observability
        Server -->|Trace Spans & Metrics| Sentry[Sentry Agent Tracing]
    end

    subgraph Headless Browser Automation
        Server -->|5. Launch Autonomous Cart Agent| BB[Browserbase Cloud Browser]
        BB -->|6. Search, Select, & Add to Cart| Store[SlothMart / Online Grocery Store]
        BB -->|7. Live Debug URL & Progress SSE| UI
    end

    UI -->|8. Visualizes Items, Recipes & Live Cart Replay| User

The Tech Stack:

  • Vision Reasoning: Gemma 2 (multimodal inference via Google GenAI SDK / local inference fallback) running structured JSON extraction.
  • Autonomous Web Automation: Browserbase SDK (@browserbasehq/sdk) + Playwright Core, running cloud-isolated browser instances to navigate store catalogs and manipulate the DOM.
  • Observability & Agent Tracing: Sentry Node (@sentry/node), instrumenting custom agent spans (ai.multimodal_reasoning, agent.tool_call, agent.dom_action) with real-time latency and status capture.
  • Frontend & Design System: Pure Vanilla CSS design system with deep slate dark mode, glowing accents, glassmorphism cards, and fluid responsive layouts.
  • Resilience Engineering: High-fidelity simulation mode so hackathon judges and evaluators without API keys can immediately experience the entire vision and agent pipeline out-of-the-box.

Why Does Open Innovation Matter?

In our opinion, kitchen automation is one of the clearest test cases for why open-weight models and open innovation matter over closed, proprietary cloud APIs:

  1. Extreme Living Space & Home Privacy: A photo of your fridge or pantry isn't just an inventory list — it exposes your living conditions, personal medication/prescriptions in the door, family photos on the fridge, and dietary habits. Handing private domestic imagery to closed AI providers risks having your living space permanently ingested into black-box training sets. Open weights ensure your home data stays your own.
  2. Eliminating the Recurring "AI Tax": Why should checking whether your milk is expiring cost a $20/month subscription or ongoing per-image token fees? Open-weight models like Gemma 2 run locally or on shared low-cost servers, freeing builders and families from corporate subscription traps.
  3. Zero Vendor Lock-In & Offline Resilience: If an API vendor changes pricing, disables accounts, or has an outage, a closed tool dies. An open-weight agent runs anywhere — even on a local mini-PC in your kitchen without an internet connection.

The Lazy Payoff

  • Dishes Saved: Down from 3-4 pans and multiple plates to exactly 1 pot or 1 mug per meal.
  • Time Saved: No standing in front of the fridge (saves ~10 minutes of daily decision fatigue) and no typing 8 individual items into a grocery search bar (saves ~15 minutes).
  • The True Sloth Metric: You can literally snap the photo, drop your phone on the bedside table, and let the agent populate your groceries while you take a nap.

Prize Categories

We are entering the following categories:

  1. Best Use of Gemma ($200): Gemma 2 powers our multimodal kitchen reasoning, pantry categorization, and minimal-dish recipe generation.
  2. Best Use of Browserbase ($100 🆕): Browserbase powers the cloud headless browser that autonomously drives supermarket store navigation, handles anti-bot challenges, and fills shopping carts.
  3. Best Use of Sentry Agent Tracing ($100): Sentry instruments every phase of the agent lifecycle — measuring model latency, tool execution spans, and DOM action success rates.
  4. Best Use of Render ($200): Packaged for zero-friction cloud deployment on Render's web service tier.

Built with ❤️ and laziness for Hacktoberfest 2026.

Top comments (0)