Scan Receipts and Products to Estimate Carbon Footprints

This is a submission for Weekend Challenge: Earth Day Edition

What I Built

Carbon Lens is an AI-powered carbon footprint tracker that lets you see the environmental cost of everything you buy. Point your camera at a grocery bag, upload a photo of a receipt, or type in a list of items — and Carbon Lens will instantly estimate the CO₂e footprint of each item, color-code them by impact level, and suggest lower-carbon swaps.

The goal: make carbon awareness as easy as taking a photo. Most people have no idea that a kilogram of beef produces ~27 kg of CO₂e while the same weight of tofu is only ~2 kg. Carbon Lens puts that information right at your fingertips, every time you shop.

Key Features

• Photo Scanning — Upload a receipt, product photo, or meal image. Gemini identifies every item and estimates its carbon footprint.
• Live Camera — Point your phone camera at items on a shelf or in your cart and tap Scan for instant analysis.
• Text Input — Type or paste a list of items (e.g. "1 kg chicken breast, 2 liters milk") for quick estimates.
• Smart Swap Suggestions — Every medium/high-impact item includes a lower-carbon alternative with exact kg CO₂ savings.
• Impact Badges — Color-coded 🟢 Low / 🟡 Medium / 🔴 High badges make it easy to spot the biggest offenders at a glance.
• No Sign-Up Required — Start scanning immediately. Receipts are stored locally in your browser. Sign in later to sync your data to the cloud.
• Live Climate Context — Real-time CO₂ readings from NOAA's Mauna Loa observatory, plus six global climate signals (temperature, renewables, ice loss, deforestation, species risk, plastic waste) from NASA, IEA, IUCN, and others — shown on the dashboard and fed into Gemini's analysis prompts for timely, grounded insights.
• Dashboard & Charts — Track your total carbon footprint over time with monthly trend charts and category breakdowns.
• Dedicated Climate Stats Page — Dive deeper into all seven climate indicators with progress bars, source attributions, and regional context.
• Scan History — Browse all past scans with expandable details, swap suggestions, and AI-generated insights.
• BYOK (Bring Your Own Key) — Users can plug in their own Gemini API key for unlimited scanning. The key stays in your browser's localStorage and is sent directly to Google — it never touches our server.
• Dark Mode — Full light/dark theme support across every component.

Demo

Carbon Lens — Live Demo

Code

miracleonyenma / carbon-lens

Carbon Lens 🌿

See the carbon cost of everything you buy. AI-powered carbon footprint tracking using Google Gemini — scan receipts, photograph products, or point your live camera at items to instantly estimate CO₂e emissions and discover lower-carbon alternatives.

Live Demo · DEV Challenge Submission

---

Features

Feature	Description
Photo Scanning	Upload receipt/product photos (JPEG, PNG, WebP, HEIC up to 10MB). Gemini identifies items and estimates carbon.
Live Camera	Point your camera at items and tap Scan. Frames are scaled to 640px and compressed to JPEG 0.6 for fast analysis.
Text Input	Type or paste items (e.g. 1 kg chicken breast) — up to 5000 characters.
Smart Swaps	Every medium/high-impact item includes a lower-carbon alternative with exact kg CO₂ savings.
Impact Badges	🟢 Low (<2 kg) · 🟡 Medium (2-5 kg) · 🔴 High (>5 kg) per item.
Dashboard	Stat cards, monthly trend chart, category breakdown pie chart, recent scans.

…

View on GitHub

How I Built It

Tech Stack

Layer	Technology
Framework	Next.js 16 (App Router, Turbopack)
Frontend	React 19, Tailwind CSS v4, shadcn/ui, Motion (animations), Recharts (charts)
AI	Google Gemini (gemini-2.5-flash) via @google/generative-ai SDK
Climate Data	NOAA GML (live CO₂), NASA GISS (temperature), IEA/IRENA, IUCN, OECD/UNEP
Database	MongoDB via Mongoose
Auth	Optional — custom JWT sessions with jose (HS256, cookie-based)
State	Zustand + custom usePersistedState hook for localStorage-backed state

The Carbon Estimation Approach

Rather than just throwing images at Gemini and hoping for the best, I built a structured prompt system with embedded carbon reference data. The AI receives a set of 30+ carbon guidelines covering food, clothing, electronics, and transport:

- Beef: ~27 kg CO₂e per kg
- Chicken/Poultry: ~6.9 kg CO₂e per kg
- Tofu: ~2 kg CO₂e per kg
- Clothing (jeans): ~33 kg CO₂e per item
- Electronics (laptop): ~300-400 kg CO₂e
- Car fuel (1L gasoline): ~2.3 kg CO₂e

Gemini is instructed to return structured JSON with per-item carbon estimates, impact levels (low < 2 kg, medium 2-5 kg, high > 5 kg), and a suggested lower-carbon swap for every medium/high-impact item. This keeps the output consistent and parseable, even across wildly different inputs — from a crumpled grocery receipt to a photo of someone's lunch.

Three Prompt Strategies

I created three specialized prompts, each tuned for its input modality:

1. Image Analysis — "Analyze this image — it could be a grocery receipt, a photo of products, a meal, clothing, electronics, or any item(s)." Handles the broadest range of visual inputs.
2. Text Analysis — "Analyze this list of items and estimate the carbon footprint for each." Optimized for typed/pasted item lists.
3. Live Camera — "You are an expert environmental analyst doing real-time carbon analysis. Look at this camera frame and identify ALL visible items." Prioritizes speed and focuses on the most prominent items in the frame.

All three share the same carbon guidelines and response format, so the frontend can render any result identically.

BYOK: Bring Your Own Key

Free-tier Gemini API keys have tight rate limits. Instead of asking users to wait, I built a Bring Your Own Key system:

// The useGeminiKey hook stores the key in localStorage via Zustand
export function useGeminiKey() {
  const { state: key, setState: setKey } = usePersistedState<string>("", {
    storageKey: "gemini-api-key",
  });
  return { key, setKey };
}

When a user provides their own key, it's sent via the x-gemini-key HTTP header. The server uses it directly with the Gemini SDK — the key never gets stored in the database. If the key is invalid or exhausted, the error handler classifies the failure and shows a contextual UI:

• Rate limited → Cooldown countdown timer on the scan button
• Zero quota → "This API key has no quota" with a link to get a new one
• Invalid key → "Update your API key" button
• No key → "Add your API key" prompt with a link to aistudio.google.com

Live Camera Optimization

Streaming full-resolution video frames to an AI API would be expensive and slow. The live camera component scales frames down to 640px max width and compresses to JPEG at 0.6 quality before sending:

const maxWidth = 640;
const scale = Math.min(1, maxWidth / video.videoWidth);
canvas.width = video.videoWidth * scale;
canvas.height = video.videoHeight * scale;
ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
const dataUrl = canvas.toDataURL("image/jpeg", 0.6);

This keeps payload sizes manageable while retaining enough visual detail for Gemini to identify products and labels.

Rate Limit Resilience

When Gemini returns a 429 with retry in Xs, the API extracts the retry delay and passes it to the client. The UI then disables the action button and shows a live countdown:

{cooldown > 0 ? (
  <>Wait {cooldown}s</>
) : (
  <><Leaf className="h-4 w-4" /> Analyze Carbon Footprint</>
)}

I also added zero-quota detection to distinguish between "wait and retry" (transient rate limit) and "this key has no access" (permanent). The Gemini SDK returns limit: 0 in the error message for keys with no free-tier allocation — catching this prevents users from waiting forever for a retry that will never work.

Beyond rate limits, the error handler classifies 503 "high demand" responses from Gemini as a distinct SERVICE_BUSY code, showing users a friendly "try again in a moment" message with a countdown instead of a cryptic failure.

Live Climate Context

The dashboard doesn't just show your personal footprint — it grounds it in the real state of the planet. Every day, Carbon Lens fetches the latest CO₂ concentration from NOAA's Mauna Loa observatory and displays it alongside six editorial climate signals sourced from NASA GISS, IEA, IRENA, IUCN, OECD, and others:

• CO₂ concentration — Live daily ppm reading from Mauna Loa
• Global temperature — +1.55°C above pre-industrial baseline
• Renewable energy — 35% of global electricity from renewables
• Ice sheet loss — 150 Gt/yr from Greenland + Antarctica
• Forest loss — 3.7M ha/yr of tree cover lost
• Species at risk — 44,000+ species on the IUCN Red List
• Plastic waste — 460M tonnes produced per year

The compact dashboard shows three highlight cards (CO₂, temperature, and your personal footprint) with a link to a dedicated /dashboard/climate page that displays all seven indicators with progress bars and source attributions.

Crucially, this climate data isn't just decorative — it's also fed into Gemini's analysis prompts via buildAnalysisContext(). When the AI generates insights for your scan results, it can reference current CO₂ levels and regional context to make suggestions more timely and grounded.

Guest-First Architecture

Most carbon trackers require sign-up before you can do anything. Carbon Lens flips this: authentication is entirely optional. You can scan receipts, use the live camera, and build up a full history without ever creating an account.

Here's how it works:

1. Anonymous scans — The scan API detects whether a session token is present. If not, it still runs the full Gemini analysis but skips the database write. The response includes anonymous: true so the client knows to save locally.

2. Local storage — A useLocalReceipts hook backed by Zustand + localStorage caches every anonymous scan result. The dashboard merges local and server stats seamlessly — you see the same charts and totals either way.

3. Sync on login — When a user eventually signs in, the dashboard shows a "You have X local receipts not yet saved" banner. One click sends them all to the server via a /api/v1/receipts/sync endpoint, then clears the local cache.
   

// Anonymous users see a "Sign in to sync" button in the header
{!isAuthenticated && (
  <Link href="/login">
    <Button variant="outline" className="gap-2">
      <LogIn className="h-4 w-4" />
      Sign in to sync
    </Button>
  </Link>
)}

This means the app's value proposition — "see the carbon cost of what you buy" — is instantly accessible. The account just adds persistence and cross-device sync.

Data Model

Each scan is stored as a Receipt document in MongoDB:

interface IReceiptItem {
  name: string;
  quantity: number;
  unit: string;           // "kg", "item", "liter"
  category: string;       // "meat", "dairy", "produce", "electronics", etc.
  carbonKg: number;
  impactLevel: "low" | "medium" | "high";
  suggestedSwap?: string;
  swapSavingsKg?: number;
}

The dashboard aggregates this data using MongoDB's aggregation pipeline to compute monthly trends, category breakdowns, and impact distributions — all served from a single /api/v1/receipts/stats endpoint.

Prize Categories

• Best Use of Google Gemini — Gemini (gemini-2.5-flash) is the core engine powering all carbon analysis. Three specialized prompts handle image, text, and live camera inputs, with structured JSON output, embedded carbon reference data, and multimodal image understanding. Live climate data from NOAA is injected into every prompt so Gemini can ground its insights in current conditions. BYOK support lets users bring their own Gemini API key for unlimited use, and a layered error handler distinguishes rate limits, safety blocks, service overload, and quota exhaustion with tailored UI for each.
• Best Use of GitHub Copilot — The entire project was built with GitHub Copilot as an AI pair programmer, from scaffolding components and writing API routes to debugging rate limit issues, building the guest-first anonymous scanning architecture, and integrating live climate data from NOAA into the analysis pipeline.