Fetch - scan your dog, then chat, plan, and hear their story

This is a submission for Weekend Challenge: Dog Days Edition

What I Built

Fetch is an AI dog companion that starts with a single photo and turns it into a small toolkit for looking after your dog.

You upload a photo, and Fetch identifies the most likely breed along with temperament, size, energy level, and a few care tips. From there, everything stays connected to that same dog:

• A chat assistant that already knows the breed, so the advice you get is specific to your dog instead of generic. After a scan it also suggests breed-specific questions about grooming, feeding, exercise, environment, training, and health.
• A one-tap care plan covering feeding, grooming, exercise, environment, training, and vet care, which you can read in the app, download as a PDF, or print.
• A storybook that writes a short story starring your dog and reads it aloud in a natural voice.

The idea was to make something that feels less like four separate tools and more like one companion that actually remembers your dog across the whole app. I wanted a new dog owner to be able to point their camera at their pup and immediately have somewhere to ask questions and get a plan.

Demo

Live app: https://fetch-dog-ai.vercel.app/

Try it by uploading a clear photo of a dog on the Scan tab, then move through Ask Fetch, Care Plan, and Storybook. The storybook narration plays right in the browser.

Code

https://github.com/amolbhandari52/fetch-dog-ai

How I Built It

The frontend is a single HTML file with plain JavaScript and CSS. No framework, no build step, so it loads fast and works the same on a phone as on a laptop. The scanned dog's profile is held in a small shared state object, which is what lets the chat, care plan, and storybook all stay aware of the same dog.

All the AI calls go through serverless functions on Vercel so that the API keys stay on the server and never reach the browser. There are two functions: one for Google Gemini and one for ElevenLabs.

A couple of decisions I am happy with:

Rather than hard-code a Gemini model name, the function asks the API which models the key can use and picks a current one. I ran into a model being retired mid-build, so I made it self-correct instead of breaking. It also retries with backoff and falls back to another model when Gemini returns a temporary high-demand error, which kept the care plan and storybook reliable during testing.

For structured output like the breed profile and the care plan, I use Gemini's JSON response mode so the app can render the results directly instead of parsing loose text.

The storybook narration is generated per page and streamed back as audio. The ElevenLabs function auto-selects an available voice on the account, so it works without me hard-coding a voice ID.

Best Use of Google AI - Gemini handles four separate jobs: vision breed identification, the breed-aware chat, the structured care plan, and the story generation, all through one endpoint.

Best Use of ElevenLabs - the storybook narration is powered by ElevenLabs text to speech, which is what turns the generated story into something you actually listen to.