DogeVault Stylist — AI Dog Fashion Stylist

This is a submission for Weekend Challenge: Dog Days Edition

What I Built

DogeVault Stylist is a Next.js app that turns a single photo of your dog into a full shopping experience. You upload a dog photo, and the app:

1. Enhances it into a studio portrait (YouCam)
2. Generates a breed profile and style recommendations (Gemini)
3. Fetches real, live products from Amazon, eBay (Apify MCP)
4. Virtually tries the outfit on your dog using image generation (Qwen)
5. Reads the stylist's reaction aloud (ElevenLabs)
6. Lets you approve a purchase through a budget-controlled Solana vault (untested)
7. Mints a compressed NFT receipt on-chain (untested)

The goal was to build something that feels like a real product — not a demo with hardcoded data. Every product card links to a live listing. Every transaction hits Solana devnet. Every voice line is synthesized.

Demo

First Demo

Second Demo

First demo has an error at the final process of virtual tryout. I added this second demo to show end to end process without error. Will let the judge decide if it can be used as it is added after deadline.

Code

Gungz / DogDaysHack

DogeVault Stylist

AI dog stylist + autonomous shopper for the Dog Days hackathon. Upload a dog photo, get a YouCam studio portrait, hear an ElevenLabs stylist reaction, fetch real products through Apify MCP, and approve a budget-limited USDC transfer on Solana devnet.

Stack

• Next.js 16 + TypeScript + Tailwind (apps/web)
• Anchor 0.32.1 Solana program (programs/doge_vault)
• YouCam Perfect Corp API: photo enhance, background replacement, and human virtual try-on (cloth-v3, hat, shoes — YouCam is human-only)
• Gemini: dog profile, stylist script, and product search planning (text only; free-tier keys have no image-generation quota)
• Qwen / Alibaba DashScope (qwen-image-3.0-pro): dog virtual try-on via image editing with reference images (preferred when DASHSCOPE_API_KEY is set)
• Apify MCP: apify/e-commerce-scraping-tool
• ElevenLabs: stylist voice
• Supabase: dog image storage + NFT receipt metadata (Metaplex cNFT points to the metadata URI)

Repository layout

apps/web                  Next.js app
  src/lib                 Server/client libraries
  src/app/api/agent        Upload

…

View on GitHub

The repo is a pnpm monorepo:

• apps/web — Next.js 16 frontend and API routes
• programs/doge_vault — Anchor 0.32.1 Solana program
• adk-apify-sample/ — Standalone Google ADK agent (Python)

How I Built It

The agent pipeline

Everything starts with one API call to /api/agent. The server-side orchestrator chains six providers in sequence, streaming progress back to the UI via SSE:

Upload → YouCam enhance → YouCam background replace → Gemini profile
→ Apify product search → Virtual try-on → ElevenLabs voice

Each step is isolated in its own module under apps/web/src/lib/, so a missing API key degrades gracefully instead of crashing the whole pipeline. The UI shows a status badge per provider (ok / missing_key / error).

Real product data with Apify MCP

This was the hardest part to get right. The Apify MCP server solved my product extraction problem. It exposes Apify Actors as tools over the Model Context Protocol. I call the apify/e-commerce-scraping-tool Actor from my Next.js backend using the official MCP TypeScript SDK:

const client = new Client({ name: "dogevault-stylist", version: "0.1.0" });
const transport = new StreamableHTTPClientTransport(
  new URL("https://mcp.apify.com/?tools=apify/e-commerce-scraping-tool,get-actor-run,get-dataset-items"),
  { requestInit: { headers: { Authorization: `Bearer ${APIFY_TOKEN}` } } }
);
await client.connect(transport);

Actor runs are async — the server returns a runId, and I poll get-actor-run until SUCCEEDED, then read get-dataset-items. The whole flow returns normalized product cards with images, prices, ratings, and direct URLs from Amazon, eBay, and other marketplaces.

Google AI (Gemini) — the brain of the stylist

Gemini handles three jobs:

1. Dog profile generation. Given the uploaded photo, Gemini identifies breed, size, coat type, and generates a style profile with color palette and fashion recommendations.
2. Stylist script. Gemini writes a short, personality-filled reaction to the dog's look — this becomes the ElevenLabs voice line.
3. Product search planning. Gemini generates the search queries that get passed to Apify (e.g., "small dog harness poodle" or "winter coat chihuahua").

The profile also tags the subject as dog or human, which determines which virtual try-on path the app takes downstream.

Virtual try-on (hybrid approach)

YouCam's virtual try-on API is human-only (I just knew about it during development) — it works great for clothes, hats, and shoes on people, but can't dress a dog. So I built a hybrid:

• Human photo detected: YouCam Virtual Try-On (cloth-v3, hat, shoes) with the Apify product image as the garment reference.
• Dog photo detected: Qwen image generation (qwen-image-3.0-pro) edits the dog portrait to wear the selected product, using both the dog photo and product image as references.

ElevenLabs voice

The stylist script from Gemini gets sent to ElevenLabs for text-to-speech. One gotcha: free ElevenLabs plans can't use library voices via the API (returns HTTP 402 paid_plan_required). The app catches this and falls back to the browser's built-in SpeechSynthesis — so the feature still works, just with a less polished voice.

Solana budget-controlled vault

The on-chain piece is an Anchor program that acts as a spending guardrail:

• initialize_vault creates a PDA vault with hard-coded single-item and total spend limits.
• The user funds the vault with USDC (devnet).
• approve_spend requires the user's signature, transfers the approved amount to the treasury, and enforces both limits on-chain.

This means even if the AI recommends a $500 Gucci dog collar, the vault won't let it through. After approval, the app mints a compressed NFT (cNFT) via Metaplex Bubblegum V2 as a receipt — containing the portrait, try-on image, product details, and transaction reference.

Prize Categories

Best Use of ElevenLabs — The AI stylist's reaction script (generated by Gemini) is converted to speech via the ElevenLabs text-to-speech API. The app handles the free-plan 402 error gracefully, falling back to browser SpeechSynthesis so the voice feature always works.

Best Use of Google AI — Gemini powers three core features: dog breed/style profiling from the uploaded photo, stylist script generation for the voice reaction, and product search query planning that feeds the Apify scraper. Gemini is the decision-making layer that ties the entire pipeline together.