𝗝𝗼𝗵𝗻

Posted on Aug 16

CanisVision: See and Sense the World Through Your Dog's Eyes

#devchallenge #weekendchallenge #ai #webdev

DEV Weekend Challenge: Dog Days Edition Submission 🐕

This is a submission for Weekend Challenge: Dog Days Edition

What I Built

Have you ever looked at your dog sniffing a seemingly empty corner or staring into the backyard and wondered: What is my dog actually experiencing right now?

Dogs do not experience reality the way humans do. While we rely heavily on rich trichromatic color vision (RGB) and crisp details, dogs live in a completely different sensory universe:

Dichromatic Vision: Dogs only have two color cones—blue (~429nm) and yellow (~555nm). Reds, greens, and oranges collapse into subtle shades of yellow, tan, and gray.
Visual Acuity of ~20/75: What a human sees clearly at 75 feet, a dog only resolves at 20 feet (relying instead on motion detection and low-light tapetum lucidum reflection).
Olfactory Dominance: With up to 300 million scent receptors, every room is a complex topographical map of fresh food volatiles, human skin cell plumes, and old trails.
Ultrasonic Hearing: Dogs hear frequencies up to 65,000 Hz, detecting appliance electrical coil whines, fluorescent light hums, and pipe friction that are completely inaudible to humans.

I built CanisVision—a real-time canine sensory simulation studio and environmental telemetry dashboard. It merges biophysical optical algorithms with Google Gemini Multimodal AI to deconstruct any photo or live environment from a dog's perspective.

Key Features:

👁️ Dual-Spectrum Split Canvas: Drag an interactive slider between human full-color vision and authentic canine dichromacy with realistic 20/75 acuity blur.
🔬 Spectrum Loupe Probe (Key L): Hover anywhere on an image to inspect the exact nanometer wavelength and biological cone response.
🧠 Dog Mind Stream (Internal Monologue): First-dog stream-of-consciousness thoughts generated by Google Gemini with English speech voice synthesis (Web Speech API).
👃 Super-Sniffer Scent Vectors: Interactive spatial hotspots pinpointing food volatiles, human pack pheromones, and neighborhood newsposts.
👂 Acoustic Ultrasound Simulator: Web Audio API frequency sweep synthesis (40Hz to 65kHz) uncovering hidden domestic noise stressors.
🛡️ Paws & Safety Audit: AI detection of slippery joint hazards, toxic house plants, and choking risks with tailored enrichment games.
🎛️ Zero-Friction Evaluator Mode: 5 pre-analyzed scenario fixtures ready to test instantly with 1-click without requiring an API key.

Demo

🌐 Live Web Application: https://johnnylemonny.github.io/canisvision/

Interactive Desktop Cockpit (Obsidian Dark Mode)

Veterinary Laboratory View (High-Contrast Light Mode)

Mobile-First Single Column Workstation (iPhone / Pixel) & Science Guide

Mobile Responsive Layout	Canine Biophysics Guide

Code

johnnylemonny / canisvision

🐾 CanisVision: Canine senses simulation & environmental telemetry studio powered by Google Gemini Multimodal AI. Built for DEV.to Weekend Challenge (Dog Days Edition).

🐾 CanisVision

Canine Senses Simulation & Behavioral Telemetry Studio

Ever wondered how your dog truly experiences the world?
CanisVision bridges canine biophysics and multimodal artificial intelligence to reveal the hidden sensory realm of dogs: dichromatic vision (blue-yellow spectrum), high-frequency ultrasound acoustics, invisible scent pathways, and an AI-powered internal monologue.

Live Application: https://johnnylemonny.github.io/canisvision/
Author: johnnylemonny

🎬 Animated Video & Studio Showcase

⚡ 9-Second Fast-Paced Visual Teaser

Demonstrating real-time Brettel-Neitz dichromacy, 20/75 visual acuity blur, and Google Gemini Multimodal telemetry.

High-definition MP4 available at public/canisvision-showcase.mp4

🎛️ Desktop Studio Cockpit (Obsidian Dark Mode)

Anchored dual-spectrum sensory canvas with independent scrolling panels for environmental controls and real-time AI telemetry.

☀️ High-Contrast Light Mode

Clean, accessible veterinary laboratory aesthetic.

📱 Mobile Single-Column Layout (iPhone 16 Pro Max / Pixel 10)

🔬 Canine Biophysics & Neuro-Sensory Guide

🌟 Why CanisVision?

Humans live in a visual world rich in reds, greens, and sharp static details. Dogs, however…

View on GitHub

Repository: https://github.com/johnnylemonny/canisvision
License: PolyForm Noncommercial 1.0.0
Tech Stack: React 19 (19.2.8), TypeScript 7 (7.0.2), Vite 8 (8.2.1 Rolldown engine), Tailwind CSS v4 (4.3.3), Lucide React, Web Audio API, Web Speech API, Remotion (4.0.512), @google/genai (2.17.1).

How I Built It

CanisVision was engineered with a privacy-first, 100% client-side architecture.

1. Biophysical Dichromatic Vision Engine

Canine color perception is computed via a real-time mathematical shader implementing the Brettel-Neitz dichromacy model:

Input sRGB pixels are converted to linear RGB space.
Transformed through the physiological human $3 \times 3$ cone matrix (Long, Medium, Short wavelengths).
Projected onto the dichromatic half-plane defined by canine S-cones (~429nm) and M-cones (~555nm).
Converted back into display color space with an optical blur convolution filter simulating natural canine static acuity (~20/75).

2. Google Gemini Multimodal Telemetry Pipeline

When a user uploads a photo of their living room, backyard, or kitchen, CanisVision sends the image to Google AI's newest gemini-3.5-flash-lite (or flagship gemini-3.7-flash) using the @google/genai SDK.

The prompt acts as a veterinary ethologist and canine biophysicist, structured with strict JSON Schema constraints:

const response = await ai.models.generateContent({
  model: 'gemini-3.5-flash-lite',
  contents: [
    {
      role: 'user',
      parts: [
        { text: promptWithBreedInstincts },
        { inlineData: { mimeType, data: base64ImageData } },
      ],
    },
  ],
  config: {
    responseMimeType: 'application/json',
    systemInstruction: canineSensorySystemInstruction,
    temperature: 0.25,
  },
});

Gemini analyzes spatial cues, textures, plants, appliances, and room ergonomics to produce:

2D Scent Vectors: Where food odors, human scents, or decay linger.
Acoustic Stress Hotspots: Pinpointing ultrasonic inverter whine in refrigerators, transformer hums, and outdoor traffic.
Internal Monologue: Characterful stream-of-consciousness thoughts adapted to the selected dog breed (Golden Retriever, Beagle, Border Collie, German Shepherd).
Actionable Hazard Audits: Detecting slippery floors, toxic plants, and choking hazards.

3. Web Audio Ultrasound & Tone Sweep Synthesis

To help pet owners understand what high frequencies sound like, CanisVision includes an interactive Web Audio tone generator synthesizing frequencies across the canine hearing range (40Hz to 65,000Hz), demonstrating ultrasonic dog whistle pitches and mechanical drone stress factors.

4. Privacy & Safe Client-Side BYOK Vault

All API keys are encrypted client-side using entropy-salt masking and stored exclusively in the user's browser localStorage. No keys or images ever touch an intermediary server.

Prize Categories

⚡ Best Use of Google AI

CanisVision leverages Google's multimodal models (gemini-3.5-flash-lite and gemini-3.7-flash) as the core cognitive engine:

Spatial Multimodal Vision: Gemini maps coordinates $(x, y)$ onto physical scene objects (sofas, windows, plants, food counters) to project invisible scent clouds and acoustic zones.
Breed-Conditioned Personality Reasoning: Gemini adapts the cognitive monologue, heart rate prediction, and arousal levels based on breed traits (Prey Drive, Scent Focus, Noise Sensitivity).
Structured JSON Telemetry: Zero hallucinated wrappers—100% clean schema output powering the real-time radar dashboards and exportable behavior reports.

Built with ❤️ for dogs and developers by johnnylemonny.

DEV Community