VoxCommerce: Multi-Modal AI for Resellers

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

I built VoxCommerce, a hands-free, voice-to-listing multi-modal platform built specifically for micro-retailers, Etsy crafters, and eBay flippers who struggle with manual data entry or suffer from physical or visual impairments.
Instead of spending 20 minutes typing out SEO keywords, sizing details, and formatting text for a single item, a seller simply snaps a few photos and speaks naturally about their product ("Hey, this is a hand-knitted vintage-style teal cardigan, super chunky wool, size medium, slight fraying on the left cuff..."). VoxCommerce instantly blends the audio narrative with the visual data to output highly optimized, marketplace-ready titles, descriptions, pricing suggestions, and search tags.

Who Would Pay For It?

Independent side-hustlers and full-time vintage resellers who manage high volumes of unique inventory. For these sellers, time is literally money. They would gladly pay a subscription fee ($15–$25/month) for a tool that reduces their listing time from minutes to seconds, turning their physical inventory backlog into live storefront cash flow.

Demo

✨ Live App: https://voxcommerce.tech
📺 Video Walkthrough: [Link to your Loom/YouTube demonstration video showing the image upload, voice processing, and final generated marketplace text]

Code

The complete open-source codebase, including the frontend UI and the backend multi-modal inference pipeline, is hosted on GitHub:

How I Built It

VoxCommerce is engineered entirely around an open-source, multi-modal AI stack designed to minimize API dependencies and maintain extreme architectural agility.
• Architecture Flow: The user interacting with the React Frontend (hosted on Render via a .tech Domain) uploads an image and audio file. This media bundle hits our FastAPI Backend running inside a Dockerized container hosted on a DigitalOcean GPU Droplet. Audio Transcription is handled via the ElevenLabs API, and the text transcription along with the image is passed directly to an instance of Gemma Vision running locally via Ollama/vLLM for final e-commerce generation.
• The Brain (Open-Weight Model): I deployed Gemma (using the vision-capable weights) via an optimized local inference engine (Ollama/vLLM) running on the backend. It takes the structured transcription alongside the raw images to extract visual characteristics (color, condition, texture) that the user might have forgotten to mention in their audio clip.
• The Ears: I integrated ElevenLabs to handle the speech-to-text processing, flawlessly converting messy, conversational, multi-dialect audio into high-fidelity text streams for our context window.
• The Cloud Pipeline: The frontend React application is securely deployed on Render. The heavy-lifting backend pipeline runs inside a Dockerized container on a DigitalOcean GPU Droplet, ensuring sub-second inference token generation.

Why Does Open Innovation Matter?

Open innovation isn't just a design choice for VoxCommerce—it is the foundational reason the business model works.

1. Marginal Cost to Zero: Relying on commercial, closed-source multi-modal APIs forces a predictable, linear cost per API call. For a high-volume seller uploading 500 items a month with multiple high-res photos, commercial API bills eat up the software profit margins entirely. By running an open-weight model like Gemma on dedicated architecture, our cost-per-listing drops asymptotically to zero as usage scales.
2. Niche Fine-Tuning Capability: Closed-source models are tuned to general-purpose assistant tasks. Because Gemma is open, I can easily fine-tune the model parameters on historical, high-performing e-commerce metadata (such as top-converting eBay headers or fashion-specific taxonomy) to output far superior copy than a generic corporate API can generate out of the box.
3. No Vendor Lock-In: We aren't vulnerable to sudden pricing changes, tier limitations, or sudden deprecations from corporate platform providers. We own our intelligence engine.

My Agent Session

I used DevRelay to log and map out the orchestration sessions while debugging the asynchronous audio/video data matching pipeline. You can review the complete development process here:
[Link to your DevRelay Agent Session]

My First User

To validate the "Side Hustle" commercial viability, I put a beta version of VoxCommerce in front of Sarah, a local vintage clothing vendor who runs an online Depop and Etsy store.
• What she did: She tested it live with a batch of 5 vintage sweaters.
• What she said: "Usually, doing descriptions makes me want to pull my hair out because I hate typing on my phone while measuring garments. I just spoke out loud while measuring the sleeves, and the app spit out a listing that looked better than what I write."
• Did she pay? She offered to pay $20 on the spot to keep access to the beta link for the rest of October, proving direct commercial validation!

Prize Categories

I am proudly entering VoxCommerce into the following Hacktoberfest challenge prize categories:
• Best Use of Gemma: For utilizing Google's open-weight model to orchestrate multi-modal listing logic.
• Best Use of DigitalOcean: For hosting our core AI models and inference pipeline on a GPU Droplet.
• Best Use of Render: For hosting our user-facing front-end client.
• Best Use of ElevenLabs: For processing fluid, conversational marketplace audio narratives into accurate data text.
• Best Use of a .tech Domain: For putting our live app production build on voxcommerce.tech.