ScamSloth 🦥 listens to your voicemails so you don't have to

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

I never listen to my voicemails. Half of them are scams, and the other half I worry might be. ScamSloth does the listening for me. Upload a voicemail and in about 4 seconds you get a Scam / Not scam verdict with red flags, a check on whether the callback number is reported as a scam online, and the full transcript. No listening and no googling numbers.

Demo

Live: https://scamslothis.mildlyintelligent.tech

Code

https://github.com/bauji/scamslothis

How I Built It

• Qwen3.5-4B (open-weight) on Tinker is the brain. It reads the transcript and returns a JSON verdict with up to 5 red flags. Qwen3.5-4B is released under Apache 2.0, so I'm free to fine-tune it, self-host it, and ship my own version.
• ElevenLabs Scribe turns the audio into text.
• SerpApi searches the callback number for scam reports. ScamSloth pulls the number out of the transcript even when it's spoken as words ("four one six…").
• FastAPI, managed with uv, deployed on Render at my own .tech subdomain.

I built it in small steps, each one leaving a working app: first a page, then the transcript, then the verdict, then the number check.

Why Does Open Innovation Matter?

A scam checker has to know about your life. While building ScamSloth, I tested it on my own voicemails, and it flagged one of my legitimate ones as a scam. The base model (Qwen3.5-4B) had no context on my life. If it had known my work history and which financial institutions I use, it wouldn't have flagged that call. The funny part is that the exact same voicemail sent to you could very well be a scam.

Generic models are great at a lot of things, but they don't know my data or my life. I see a future where intelligence works for everyone: open models with open weights, fine-tuned on each person's own data. Everyone gets intelligence customized for them, and without ads 😉.

To be upfront: transcription (ElevenLabs) and the number search (SerpApi) are closed services today. The roadmap swaps in open speech-to-text and local inference.

My Agent Session

The Lazy Payoff

I ran it on 4 of my own voicemails this week. Each took about 4 seconds instead of listening and googling the number. It caught both spam calls, but it also flagged one real call, and that's exactly the mistake my fine-tune is going to fix.

Prize Categories

• Best Use of Render: the whole app (FastAPI front end and AI pipeline) runs on Render.
• Best Use of ElevenLabs: Scribe transcribes the voicemail audio for the open-weight model.
• Best Use of SerpApi: live web search checks whether the callback number is reported as a scam.
• Best Use of a .tech Domain: live at scamslothis.mildlyintelligent.tech.

What's Next

• Fine-tune on my own labelled voicemails and score it against the base model
• Turn it into an agent that decides when to look up a number
• Open speech-to-text and local inference