This is a submission for the Gemma 4 Challenge: Build with Gemma 4

Note: This project is aimed at solving a critical social issue in Brazil, but we are submitting it in English to align with the global nature of the Dev.to Challenge. Our ultimate goal is to localize the final platform to Brazilian Portuguese (pt-BR) to serve our local communities.

What I Built

GemmaFin - Breaking the Cycle of Debt with Conversational AI

In 2026, Brazil is facing a structural household debt crisis. A staggering 80.9% of families are in debt, and a large portion of the population is trapped in a cycle of high-interest credit card revolving debt and the rising epidemic of unregulated online gambling ("Bets"). Traditional budgeting apps have failed to solve this because they require a high level of financial literacy and impose severe cognitive friction—forcing exhausted workers to manually type numbers, navigate drop-down menus, and categorize every penny.

This crisis is severely exacerbated by historical gaps in the public education system. Millions of vulnerable Brazilians lack basic financial literacy, making it nearly impossible for them to decode complex banking jargon, comprehend the crushing compound interest rates applied to their debts, or make informed strategic decisions about their money.

GemmaFin subverts this paradigm. Built specifically for low-income families and informal workers, GemmaFin is a Progressive Web App (PWA) that mimics a simple messaging interface. In Brazil, WhatsApp is more than an app; it is the default operating system for communication. Studies show that over 99% of smartphones in the country have it installed. Even individuals with low digital literacy use it daily with ease. By adopting this highly familiar, chat-based UI, GemmaFin removes the steep learning curve of traditional financial apps, ensuring true accessibility.

Instead of filling out spreadsheets, the user simply "talks" to the app:

• Audio Inputs for the Informal Economy: A user can send a voice memo saying, "I did some electrical work today and got 200 Reais on Pix, but spent 50 at the barber."

• Visual Inputs for Messy Realities: A user can snap a photo of a crumpled, faded grocery receipt or upload a screenshot of their bank statement.

• Real-Time Point-of-Sale Advisor: Users frequently face complex daily dilemmas, such as deciding whether to buy an appliance in 12 installments or pay cash for a 5% discount. Users can ask GemmaFin in real-time, and the AI will analyze their current liquidity, the discount offered, and potential yield if the money were invested, delivering a simple, mathematically sound recommendation.

• Debt Rescue & Refinancing: When GemmaFin detects that a user is about to enter high-interest revolving credit (which can exceed 400% annually in Brazil), it proactively intervenes. The AI acts as an "Agentic Nudge," suggesting safer alternatives, like taking a lower-interest personal loan to pay off the credit card, acting as a real-time financial shield.

The AI autonomously parses these unstructured multimodal inputs, categorizes the income and expenses, updates the local ledger, and generates simplified visual dashboards to prevent insolvency before it happens.

Demo

Code

vfcarida / Gemma-4-Challenge_finance

Gemma-4-Challenge_finance

GemmaFin - Breaking the Cycle of Debt with Conversational AI

An AI-powered personal finance Progressive Web App (PWA) specifically designed for low-income Brazilian families and informal workers. It leverages the multimodal power of Gemma 4 E2B to turn financial management from a burden into a conversation.

---

📖 Overview

In 2026, Brazil is facing a structural household debt crisis. A staggering 80.9% of families are in debt, and a large portion of the population is trapped in a cycle of high-interest credit card revolving debt and the rising epidemic of unregulated online gambling ("Bets"). Traditional budgeting apps have failed to solve this because they require a high level of financial literacy and impose severe cognitive friction—forcing exhausted workers to manually type numbers, navigate drop-down menus, and categorize every penny.

This crisis is severely exacerbated by historical gaps in the public education system. Millions of vulnerable Brazilians lack basic financial literacy…

View on GitHub

How I Used Gemma 4

To build a financial assistant for vulnerable populations, two things are absolutely non-negotiable: Privacy and Zero-Friction Multimodality. For these reasons, the Gemma 4 E2B (Effective 2 Billion) model was the perfect and only logical fit for this architecture.

Here is how Gemma 4 E2B powers the core of GemmaFin:

• On-Device Privacy (Zero Data Exposure): Financial data (bank statements, income audio memos) is highly sensitive. Sending this data to the cloud is a massive privacy risk. Because the E2B model is optimized for edge devices and runs locally with an incredibly small memory footprint, all inference happens completely on-device. The user's financial life never leaves their phone.

• Native Audio & Vision Understanding: Classical OCR fails miserably on crumpled, poorly lit receipts. Gemma 4 E2B natively supports text, high-resolution images, and raw audio. The model directly listens to the user's voice memos about informal jobs and reads bank screenshots end-to-end without relying on fragile, third-party speech-to-text or OCR pipelines.

• Agentic Workflows via <|think|> and Function Calling: GemmaFin doesn't just chat; it acts. Using Gemma 4's native <|think|> reasoning mode, the model mathematically calculates the user's new balance internally before responding. It then leverages native structured JSON function calling to autonomously trigger the local update_ledger function. This transforms unstructured real-world chaos into clean, categorized SQLite/LocalStorage database entries entirely in the background.

Key Features

GemmaFin is a complete interactive financial assistant capable of performing historical analysis and answering complex questions in natural language.

🚀 Core Functionalities

• Multimodal Processing (Simulated Gemma 4): GemmaFin is no ordinary chat. It simulates the advanced orchestration of Gemma 4 to process different types of media:
  • Voice (Audio-to-Action): Record earnings and expenses via voice. The AI transcribes, categorizes, and updates your balance instantly.
  • Vision (Bank Statement OCR): Send a screenshot of your bank statement or a photo of a receipt. The AI automatically identifies transactions, saving the user from manual typing.
  • Exposed Reasoning (<|think|>): Animated thought blocks show the AI's "step-by-step" logic (transcription -> extraction -> calculation), increasing transparency and user trust.
• Interactive Data Visualization: We transform dry numbers into easy-to-understand visual insights:
  • Comprehensive Monthly Report: Comparative bar charts (Income vs. Expenses) showing the evolution over the last 3 months.
  • Trend Analysis: The AI detects and warns about price variations in critical categories like Groceries and Gas.
  • Budget Goals: Circular progress indicators that help the user stay within the limits set for the month.
• "WhatsApp-First" Conversational Experience:
  • Familiar Interface: Design optimized for the Brazilian user, using visual patterns to lower the learning curve.
  • Smart Suggestions (Quick Replies): Quick response buttons that guide the user to the next logical actions.
  • Personalized Tips: Savings advice based on the actual consumption profile, such as brand substitution suggestions or service optimization.

🧠 Brain with Historical Context

Implemented a simulated database with 3 months of financial history (March, April, May). When you send a photo of a bank statement, GemmaFin doesn't just read the expense, it compares it to last month: "Last month you spent R$ 72.00 at the grocery store... an 11% increase."

⚙️ Smart Engine

Replaced generic messages with an intent-understanding engine. You can ask:

• "How much did I spend on groceries?" → Opens the new Comparison Card.
• "View month summary" → Opens the Comprehensive Monthly Report.
• "Set goal" → Shows goal progress by category.
• "Savings tips" → Generates personalized advice based on the low-income profile.

📱 Visualization Components

• FullReportCard: Grouped bar chart (Income vs. Expenses) for the last 3 months and trend table by category.
• ComparisonCard: Comparative horizontal bars for specific categories.
• BudgetGoalCard: Circular progress indicators for spending goals.
• Quick Replies: Suggestion chips to facilitate navigation after each response.
• Typing Indicator: "GemmaFin is typing..." animation to make the interaction more human.