DEV Community

Aditya Modani
Aditya Modani

Posted on

ExpenseAI — Privacy-First Local AI Expense Tracker

#devchallenge #weekendchallenge #hf26challenge

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

The Story: It Started With a ₹450 Coffee

When I saw the Hacktoberfest #BuildForAFriend challenge, I knew exactly what I was going to build. Because the idea was already sitting there, born from a simple ₹450 coffee.

A friend of mine had been trying to track his expenses for weeks. He genuinely wanted to build the habit, but one evening, after we grabbed coffee, I watched him pull out his phone to record the expense. He opened his budgeting app, and then came the rigid dropdowns: Category. Amount. Description. Date. Save.

He stared at the screen, sighed, and closed the app. "I'll add it later."

I knew what “later” meant. It meant the ₹450 would disappear into a mental pile of unrecorded expenses. The software was asking him to think like a database when he was simply trying to remember what he had spent.

So I asked him: “Why can't you just tell the app what you spent?”
He laughed, and then said something that stuck with me: “Because apps don't understand the way I talk.”

He was right. If he told me, "four fifty coffee pe spent kiya," I would immediately understand: ₹450. Coffee. Expense. But traditional software sees that sentence as a problem that needs to be squeezed into predefined fields—often failing on regional currency speech and turning "four fifty" into $4.50 instead of ₹450.

That's when I decided to build ExpenseAI: an expense tracker that adapts to the person, instead of forcing the person to adapt to the software.

What I Built

ExpenseAI is a 100% private, database-free, local-first financial intelligence tracker. You type or speak naturally (e.g., "four fifty CCD coffee pe spent kiya" or "got 25000 salary"), and an open-source LLM running locally via Ollama parses the input into structured data—keeping every rupee of your financial history strictly on your own device

Core Highlights:

  • Local AI Natural Language Parsing: Processes conversational queries using a local llama3.2 model managed by Ollama.
  • INR Spoken Currency Engine: Specifically trained via system prompts to interpret Indian spoken number formats correctly (e.g., "four fifty" $\rightarrow$ 450, "fifteen hundred" $\rightarrow$ 1500).
  • Strict JSON Schema Sampler: Enforces strict structural output directly at the model sampler level to eliminate schema drift and prevent parsing errors.
  • Resilient Offline Fallback: Includes a heuristic keyword parser that seamlessly takes over if Ollama is unreachable or times out during cold starts.
  • Database-Free Local Storage: Operates completely offline without external databases, storing state in browser localStorage and Express memory.
  • Private Client-Side Excel/CSV Import: Drag and drop bank statements (.xlsx, .csv) to parse transactions client-side using SheetJS without sending a single byte to a server.
  • Dynamic Analytics & One-Click Exports: Real-time category breakdown via Chart.js with high-res PNG/JPG chart image downloads and Excel-compatible CSV exports.

Demo

Code

GitHub logo Aditya4453 / DEVchallenge

ExpenseAI

ExpenseAI is a React and Express personal-finance dashboard. Enter an expense or income item in natural language, review monthly analytics, filter and sort transactions, and export the current view.

Transactions are stored locally:

  • The backend keeps an in-memory array while it is running.
  • The frontend mirrors transactions in browser localStorage, so browser reloads do not lose them.
  • Backend memory is cleared when the backend process restarts.

What it includes

  • English and Hinglish natural-language transaction input
  • Local Ollama parsing with strict JSON output
  • Keyword-based parser fallback when Ollama is unavailable
  • INR phrase handling such as four fifty -> 450
  • Monthly summary cards and Chart.js category breakdown
  • Category filters and newest/oldest/amount sorting
  • Excel-compatible CSV transaction export
  • Full PNG/JPG chart export with legend and totals
  • Responsive React UI with Framer Motion

The current React interface does not use voice/STT/TTS. Legacy ElevenLabs backend code remains available but is not part of the…

View on GitHub

How I Built It

ExpenseAI is built on a decoupled full-stack architecture designed for privacy, speed, and offline resilience:

  • Frontend: A responsive, interactive React UI scaffolded with Vite. It uses framer-motion for fluid micro-animations, Chart.js for dynamic visual analytics, and SheetJS for handling local CSV/Excel imports. Data is persisted securely in the browser's localStorage to ensure persistence across reloads.
  • Backend: A lightweight Node.js/Express server that acts as the middleware. It handles cross-origin requests, validates the incoming natural language prompts, and orchestrates the AI parsing.
  • AI Integration: The backend sends a meticulously crafted system prompt and strict JSON schema to a local instance of Ollama running llama3.2. If the model is not running, the Express API instantly routes the request to a regex-based heuristic fallback parser to ensure the dashboard remains fully functional.
  • Storage: No MongoDB, Postgres, or cloud database is required. The backend uses a volatile in-memory array for session continuity, heavily complementing the frontend's local storage solution.

Why Does Open Innovation Matter?

Personal financial logs represent highly sensitive data. Sending every daily meal, utility bill, or salary deposit to closed, proprietary cloud APIs creates unnecessary privacy risks, rate-limit constraints, and subscription costs.

Open innovation and open-weight models made ExpenseAI possible by enabling:

  1. Absolute On-Device Privacy: Financial data never leaves the user's browser or local machine.
  2. Zero API Cost & Unlimited Usage: Free, unlimited local inference without API keys, monthly bills, or rate limits.
  3. True Offline Independence: Fully functional without an active internet connection—allowing users to track expenses anywhere, anytime.

My Agent Session

We utilized AI agent workflows throughout the build to refactor the project's default models to llama3.2, handle git history and commits properly, test local model edge cases, and add debugging utilities directly into the codebase.

Team Members

Credit to my teammate who contributed to this project:

Prize Categories

  • Main Challenge: Build for a Friend
  • Best Use of Local Inference / Ollama

Top comments (0)