DEV Community: Aditya Modani

I Told My Laptop “attendance” and It Answered Out Loud: A Local, Voice-Powered Portal Agent

Aditya Modani — Sat, 24 Oct 2026 16:23:05 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

Attendance Monitor is a voice-enabled command-line agent that does the boring portal chores for me. You give it a task name, and it opens the website, logs in, finds the right page, reads it, and tells you the answer out loud like a person would.

The first task is attendance. I type one word:

attendance

and it replies:

"Hey, good news! Your overall attendance is eighty six percent. You're above seventy five percent in every course."

Attendance is only the first use, though. The agent has two parts: a fixed browser routine (login, navigate, screenshot) and a reading step (a local vision model plus a spoken-style reply). Both are driven by a config file with the URL, the click path and the question to ask. Pointing it at a different task means changing the config and the question, not rewriting the agent. Examples:

Fee or exam-form deadlines
Timetable or room changes
Assignment and results pages
Any dashboard you check every day

Who it saves effort for: anyone who repeats the same login, scroll, click, read routine on a portal, and anyone who'd rather hear the answer than open the page.

Demo

Code

Aditya4453 / attendance_monitor

attendance-agent 📊

An automated Python CLI agent that logs into your TCS iON portal, navigates to the Period wise attendance page, captures a full-page screenshot, and uses a local Ollama vision model (such as qwen2.5vl or llava) to extract and summarize your attendance statistics with a clean terminal UI powered by Rich.

📁 Project Structure

attendance-agent/
├── .env                 # Your portal credentials (NEVER commit this)
├── .env.example         # Template with variable names
├── .gitignore           # Ignores credentials, screenshots, logs, and session files
├── config.yaml          # Portal URL, CSS selectors, model name, screenshot settings
├── pyproject.toml       # Defines the `attendance` global console command
├── agent/
│   ├── __init__.py
│   ├── cli.py           # Typer CLI application & Rich output formatter
│   ├── browser.py       # Playwright browser automation (session persistence, lazy scroll, screenshot)
│   ├── vision.py        # Pillow image shrinking & Ollama vision model API integration
│   └── models.py        # Pydantic AttendanceSummary

…

View on GitHub

attendance_monitor/
├── agent/
│   ├── cli.py        # the `attendance` command
│   ├── browser.py    # login, navigation, screenshot
│   ├── vision.py     # image prep + local model call
│   ├── speech.py     # turns results into a casual message
│   ├── voice.py      # ElevenLabs text-to-speech
│   └── models.py     # Pydantic result schema
├── config.yaml       # URL, click path, selectors, model name
└── .env              # credentials and API keys (never committed)

How I Built It

Job	Tool
Browser automation	Playwright
Reading the screenshot	Open-weight vision model running locally through Ollama ([Moondream])
Image prep	Pillow
Voice	ElevenLabs text-to-speech
CLI	Typer
Output validation	Pydantic
Logging	loguru

The flow:

attendance starts the run.
Playwright logs in with credentials from a separate .env file and reuses the saved session next time.
It follows the click path from config.yaml, scrolls to load the page, and takes a screenshot.
Pillow shrinks the image for faster inference.
The local vision model reads it and returns structured JSON.
Pydantic validates the JSON, and a formatter turns it into a short, friendly message.
ElevenLabs speaks the message. --mute skips this step.

What I learned:

Playwright can't mix CSS and text= selectors in one comma list, so combining locators with .or_() is the fix.
College portals often use iframes, so content has to be found inside the frame.
Scraped summary cards can look like course rows, so I validate the data before speaking it.
Text meant for speech has to be written differently, with percentages as words and no symbols or tables.

Why Does Open Innovation Matter?

A student portal holds personal data: logins, grades, attendance. Because the vision model is open-weight and runs locally, the screenshot of my logged-in portal never leaves my laptop. A closed vision API would have meant uploading that screenshot to someone else's server on every run, with per-image costs and rate limits.

Open models also let me:

Swap models freely and pick the smallest one that reads the page well.
Run offline, since the reading step needs no internet beyond the portal.
Adapt the tool to any task, because I control the whole pipeline.

The only cloud piece is the optional ElevenLabs voice, which receives just the short text reply and never the screenshot.

The Lazy Payoff

Before: about [X] minutes per check, [Y] times a week, from login to scrolling to squinting at a table.
After: one command and about [Z] seconds, with the answer read aloud while I do something else.
That's about [total] minutes saved a month, and I no longer forget to check.

Prize Categories

Best Use of ElevenLabs: the agent's replies are written for speech and read out through ElevenLabs text-to-speech.
Open-weight models: the screenshot is read by an open-weight vision model running locally through Ollama.
Overall

ExpenseAI — Privacy-First Local AI Expense Tracker

Aditya Modani — Mon, 05 Oct 2026 06:14:24 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

The Story: It Started With a ₹450 Coffee

When I saw the Hacktoberfest #BuildForAFriend challenge, I knew exactly what I was going to build. Because the idea was already sitting there, born from a simple ₹450 coffee.

A friend of mine had been trying to track his expenses for weeks. He genuinely wanted to build the habit, but one evening, after we grabbed coffee, I watched him pull out his phone to record the expense. He opened his budgeting app, and then came the rigid dropdowns: Category. Amount. Description. Date. Save.

He stared at the screen, sighed, and closed the app. "I'll add it later."

I knew what “later” meant. It meant the ₹450 would disappear into a mental pile of unrecorded expenses. The software was asking him to think like a database when he was simply trying to remember what he had spent.

So I asked him: “Why can't you just tell the app what you spent?”
He laughed, and then said something that stuck with me: “Because apps don't understand the way I talk.”

He was right. If he told me, "four fifty coffee pe spent kiya," I would immediately understand: ₹450. Coffee. Expense. But traditional software sees that sentence as a problem that needs to be squeezed into predefined fields—often failing on regional currency speech and turning "four fifty" into $4.50 instead of ₹450.

That's when I decided to build ExpenseAI: an expense tracker that adapts to the person, instead of forcing the person to adapt to the software.

What I Built

ExpenseAI is a 100% private, database-free, local-first financial intelligence tracker. You type or speak naturally (e.g., "four fifty CCD coffee pe spent kiya" or "got 25000 salary"), and an open-source LLM running locally via Ollama parses the input into structured data—keeping every rupee of your financial history strictly on your own device

Core Highlights:

Local AI Natural Language Parsing: Processes conversational queries using a local llama3.2 model managed by Ollama.
INR Spoken Currency Engine: Specifically trained via system prompts to interpret Indian spoken number formats correctly (e.g., "four fifty" $\rightarrow$ 450, "fifteen hundred" $\rightarrow$ 1500).
Strict JSON Schema Sampler: Enforces strict structural output directly at the model sampler level to eliminate schema drift and prevent parsing errors.
Resilient Offline Fallback: Includes a heuristic keyword parser that seamlessly takes over if Ollama is unreachable or times out during cold starts.
Database-Free Local Storage: Operates completely offline without external databases, storing state in browser localStorage and Express memory.
Private Client-Side Excel/CSV Import: Drag and drop bank statements (.xlsx, .csv) to parse transactions client-side using SheetJS without sending a single byte to a server.
Dynamic Analytics & One-Click Exports: Real-time category breakdown via Chart.js with high-res PNG/JPG chart image downloads and Excel-compatible CSV exports.

Demo

GitHub Repository: Aditya4453/DEVchallenge
Live Demo Link: https://de-vchallenge.vercel.app/ (Only for frontend)
Visuals:

Code

Aditya4453 / DEVchallenge

ExpenseAI

ExpenseAI is a React and Express personal-finance dashboard. Enter an expense or income item in natural language, review monthly analytics, filter and sort transactions, and export the current view.

Transactions are stored locally:

The backend keeps an in-memory array while it is running.
The frontend mirrors transactions in browser localStorage, so browser reloads do not lose them.
Backend memory is cleared when the backend process restarts.

What it includes

English and Hinglish natural-language transaction input
Local Ollama parsing with strict JSON output
Keyword-based parser fallback when Ollama is unavailable
INR phrase handling such as four fifty -> 450
Monthly summary cards and Chart.js category breakdown
Category filters and newest/oldest/amount sorting
Excel-compatible CSV transaction export
Full PNG/JPG chart export with legend and totals
Responsive React UI with Framer Motion

The current React interface does not use voice/STT/TTS. Legacy ElevenLabs backend code remains available but is not part of the…

View on GitHub

How I Built It

ExpenseAI is built on a decoupled full-stack architecture designed for privacy, speed, and offline resilience:

Frontend: A responsive, interactive React UI scaffolded with Vite. It uses framer-motion for fluid micro-animations, Chart.js for dynamic visual analytics, and SheetJS for handling local CSV/Excel imports. Data is persisted securely in the browser's localStorage to ensure persistence across reloads.
Backend: A lightweight Node.js/Express server that acts as the middleware. It handles cross-origin requests, validates the incoming natural language prompts, and orchestrates the AI parsing.
AI Integration: The backend sends a meticulously crafted system prompt and strict JSON schema to a local instance of Ollama running llama3.2. If the model is not running, the Express API instantly routes the request to a regex-based heuristic fallback parser to ensure the dashboard remains fully functional.
Storage: No MongoDB, Postgres, or cloud database is required. The backend uses a volatile in-memory array for session continuity, heavily complementing the frontend's local storage solution.

Why Does Open Innovation Matter?

Personal financial logs represent highly sensitive data. Sending every daily meal, utility bill, or salary deposit to closed, proprietary cloud APIs creates unnecessary privacy risks, rate-limit constraints, and subscription costs.

Open innovation and open-weight models made ExpenseAI possible by enabling:

Absolute On-Device Privacy: Financial data never leaves the user's browser or local machine.
Zero API Cost & Unlimited Usage: Free, unlimited local inference without API keys, monthly bills, or rate limits.
True Offline Independence: Fully functional without an active internet connection—allowing users to track expenses anywhere, anytime.

My Agent Session

We utilized AI agent workflows throughout the build to refactor the project's default models to llama3.2, handle git history and commits properly, test local model edge cases, and add debugging utilities directly into the codebase.

GitHub Repository: Aditya4453/DEVchallenge

Team Members

Credit to my teammate who contributed to this project:

Samridhi (@justsamridhi)

Prize Categories

Main Challenge: Build for a Friend
Best Use of Local Inference / Ollama