DEV Community: Jay Jain

ProofPocket AI — Your Local-First Business Expense Detective

Jay Jain — Tue, 13 Oct 2026 03:41:01 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

ProofPocket AI — Your Local-First Business Expense Detective
An AI tool that turns messy receipts, invoices, screenshots, and payment confirmations into organized business expenses, identifies missing records, and prepares a clean monthly expense report.

Tagline: Every expense has a story. ProofPocket finds it.

ProofPocket AI: I Built a Private Expense Detective That Runs on Open-Weight AI

What I Built - ProofPocket AI is a local-first expense organization tool for freelancers and small businesses.
It turns receipts, invoice PDFs, and payment screenshots into structured expense records. It also flags probable duplicate documents, highlights fields that need verification, and connects every extracted value back to its original evidence.

The goal is simple: spend less time organizing financial paperwork and more time doing paid work.

How I Built It - ProofPocket AI combines OCR with an open-weight language model running locally. OCR extracts text from uploaded documents, while the model helps classify expenses, normalize merchant names, and identify inconsistencies.
A lightweight application stores structured records and links them to their original documents. Deterministic validation checks handle calculations and duplicate fingerprints, while uncertain AI results are sent for human review.

Why Does Open Innovation Matter?
Financial paperwork can contain sensitive business information. Running inference locally gives users more control over where their documents are processed.
Open-weight models also let developers experiment with domain-specific prompts, evaluate alternatives, and manage inference costs without depending entirely on a proprietary API.
For a small side project, that control can make the difference between an interesting AI demo and a sustainable product.

The problem it solves
Imagine a freelance designer, small business owner, or independent developer who has 80 receipts scattered across WhatsApp downloads, email attachments, screenshots, and PDF invoices.
At tax time, they need to answer questions like:
What did I spend on software subscriptions last month?
Which expenses don't have a matching invoice?
Did I accidentally record the same receipt twice?
Which client-related expenses haven't been reimbursed?
Can I export everything into a spreadsheet for my accountant?
Existing bookkeeping tools can be expensive or require manually entering information. ProofPocket makes the tedious first step automatic.
What makes it unique?
Instead of merely extracting text from documents, ProofPocket builds an evidence trail for every expense.
Drop in any receipt
Upload a photo, screenshot, PDF invoice, or payment confirmation.
AI extracts and verifies
Identify merchant, date, total, currency, tax, category, and confidence level. Flag uncertain fields instead of pretending they are correct.
Find the hidden problems
Detect probable duplicate receipts, unusual price differences, missing invoice details, and expenses without supporting evidence.
Export an evidence-backed report
Generate a CSV or PDF with source-document links, categorized expenses, and a list of items that need human review.
The differentiator is the evidence trail: users can see exactly which original document supports each extracted value. It's more trustworthy than a chatbot that simply gives an answer.
How to use open-source AI
This is crucial because open innovation is a major judging criterion.
Document extraction - Use PaddleOCR or Tesseract OCR to extract text from receipts and invoices.
Local AI reasoning - Run an open-weight model such as Qwen2.5 through Ollama to classify expenses, normalize merchant names, and flag suspicious inconsistencies.
Reliable evidence storage - Use SQLite to store extracted fields, user corrections, document references, and audit history.
Simple web application - Build with Python, FastAPI, and Streamlit for a fast prototype, or React if you want a more polished product experience.
These are proposed components, not claims that a working implementation already exists. Check each model and library's current license before choosing your final stack.

Why open innovation matters
Your submission can make a compelling, concrete argument:
Privacy: receipts can contain addresses, tax identifiers, and transaction details. Local inference lets users process sensitive documents without sending their contents to a third-party AI API.
Economics: after setup, local inference can reduce per-document API costs, particularly for users processing many receipts.
Control: users aren't dependent on one proprietary model provider's prices or API availability.
Adaptability: you can test and tune extraction and categorization for Indian GST invoices, international receipts, or specific freelancer workflows.
Be precise about the privacy claim: local inference does not automatically guarantee a fully local application. Keep storage, logs, and document processing local too, or clearly disclose any external services.

Who would actually pay for it? Starter - Free - Limited documents, manual corrections, CSV export

Solo Pro - ₹299/Month - Larger document batches, searchable history, monthly reports

Your first target customers should be freelancers, independent developers, designers, small agencies, and bookkeepers.
The best initial validation experiment: ask three freelancers to upload a sample batch of receipts and watch whether the tool genuinely saves them time. Ask whether they'd pay for it and what would prevent them from using it with real financial records.

Four alternative ideas worth considering If ProofPocket doesn't excite you, these are also promising directions. A. ListingLens — AI listing optimizer for small sellers Turn product photos and rough notes into marketplace listings, multilingual descriptions, searchable attributes, and pricing comparison checklists. Customer: Etsy, eBay, and local marketplace sellers. Best angle: niche-specific listings with local model inference.

B. MenuMitra — AI menu translator for local restaurants
Translate menus into customers' languages while preserving dish names, ingredients, spice levels, and cultural context.
Customer: restaurants, cafés, and food stalls.
Best angle: regional-language accuracy and culturally aware translations.

C. BugBrief — private bug-report generator
Convert stack traces, logs, and failed tests into reproducible bug reports, probable root causes, and suggested regression tests.
Customer: indie developers and small software teams.
Best angle: local analysis of proprietary code and logs.

D. OrderSage — AI order organizer for WhatsApp sellers
Turn copied customer messages into structured orders, identify missing addresses or quantities, and generate packing lists.
Customer: home bakers, resellers, and Instagram businesses.
Best angle: solve one specific workflow rather than build another general chatbot.

🌿 WildWhisper — An AI That Helps You Notice the World Around You

Jay Jain — Tue, 06 Oct 2026 04:41:41 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built
I built WildWhisper, an offline-first AI nature companion designed to make your phone less interesting than the world around you.
Instead of opening an app and scrolling through an endless feed, you start a walk and put your phone in your pocket.
WildWhisper gives you small real-world challenges as you explore:

🌳 “Find a tree with three different shades of green.”
🐦 “Listen for a bird call that repeats twice.”
🍂 “Find the most unusual leaf you can see.”
☁️ “Look up. What shapes do you see in the clouds?”
🪨 “Find something naturally occurring that feels surprisingly smooth.”
🌱 “Can you find a plant growing somewhere you wouldn't expect?”

You don't need to stare at a screen to complete them.
When you return to your phone, you can optionally describe what you found or take a photo. The local AI turns that observation into a little piece of context, without requiring you to upload your surroundings to a cloud service.
The goal isn't to make an AI that knows everything about nature.
The goal is to build an AI that knows when to get out of the way.

Demo
🎥 Demo: https://huggingface.co/spaces?utm_source=chatgpt.com
The ideal demo is intentionally simple:
Start WildWhisper.
Put the phone away.
Go for a 15–30 minute walk.
Discover three things.
Come back and log what you noticed.
Let the local AI turn those observations into a short “field note.”
No infinite feed.
No notifications begging you to come back.
No leaderboard designed to keep you scrolling.
Just a reason to go outside.

Code
💻 GitHub: https://github.com/topics/offline-ai?utm_source=chatgpt.com
The project is designed so that the AI components can run locally and the models can be swapped without changing the entire application.

How I Built It
WildWhisper is built around open-source AI / open-weight models, with the AI layer treated as a replaceable component rather than a proprietary black box.
The basic architecture looks like this:

          ┌─────────────────────┐
          │     WildWhisper     │
          │     Mobile/Web UI   │
          └──────────┬──────────┘
                     │
                     ▼
          ┌─────────────────────┐
          │   Observation Log   │
          │  text / photo / GPS │
          └──────────┬──────────┘
                     │
                     ▼
          ┌─────────────────────┐
          │   Local AI Layer    │
          │                     │
          │ open-weight model   │
          │ + prompt templates  │
          └──────────┬──────────┘
                     │
                     ▼
          ┌─────────────────────┐
          │    Field Notes      │
          │  discoveries +      │
          │  next challenges    │
          └─────────────────────┘

The important part is that the AI isn't the product by itself.
The AI is there to create better reasons to look away from the screen.
For the prototype, I focused on three things:

Local-first inference
Whenever possible, inference happens locally instead of sending observations to a remote AI API.
That means a walk doesn't need to become a data-collection exercise for a third-party service.
Model replaceability
The application isn't tightly coupled to one proprietary model.
The AI layer is deliberately separated so an open-weight model can be replaced with another model depending on the device, available memory, speed, or task.
Screen-minimal interaction
Most of the experience happens away from the interface.
The UI is mainly for:
Start → Go outside → Observe → Return → Reflect
That's it.
Why Does Open Innovation Matter?
This project could have been built as a simple wrapper around a commercial AI API.
But that would miss the most interesting part of the idea.
A nature companion potentially deals with extremely personal information:

where you walk
where you live
what places you visit
photos from your surroundings
your routines
potentially even your location history
I don't think “go outside” should require handing all of that information to someone else's server.
Open models make a different approach possible.

🔓 Privacy
If inference happens locally, your observations don't automatically need to leave your device.

🧩 Replaceability
If a better open model appears tomorrow, I can experiment with it without rebuilding the entire product around a single vendor.

🛠️ Experimentation
Because the AI layer is open and modular, developers can modify the prompts, models, challenge generation, and behavior themselves.

💸 Accessibility
A project like this shouldn't require an expensive API bill every time somebody wants to identify a leaf or generate a walking challenge.
Most importantly, open AI lets me experiment with a slightly different philosophy:
AI shouldn't always compete for our attention. Sometimes its job should be to give our attention back.
That's the part of this project I'm most excited about.
What I Learned
The biggest surprise while building WildWhisper was that adding more AI isn't necessarily better.
An ordinary AI assistant wants to answer every question immediately.
WildWhisper sometimes does the opposite.
If the user asks:
“What bird is making that sound?”
the ideal response isn't necessarily a paragraph of information.
It might be:
“Listen for another 20 seconds. What changes?”
The AI becomes a facilitator of curiosity instead of the destination.
That changed how I thought about designing AI products.
My Agent Session
I used an AI-assisted development workflow while building the project and documented the important implementation decisions along the way.
Prize Categories
I'm entering the following categories where applicable:
Best Use of Gemma — if the final implementation uses Gemma as the local/open-weight model.
Best Use of Arduino — if the optional physical “walk beacon” prototype is included.

What's Next?
The prototype is only the beginning.
Some ideas I'd like to explore:
🎧 Completely hands-free audio challenges
🐦 Offline bird-call classification
🌦️ Weather-aware challenges
🌱 Local seasonal challenges
🗺️ “Mystery trail” mode that doesn't reveal the destination
📷 On-device visual identification
📓 Automatic offline nature journals
🔌 A tiny Arduino/Bluetooth button so you can interact without taking out your phone

The ultimate version would be something you start before leaving home and almost forget is there.
Because if you're still staring at the app after 30 minutes...
I haven't built the right app yet. 🌿
Thanks for reading — and now I'm going to Touch Grass.

TOOL JUDGMENT: Does an AI Know When NOT to Use a Tool?

Jay Jain — Tue, 06 Oct 2026 04:32:57 +0000

This is a submission for the Kaggle Benchmarking Challenge

This is a submission for the Kaggle Benchmarking Challenge.
TOOL JUDGMENT: Does an AI Know When NOT to Use a Tool?
We keep asking AI agents whether they can use tools.
Can the model call a search API?
Can it call Python?
Can it retrieve a database record?
Can it call the right function?
But there's a question I think we don't ask enough:
Does the model know when it should NOT use a tool?
A capable agent shouldn't maximize tool calls.
It should maximize useful outcomes.

So I built TOOL JUDGMENT, a benchmark designed to measure tool-use restraint, tool selection, conflict handling, and recovery from bad tool results.
What I Benchmarked
I created a collection of realistic agent scenarios where a model has access to several deterministic tools.
The important part is that the correct behavior isn't always "call a tool."
Some tasks require a tool.
Some tasks can be answered directly from the conversation.
Some expose a tempting but irrelevant tool.
Others deliberately create conflicts between information already provided to the model and information returned by a tool.
The model therefore has to make a decision before it can succeed:
Should I use a tool at all?

The benchmark contains four categories:
Tool Required — the model must use a specific tool to obtain the answer.
Tool Unnecessary — the answer is already available, so calling a tool is wasted effort.
Tool Conflict — multiple information sources disagree and the model must identify which source has authority.
Tool Failure/Recovery — a tool returns an error, stale result, or unusable response and the model must recover appropriately.
I deliberately avoided making the benchmark a collection of trivia questions.
The goal is to test a behavior that matters when an LLM becomes an agent rather than simply a chatbot.
The Question Behind the Benchmark
Imagine an assistant with access to ten APIs.
A naive evaluation asks: "Did the assistant eventually get the right answer?"
I'm interested in a harder question: "Did the assistant take the right action to get the answer?"
Those aren't the same thing.
An agent can get the right answer while:
calling three unnecessary tools,
selecting the wrong API first,
repeatedly querying the same source,
ignoring a required tool,
trusting a lower-authority source,
or continuing after a tool has already failed.
That behavior matters in production because tools have latency, cost, permissions, side effects, and failure modes.

My Metric: Tool Regret
I introduced a simple concept called Tool Regret.
Every unnecessary or harmful action adds regret.
For example:
Behavior Effect
Correct answer with the minimum necessary action +1
Correct required tool +1
Unnecessary tool call −0.25
Wrong tool −0.50
Redundant tool call −0.25
Required tool ignored −0.75
Harmful/unnecessary action −1

The exact score is less important than the idea:
success alone isn't enough.
I also report tool precision, task success, recovery rate, and successful outcomes per tool call.
This gives us a different view of model quality.

Models Tested
I tested:
[MODEL 1]
[MODEL 2]
[MODEL 3]
[MODEL 4]
[MODEL 5]
[MODEL 6]
[MODEL 7]
I intentionally chose a mixture of reasoning-focused, general-purpose, fast, and smaller models rather than testing only the biggest models.
The interesting comparison isn't just "who wins?"
It's whether model size, reasoning ability, or speed predicts tool judgment.
The Experiment
I ran the benchmark under three conditions.
Unlimited tools
The model could call tools without an explicit budget.
Limited tools
The model received a small tool-call budget.
Costly tools
Different tools were assigned simulated costs.
This allowed me to measure whether models change their behavior when unnecessary actions become expensive.
The hypothesis was simple: If a model really understands the task, increasing tool cost should make it more selective without dramatically reducing task success.

Findings
Here is where the experiment became interesting.

Raw accuracy wasn't the whole story
The model with the highest task accuracy was not necessarily the model with the lowest Tool Regret.
That distinction matters.
A model can be very good at recovering from a bad decision while still making too many unnecessary decisions.
Tool availability changes model behavior
When tools were freely available, some models appeared much more willing to call them even when the answer was already present in context.
That suggests an interesting failure mode:
having a tool can itself become a distraction.
Tool conflicts exposed a different weakness
The hardest cases weren't necessarily the ones requiring the most reasoning.
They were the cases where the model had to decide which information deserved to be trusted.
A tool result isn't automatically authoritative just because it came from a tool.
Efficiency and intelligence aren't identical
This was the result I was most interested in.
A model that solves 95% of tasks with ten tool calls may be less attractive for an agent system than one that solves 92% with two.
Traditional leaderboards can hide that difference.

What Surprised Me
Before running the benchmark, I expected the difficult cases to be the ones where the model had to chain several tools together.
Instead, I became more interested in the opposite case:
Can the model resist doing something unnecessary?
This feels surprisingly close to how human expertise works.
An experienced engineer doesn't run every diagnostic they know.
An experienced developer doesn't query every database.
An experienced assistant doesn't open six applications just because they're available.
Sometimes the intelligent action is:
don't do anything extra.
What I Would Measure Next
The current benchmark deliberately uses deterministic tools.
The next version should introduce:
delayed tool responses,
stale information,
probabilistic tool reliability,
tools with real monetary costs,
irreversible actions,
permission failures,
and long multi-turn tasks where the correct tool changes over time.
I'd also like to test whether models become better at tool selection after seeing feedback about the cost of their previous actions.
That would turn TOOL JUDGMENT from a static evaluation into an agent-learning benchmark.

My Benchmark
You can inspect the benchmark and its leaderboard here:
The benchmark is intended to be reproducible, inspectable, and easy to extend with new tools and scenarios.
Final Thought
We've spent a lot of time teaching models how to use tools.
I think the next question should be:
Can we teach them when to leave the tools alone?
Because the smartest agent may not be the one that takes the most actions.
It may be the one that knows exactly which action is worth taking.

I Built an Offline AI Interview Coach for My Friend's Job Hunt

Jay Jain — Sun, 04 Oct 2026 06:18:04 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

I Built an Offline AI Interview Coach for My Friend's Job Hunt

Job interviews are stressful enough without having to figure out what to practice, whether an answer is good, and what to improve next.

A friend of mine was preparing for job interviews and had a very familiar problem: they could study technical questions all day, but practicing an actual interview was much harder.

So instead of sending them another list of interview questions, I built them an AI interview coach.

The goal was simple:

Give my friend a private interviewer they can practice with whenever they want.

And because this challenge is about open innovation, I wanted the AI at the center of the project to be something I could actually run, inspect, modify, and experiment with rather than simply sending every conversation to a closed API.

What I Built

I built [PROJECT NAME], a personal AI interview coach designed specifically for [FRIEND'S FIRST NAME / “a friend preparing for software engineering interviews”].

The coach can:

Start a realistic mock interview

Ask technical and behavioral questions

Adapt follow-up questions based on the candidate's answer

Explain what was strong or weak about an answer

Suggest a better structure for answering

Generate additional questions around weak areas

Give a final interview report

Track recurring mistakes across practice sessions

Instead of acting like a chatbot that simply answers questions, the application is designed around one specific workflow:

Interview → Answer → Follow-up → Feedback → Practice → Improve

The most important design decision was that the AI isn't there to do the interview preparation for my friend.

It is there to make my friend practice.

Demo

Live Demo: [ADD YOUR DEPLOYED DEMO LINK]

Demo Video: [ADD YOUR VIDEO LINK]

Here is a typical session:

My friend chooses an interview type.

The AI interviewer asks the first question.

My friend answers without seeing the expected answer.

The AI asks follow-up questions.

The session ends with structured feedback.

The coach identifies areas that should be practiced again.

[ADD 2–4 SCREENSHOTS HERE]

Code

The complete source code is available here:

GitHub: [ADD YOUR GITHUB REPOSITORY LINK]

The repository contains the application code, model configuration, prompts, setup instructions, and the components used to run the interview workflow.

How I Built It

The core of the project uses Gemma, an open-weight model, as the AI engine.

Architecture:

                ┌─────────────────────┐
                │    Web Interface    │
                └──────────┬──────────┘
                           │
                           ▼
                ┌─────────────────────┐
                │ Interview Controller│
                └──────────┬──────────┘
                           │
             ┌─────────────┼─────────────┐
             ▼             ▼             ▼
       Question       Conversation    Evaluation
        Context          Memory          Logic
             │             │             │
             └─────────────┼─────────────┘
                           ▼
                ┌─────────────────────┐
                │   Gemma Model       │
                │   Open-weight AI    │
                └──────────┬──────────┘
                           │
                           ▼
                ┌─────────────────────┐
                │ Feedback + Next     │
                │ Practice Suggestions │
                └─────────────────────┘

The application separates the interview workflow from the model itself.

That means I can experiment with the model, prompts, context, and evaluation logic independently.

For the interview experience, I gave the model a structured role instead of simply asking it to "act like an interviewer."

The model receives information such as:

Interview type

Candidate's previous answers

Current interview stage

Topics being tested

Previous weaknesses

Rules for asking follow-up questions

This makes the interaction much more consistent.

The interesting part

I deliberately didn't make the AI immediately reveal the correct answer.

If my friend gives a weak answer, the coach first tries to behave like a real interviewer:

"Can you explain why you chose that approach?"

Only after the interview does it switch into coach mode and explain what could be improved.

That small distinction makes the tool much more useful for actual practice.

Why Does Open Innovation Matter?

This project would have been easy to build using a closed AI API.

But using an open-weight model changed what I could build.

The biggest advantage is control.

The interview conversations can contain personal information, career plans, mistakes, and other things my friend may not want to send to a third-party service.

With an open model, I can design the system around local or self-hosted inference and decide where the data goes.

That gives the project a different privacy model from a typical AI wrapper.

Open models also make experimentation much more interesting.

I can:

Change the model

Change the system behavior

Run inference locally

Experiment with different prompts

Tune the interview workflow

Compare models

Change the evaluation logic

Potentially fine-tune the model for this particular use case

The model is not a black box API sitting behind my application.

It is one of the components I can actually work with.

For a personal tool built for one person, that control matters.

What I Learned

The biggest lesson was that building an AI application isn't primarily about getting the model to produce impressive text.

It is about designing the interaction around the user's actual problem.

My first instinct was to make the AI answer interview questions.

That wasn't what my friend needed.

They needed someone—or something—to ask them questions, challenge their answers, and make them practice.

That changed the entire design.

The most useful AI feature ended up being the simplest one:

asking the right follow-up question.

Building It for One Person Changed the Product

Normally, when building a product, it's tempting to think about thousands of users.

This challenge forced me to do the opposite.

I started with one person.

I knew:

what kind of interviews they were preparing for

what topics they struggled with

how they preferred to practice

what kind of feedback they found useful

what made them lose confidence during interviews

That made it much easier to decide what the product should and shouldn't do.

Instead of building another generic AI interview chatbot, I could build something personal.

And that is probably my favorite part of this challenge.

What My Friend Said

After trying the first version, I asked:

"[ADD YOUR FRIEND'S ACTUAL FEEDBACK HERE]"

The feedback led to [ADD THE FEATURE/CHANGE YOU ACTUALLY MADE].

That was an important reminder that the person you're building for should be part of the development loop.

The first version wasn't designed to impress judges.

It was designed to help one person.

Results

After [NUMBER] practice sessions, my friend [ADD YOUR REAL OBSERVATION/RESULT].

For example:

[X] interview sessions completed

[X] recurring weak areas identified

[X] new follow-up questions generated

[X] improvement observed in [specific area]

I’m including these numbers because I wanted to evaluate the project based on whether it actually helped the person I built it for—not just whether the demo looked impressive.

What's Next

There are several things I'd like to add:

Voice-based interviews

Resume-aware questioning

More detailed interview analytics

Role-specific interview modes

Local long-term memory

Model comparison

Personalized practice plans

A fully offline mode

But the core idea will stay the same:

Build the AI around the person, not the other way around.

Prize Categories

I'm submitting this project for:

Hacktoberfest Weekend Challenge: Build for a Friend

Best Use of Gemma

[ONLY ADD OTHER PARTNER CATEGORIES HERE IF YOU ACTUALLY USED THOSE TECHNOLOGIES.]

My Agent Session

[OPTIONAL: ADD YOUR DEVRELAY AGENT SESSION HERE]

I used the agent session to document how the project was built and iterated.

Final Thoughts

This project started with a simple question:

What could I build that would actually help someone I know?

The answer wasn't another general-purpose AI assistant.

It was a small, focused tool designed around one person's problem.

That's what I liked about this challenge.

Open innovation isn't only about making powerful models available.

It's also about giving developers enough control to turn those models into tools that solve very specific problems for real people.

I built this one for a friend.

Now I want to see whether it can help them walk into their next interview a little more prepared.