DEV Community: Abhisek Roy

Built a Local AI Code Reviewer with Ollama | Private & Open Source 🚀

Abhisek Roy — Thu, 22 Oct 2026 08:10:21 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

Code reviews are essential but draining. Reading every changed line of a junior’s or colleague’s PR, spotting edge cases, and explaining fixes takes focus that’s hard to find at the end of a long day, so issues slip through.

Local AI PR Reviewer does the first pass for you. It watches your chosen repos, reviews every new or updated PR with a local open-weight model on Ollama, and posts senior-engineer-style feedback: critical issues, why they matter, and suggested fixes. It’s comment-only, free to run, and your code never leaves your machine.

Demo

Code

abhisek247767 / local-ai-pr-reviewer

A Python-based GitHub Pull Request reviewer powered by a local Ollama LLM. Manually review PR diffs for potential bugs, security issues, logic errors, and missing tests, with human-controlled review comments.

View on GitHub

How I Built It

The whole project runs on open-source, local AI. No paid APIs, no cloud model calls.

The model

Reviews are generated by Qwen2.5-Coder, an open-weight coding model, running locally through Ollama. The model is set with a single line in .env, so I can switch to a bigger or different model without changing any code:

OLLAMA_MODEL=qwen2.5-coder:3b
GITHUB_TOKEN= Your GitHub access token

The stack

Python, managed with uv
Ollama for local inference (/api/chat in JSON mode)
GitHub REST API for reading pull requests and posting reviews
python-dotenv for configuration

Why Does Open Innovation Matter?

Code is sensitive, and many teams can't send it to third-party AI APIs. Because this reviewer runs an open-weight model locally through Ollama, every diff stays on my own machine. It also costs nothing to run, so I can review every PR and every new commit without worrying about API bills. Open tools give me full control: I can swap models with one line, tune the prompt, and never deal with rate limits or deprecated models. A local model isn't as sharp as the biggest closed ones, but for a first-pass reviewer, being private, free, and always on matters more.

The Lazy Payoff

I ran it on my own demo repo this week. On the note deletion and search PR, it posted a review in about 2 minutes without me opening a single file, and caught two real bugs: a missing useEffect cleanup and unhandled errors in deleteNote. Normally a careful first pass on a PR like that takes me [15–20 minutes]. Now I start from a list of real issues instead of a blank diff, and spend my time only on design decisions. The reviewer works while I don't, which is exactly how a sloth would want it. 🦥

Prize Categories

Best Use of ElevenLabs
Best Use of MongoDB Atlas
Best Use of Mastra
Best Use of GitHub Copilot
Best Use of Auth0 for AI Agents

InvoicePilot AI: Smarter Invoice Automation with TEv1 and MCP

Abhisek Roy — Fri, 16 Oct 2026 06:22:59 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

I built a local invoice assistant for freelancers who lose hours chasing late payments. You upload a photo or screenshot of an invoice, a vision model running on your own laptop reads the client, amount, and due dates, and after you confirm the fields it saves them to a local database. You can then ask plain questions like “which invoices are overdue?” or “how much does this client owe me?”, and it drafts a polite reminder email with the exact amount and days overdue. Freelancers would pay for it because it turns an awkward, tedious chore into one click, without ever sending their clients’ details to a cloud service.

Demo

Code

InvoicePilot AI is split into two repositories to keep the AI agent and invoice tools separate.

1. MCP Client — The AI Agent

This repository contains the agent logic, TEv1 tool-selection flow, and LLM integration. It selects the appropriate invoice tool and handles argument generation and tool execution.

GitHub: MCP Client

2. MCP Server — Invoice Tools

This repository exposes invoice operations as MCP tools, allowing the client to interact with invoice data through a standardized interface.

GitHub: MCP Server

How I Built It

I built this project using TEv1, Ollama, open-weight language models, and the Model Context Protocol (MCP) to explore reliable AI agent tool selection and execution.

Tech Stack

gemini-flash-latest (vision): Reads invoice images locally and extracts the client, amount, and dates as structured JSON.
SQLite: Stores confirmed invoices on the user's own machine.
TEv1: Tool selection and decision-making.
Ollama: Run open-weight models locally.
Qwen 2.5 3B: Local language model experimentation.
MCP: Standardized communication between the AI agent and its tools.
Python: Agent orchestration and tool execution.

The project explores whether combining a lightweight decision model with an LLM can improve tool-calling reliability while reducing input-token usage.

By running the decision model locally, the project avoids additional API costs for tool selection while keeping the main LLM responsible for generating tool arguments.

Why Does Open Innovation Matter?

Open innovation lets me build and experiment with AI agents without depending entirely on closed, proprietary APIs. Using Ollama and open-weight models, I can run AI locally and test different approaches without per-token costs for tool selection. TEv1 helps select the right invoice tool, while MCP connects the client and server. This project shows how open models and protocols can make AI agents more reliable, accessible, and customizable.

My First User

My first user is me. I haven't put it in front of a freelancer yet, so I can't claim a paying customer, but I did pay for this project in a different currency: time and curiosity. Building it taught me things I couldn't pick up from tutorials alone: how an MCP server and client actually talk to each other, why a small router model like tev1 can pick tools better than a big model guessing, how a local vision model reads a messy invoice, and how much a real database query beats asking an LLM to add up numbers.

Using it on my own sample invoices also showed me what a real user would ask for next: PDF support, editing a saved invoice when the model misreads a date, and marking invoices as paid. That's my roadmap, and my next step is to hand it to a freelancer friend and see if they'd pay for it.

Prize Categories

Featured Categories

*Best Use of Gemma

Partner Categories

*Best Use of a .tech Domain

I Have 1 Hour Free" A Walk Plan in 10 Seconds, Powered by Local AI

Abhisek Roy — Sat, 10 Oct 2026 09:54:41 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Free Time Walk Planner is a local AI agent that turns "I have 1 hour free" into a real plan for getting outside.

It picks a nearby park, tells you when to leave and when to turn back so you're home before sunset, and says what to carry based on the weather.

It's for anyone with a short gap in their day who would otherwise spend it scrolling. The answer takes about 10 seconds to read, and then you're out the door. The screen is the shortest part of the experience.

Tell the agent how much free time you have and where you are.

A short plan you can read in about 10 seconds: where to go, when to leave, and when to turn back.

Nearby parks and trails, pulled from OpenStreetMap.

Full transparency: every tool the local models chose, and what each one returned.

Code

abhisek247767 / free-time-agent

A local-first AI agent that turns your free time into a concrete outdoor plan using Ollama, LangChain, weather, sunset, and nearby green-space tools.

View on GitHub

How I Built It

Two small local models on Ollama, each with one job:

tev1:0.8b: a tiny decision model that only decides which tools a request needs (plan a walk, weather, daylight, or politely decline).
qwen2.5:3b: fills in the tool arguments via LangChain tool calling, then writes a short, friendly headline.

Free, keyless open data:

Open-Meteo: weather forecast
Nominatim (OpenStreetMap): place name → coordinates
Overpass API (OpenStreetMap): nearby parks, gardens and trails
OSRM: walking routes and distances
astral: sunrise, sunset and golden hour, calculated fully offline

Safety logic in plain Python, not the LLM:
Leave-by, turn-back and home-by times, plus warnings for rain, wind or a return close to sunset, are all calculated in code, so the numbers are always correct. The LLM only explains them. Tool results are cached locally, so a plan still works when the network drops.

Frontend:
A Streamlit app with live progress, a map of nearby parks, and a "how this answer was made" view that shows every tool call the agent made.

Why Does Open Innovation Matter?

Private by design: Both models run on my machine. My location and walking habits are never sent to a closed AI provider.
Zero cost: No API keys, no usage bills. A daily "go outside" nudge costs nothing.
Right-sized models: Open weights let me pair a 0.8B model for decisions with a 3B model for writing. That split is fast on a laptop, and I can swap either model without touching the rest.
Open data for the real world: OpenStreetMap and Open-Meteo give the agent real parks and real weather for free.

A closed API would have turned a personal "go touch grass" tool into something that costs money per request and sends my location to someone else's server.

Prize Categories

Best Use of Gemma (Featured categories)
Best Use of GitHub Copilot (Partner categories)

AgentToolEval: Grading How LLM Agents Use Tools, Not Just What They Answer

Abhisek Roy — Thu, 08 Oct 2026 06:51:00 +0000

Written for: dev.to readers and the Kaggle Benchmarking Challenge judges.

I kept your format and headings, shortened the intro, added the two local models and the decision test, and filled the Ollama placeholders with numbers from your latest results.

Before you paste it, two corrections affect what the post can claim:

tev1 is not faster than the cloud models on fresh prompts. The 0.096s in your decision results came from Ollama's prompt cache, because your full run reused the exact prompts from my earlier test run. Measured fresh, it takes about 2.1s per decision, against 0.66–1.43s for the cloud models. I re-ran tev1 fresh and replaced its rows in results/decisions/, so the time chart is now correct. The post says "close to cloud speed while running on a laptop", which is true. "Faster" would not be.
gemma4:31b and gpt-oss didn't run on your machine. They ran on Ollama Cloud (your .env points at ollama.com). Only tev1 and qwen2.5:3b ran locally. The post now says this.

This is a submission for the Kaggle Benchmarking Challenge

What I Benchmarked

I built AgentToolEval, a small benchmark that checks whether LLM agents use tools correctly, not just whether they reach the right answer. Did the agent check stock or guess? Did it make five calls when two would do? When a tool failed, did it retry, give up, or make something up? A final-answer check hides all of that.

The model plays a shopping assistant for a fake electronics store with four tools: search_products (prices but not stock), check_stock, place_order, and get_shipping_estimate, a distractor no task needs. The tools are deterministic, so every run is reproducible, and failures are injected on purpose (a timeout once, a service that never recovers, an out-of-stock order).

Scenario	The trap
Cheapest in-stock laptop	The cheapest laptop overall is out of stock
Laptops under ₹50,000 in stock	One laptop costs exactly ₹50,000 ("under" means strictly below)
Order an out-of-stock phone	The model must refuse, not claim success
MacBook price	The search fails once, so the model must retry
Acer stock check	The stock service never recovers, so the model must answer UNKNOWN

Each scenario scores 0 to 1: 60% correct answer, 15% right tools, 15% correct arguments, 10% efficiency (only when the answer is right). The model never sees the answer key; grading happens afterwards, from a log of every tool call.

Models Tested

I ran 8 models on Kaggle and 5 models with Ollama, chosen in groups that each answer a specific question.

On Kaggle ([5] scenarios):

Model	Maker	Why it's in the lineup
`claude-opus-5-5-default`	Anthropic	Newest frontier model, paired with Opus 4.5
`claude-opus-4-5-20251101`	Anthropic	Previous generation, to see whether newer means better at tools
`gpt-5.4-2026-03-05`	OpenAI	Frontier closed model
`gpt-oss-20b`	OpenAI	Small open model from the same maker
`gemini-3.5-flash`	Google	Fast closed model
`gemma-4-26b-a4b-it`	Google	Small open model from the same maker
`glm-5`	Z.ai	Open model from another maker
`grok-4.20-0309-non-reasoning`	xAI	Non-reasoning model from a fourth maker

With Ollama (12 scenarios):

Model	Where it ran	Why
`gemma4:31b`	Ollama Cloud	Counterpart to the Gemma model on Kaggle
`gpt-oss:120b`	Ollama Cloud	Large vs small from the same family...
`gpt-oss:20b`	Ollama Cloud	...to test whether size improves tool use
`qwen2.5:3b`	My laptop	A small general model, as a baseline
`tev1:0.8b`	My laptop	A 0.8B decision model built to pick one option from a list, not to run as an agent

The lineup lets me ask:

Does a newer generation help? Claude Opus 5.5 vs Opus 4.5.
Closed vs open from the same maker: GPT-5.4 vs gpt-oss-20b, and Gemini 3.5 Flash vs Gemma 4.
Does size help? gpt-oss 20b vs 120b, and 1–3B local models against cloud ones.
Is a small decision model good enough to choose an agent's next step? tev1 vs the general models.

Findings

Kaggle leaderboard

Rank	Model	Maker	Score
1	gemini-3.5-flash	Google	0.99
1	claude-opus-5-5-default	Anthropic	0.99
3	glm-5	Z.ai	0.86
4	gemma-4-26b-a4b-it	Google	0.85
5	gpt-5.4-2026-03-05	OpenAI	0.84
5	claude-opus-4-5-20251101	Anthropic	0.84
7	grok-4.20-0309-non-reasoning	xAI	0.71
8	gpt-oss-20b	OpenAI	0.69

The top two tied at 0.99, and there's a clear gap after them. Both solved every scenario correctly. The 0.01 they lost comes from [e.g. one extra tool call on one scenario]. The next group (GLM-5, Gemma, GPT-5.4, Opus 4.5) is packed between 0.84 and 0.86, so the order within that group isn't meaningful.

A newer generation made a big difference. Claude Opus 5.5 scored 0.99 and Opus 4.5 scored 0.84, a jump of 15 points from the same maker. [Which scenario Opus 4.5 failed, from its trace.]

A small open model kept up with frontier models. Gemma 4 26B (0.85) scored about the same as GPT-5.4 (0.84) and Opus 4.5 (0.84), and close to GLM-5 (0.86). On structured tool tasks, size wasn't decisive.

Fast models did well. Gemini 3.5 Flash tied for first. Speed and tool discipline don't seem to conflict here.

The bottom two: Grok non-reasoning (0.71) and gpt-oss-20b (0.69). [What went wrong, from the traces.]

Ollama: Gemma was the most efficient

Model	Where	Correct answers	Avg tool calls	Tokens per task	Time per task
gemma4:31b	Cloud	91.7%	2.5	2,073	3.0s
gpt-oss:120b	Cloud	91.7%	2.75	2,776	3.0s
gpt-oss:20b	Cloud	83.3%	2.75	2,689	4.8s
tev1:0.8b	Laptop	41.7%	2.92	3,435	9.5s
qwen2.5:3b	Laptop	16.7%	1.0	1,282	4.9s

gemma4:31b tied gpt-oss:120b on correct answers (91.7%) but got there with fewer calls and 25% fewer tokens, so it was the cheapest per correct answer. gpt-oss:120b beat its 20b sibling by 8 points, so size did help within one family.

The small local models struggled as agents. qwen2.5:3b made one call on average and then guessed. tev1 did more work but often skipped the required FINAL: answer line, and twice took actions nobody asked for (below). Fewer tokens didn't mean cheaper: both small models cost about 3.5× more than Gemma per correct answer.

Chart: accuracy by model

Chart: tokens per task, input + output

Chart: time per task

Chart: cost per task

The decision test: tev1 is good at choosing, not at acting

tev1 is a decision model, so judging it only as an agent felt unfair. I added a second test: 20 single decisions taken from the same scenarios (which tool first, which tool next, when to stop, retry or give up). Every model saw the same situation and the same five options. tev1 answered through its decision endpoint; the other models answered a plain prompt.

Model	Where	Correct decisions	Input tokens	Output tokens	Time per decision
gemma4:31b	Cloud	100%	232	4	0.66s
gpt-oss:120b	Cloud	95%	262	75	0.92s
gpt-oss:20b	Cloud	95%	262	78	1.43s
tev1:0.8b	Laptop	85%	315	1	2.13s
qwen2.5:3b	Laptop	55%	229	3	2.44s

Same model, two very different results: tev1 chose correctly 85% of the time, but completed only 42% of tasks as an agent. Choosing the next step works; carrying out a whole task doesn't.
It beat a model nearly 4× its size: 85% vs 55% for qwen2.5:3b, both on the same laptop.
One output token per decision, against about 75 for gpt-oss. Its total tokens (~316) were in line with the cloud models.
Close to cloud speed on a laptop: about 2.1s per decision, against 0.7–1.4s for cloud models running on data-centre hardware.
Its three misses were telling: it checked stock before finding the product, chose to order again after an order was already confirmed, and gave up after a "please retry" error.

_Chart: accuracy by mode _

Chart: tokens per task, input + output

Chart: time per task

Specific failures from the traces (Ollama run)

Persistent failure (Acer stock): all three cloud models answered UNKNOWN. tev1 said the laptop was in stock and placed an order nobody asked for.
The ₹50,000 boundary: every Ollama model that answered with product IDs included the laptop priced at exactly ₹50,000. This was the hardest trap in the set.
Out-of-stock order: no model claimed success. gpt-oss:20b answered UNKNOWN instead of OUT_OF_STOCK.
Distractor tool: no model called get_shipping_estimate.
Unneeded tools: for "what is 15% of 2000?", tev1 placed an order for 300 laptops, four times. The cloud models answered directly.

What surprised me

I expected the frontier models to be clearly on top. Instead, a fast model tied for first, and a 26B open model scored the same as GPT-5.4 and Claude Opus 4.5. The biggest single difference was between two generations of the same model, not between makers or sizes.

The second surprise was tev1: the same 0.8B model looked weak as an agent and strong as a decision-maker. How a model is used mattered as much as how big it is.

What changed in how I think about these models: grading the path, not just the answer, showed that models with the same final answer can behave very differently. One checks stock properly and the other gets lucky. Defining "correct" also forced decisions I hadn't expected, like whether ₹50,000 counts as "under ₹50,000."

Caveats

With [5] scenarios on Kaggle, one scenario is worth up to 20 points, so scores within a few points (0.84–0.86) are effectively tied.
Each model ran once. Models can behave differently on repeat runs.
Laptop and cloud times run on different hardware, so compare them loosely. I measured local times on fresh prompts, because repeated prompts hit Ollama's cache and look 20× faster.
Token counts aren't fully comparable across model families, because each uses its own tokenizer and chat template.

What I'd measure next

A small decision model inside an agent: let tev1 pick each next step and a larger model fill in the arguments, to see whether the pair beats either model alone on cost.
Reasoning on vs off: Grok's reasoning version alongside the non-reasoning one.
Held-out phrasings: paraphrases, typos, and Hinglish ("MacBook Air M2 ka price kya hai?").
Repeat runs per model, to measure consistency.

My Benchmark

AgentCallBench: Tool-Calling Reliability on Kaggle

The task code, including the products, scenarios, fake tools, and scoring, is in the benchmark's task notebook.

My Parents Kept Asking "Is This SMS a Scam?" So I Built Her an Offline AI That Explains Every Message

Abhisek Roy — Sat, 03 Oct 2026 04:24:00 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

Every few days my phone buzzes with a screenshot from my mom. It's always an SMS, and it's always followed by the same question: "What does this mean? Should I do something?"

Sometimes it's harmless: a subscription renewal, an EMI reminder, a recharge confirmation. Sometimes it's a scam: "Your KYC has expired, your account will be blocked today, click this link." To her, they look exactly the same. Bank messages are full of short forms like A/c XX1234, NEFT, Avl Bal, and debited, and scammers copy that style on purpose.

I'm not always free to answer right away. And the worst time for her to be confused is when a scammer is creating urgency.

So I built Offline SMS Buddy: a small app on her laptop where she pastes any confusing SMS and presses one big button. It answers three questions in plain language:

🧠 What does this SMS mean?
⚠️ What should I do? (often: "nothing, this is just information")
🛑 What should I be careful about?

For example:

SMS: Your account will be debited ₹499 on 5 October for your recurring subscription.

Offline SMS Buddy: ₹499 will be automatically taken from your account on 5 October for a subscription. You don't need to do anything if you know about this subscription. If you don't recognise it, call your bank using the number on your card.

And when a message looks like a scam, the app shows a clear red warning: don't click the link, don't share any OTP, and call the bank directly.

The most important part: her SMS never leaves her computer. No cloud, no account, and it even works with the Wi-Fi turned off.

Demo

Demo Video

Code

offline-sms-buddy

How I Built It

The stack is entirely open source and runs locally:

Qwen 2.5, an open-weight LLM. I use the 3B version so it runs comfortably on an ordinary laptop with 8 GB RAM.
Ollama, which runs the model locally and serves it at localhost:11434.
Streamlit for the UI.
Python + uv for packaging.

1. Structured output instead of free text

I didn't want to parse a paragraph and guess where "what to do" starts. Ollama lets you pass a JSON schema, so I define the answer shape with Pydantic and the model must fill in exactly those three fields:

class SmsExplanation(BaseModel):
    meaning: str
    what_to_do: str
    be_careful: str

response = client.chat(
    model=MODEL,
    messages=[{"role": "user", "content": PROMPT.format(sms=sms)}],
    format=SmsExplanation.model_json_schema(),
    options={"temperature": 0.2},
)

Each field goes straight into its own section on the page. The low temperature (0.2) keeps the model close to what the SMS actually says instead of getting creative.

2. A prompt written for safety, not cleverness

The prompt tells the model to use short sentences and avoid banking jargon. It also sets some strict rules: don't invent information that isn't in the SMS, and never suggest sharing an OTP. If a message asks her to click a link, update KYC, or threatens to block her account, the model must tell her to verify with the official bank or company first.

3. A safety net that doesn't depend on the AI

Small models are good, but they aren't perfect, and with scams a single miss matters. So the app also runs a plain regex check. If an SMS contains a link, a shortened URL, "KYC" or "blocked", the red warning appears no matter what the model says. If it mentions an OTP, there's always a "never share this code" reminder. The AI explains, and the simple code makes sure the most important warning is never skipped.

4. Designed for my mom, not for developers

Text is about 21px, and the button is huge and full width
There's one text box and one button, nothing else to figure out
Errors are in plain language: "The AI helper is not running. Please start Ollama and try again." instead of a Python stack trace

5. Privacy that's actually true

Why Does Open Innovation Matter?

For this project, open-source AI isn't just a nice extra. Without it, the idea doesn't work.

1. Her SMS messages are some of the most private data she has.
They contain bank balances, partial account numbers, OTPs, loan EMIs, and delivery addresses. Sending all of that to a cloud API I don't control (with its own logging, retention and terms of service) would create a new privacy risk inside a tool meant to protect her. With an open-weight model running through Ollama, the message goes from the text box to localhost and back. That's the whole journey.

2. It works without internet.
Once the model is downloaded, the app works fully offline. I tested it with the Wi-Fi switched off. That matters for a tool she should be able to trust at any moment, not only when the connection is good.

3. It costs nothing to run.
There are no API keys, no per-request billing, and no subscription that runs out. She can paste a hundred messages a day and it costs ₹0. A closed API would mean I'd have to manage billing for my mom's laptop forever.

4. I can swap or tune the model freely.
The model name is a single constant in llm.py. If a better small open model comes out next month, I run ollama pull and change one line. With a closed API, the provider decides when the model changes, and that can change the app's behaviour without warning.

5. Nobody can take it away.
If an API gets deprecated, a price changes, or an account gets locked, the app keeps working. The model file sits on her disk.

Prize Categories

Featured categories
Partner categories