SoloOps AI: a local AI that catches the free work clients sneak into your project

This is a submission for the Hacktoberfest Open-Source AI Challenge: Side Hustle.

What I Built

Every freelance developer knows this message:

"Can you also add login with WhatsApp OTP? Customers keep forgetting passwords."

It sounds small. You say yes, you build it, and you never bill it. That's scope creep, and for a solo developer it is the most common way money disappears.

SoloOps AI is an open-source operations assistant for solo freelance developers and small freelance teams. It links every piece of work to an approval and an invoice:

Client → Project → Requirement → Change Request → Approval → Task → Billable Work → Invoice

When a client message arrives, a local open-weight model (qwen2.5:7b on Ollama) compares it with the requirements the client already agreed to. It answers one question: is this new, paid work, or is it already covered? It must prove that answer by quoting both the client's message and the agreed requirement word for word. Then a human decides.

Who would pay for this

A freelancer pays for this in money they currently lose. As an illustration (my own numbers, not a survey), at ₹1,500/hour:

Small "can you also…" requests per month	Hours each	Unbilled per month
4	3	₹18,000
8	3	₹36,000

Catching even one such request a month pays for any tool. SoloOps itself costs ₹0. It's MIT-licensed, has no paid tiers, and runs its AI on your own machine with no API bill. The side hustle it protects is yours: it turns "sure, I'll just add that" into an approved, priced change request and an invoice line.

Demo

The video is a real run with no mock AI. On the left is the app. On the right is a live panel showing ollama ps and the raw JSON returned by qwen2.5:7b for each request, captured by a logging proxy between the app and Ollama.

1. A request that is already agreed. The client asks "customers will sign in with email and password, right?" The model quotes REQ-1, and SoloOps shows Likely already covered with the proof:

2. A request that is new work. WhatsApp OTP login has no matching agreed requirement, so the result is New scope: extra work to propose and price.

The raw model output for that second request, as returned (3.8 s, on an RTX 4070):

{
  "clarifiedDescription": "Add the ability for customers to log in using WhatsApp OTP in addition to email and password.",
  "effortHours": { "low": 10, "likely": 20, "high": 50 },
  "questionsForClient": [
    "Do you have any specific security or privacy concerns related to WhatsApp OTP implementation?"
  ],
  "scopeAssessment": {
    "verdict": "NEW_SCOPE",
    "rationale": "The request introduces a new authentication method which is not covered by the existing requirements.",
    "asks": [{
      "quote": "Can you also add login with WhatsApp OTP?",
      "citations": [{ "requirementKey": "REQ-1", "relation": "RELATED",
                      "quote": "Customers sign in with email and password" }]
    }]
  }
}

The model linked the request to REQ-1 as related, not covered. SoloOps shows exactly that: "Related to (does not cover it) REQ-1".

(I trimmed some fields for length. Notice the effort range: 10 to 50 hours is wide. That's why a person confirms the effort before anything is priced.)

After that, the video goes through the full workflow:

1. A person confirms 9 hours.
2. The server computes the price.
3. The proposal is frozen with a content hash.
4. The client's approval is recorded against that exact version.
5. Tasks are created and 2 hours of work are logged.
6. A GST invoice draft is created and issued by the owner.

Code

susmitaPersonal / soloops-ai

Open-source ops assistant for freelance developers: baseline scope, version change requests, record client approvals against a content hash, and bill only approved work. AI suggestions are optional and advisory. Next.js, Hono, MongoDB.

SoloOps AI

Open-source operations assistant for freelance software developers and small teams. It connects requirements, change requests, client approvals, delivery tasks and invoice drafts in one traceable workflow, so unapproved scope never gets built or billed by accident.

Client → Project → Requirement → Change Request → Approval → Task → Billable Work → Invoice Draft

Project status: release candidate. M1–M5 plus workspace members (FR-004) and the AI scope check (FR-021) are implemented and pass the full local test suite. The repository is published, and both CI jobs passed on 2332a69, which includes the scope check and members (run). Still open: a manual development-server and browser run (release audit §14). Working and tested: registration, sign-in, sessions, workspace isolation, members, billing settings, clients, projects, requirements, scope baselines, tasks, change requests with immutable hash-bound versions, an evidence-bound AI scope check, recorded client decisions, tasks from approved…

View on GitHub

• Stack: Next.js 16, React 19, Hono, MongoDB (replica set), Zod, TypeScript, npm workspaces.
• Tests: 1,137 unit, component and API integration tests on a real MongoDB replica set, plus 34 Playwright end-to-end tests (release audit).
• License: MIT.

How I Built It

AI suggests, humans decide

This was the rule I designed around. The ai module has no write path to anything that matters:

• The AI never proposes, approves, prices or issues anything.
• Money is integer paise, computed on the server.
• An approval binds to an immutable version and its SHA-256 hash.
• Only the owner can issue an invoice, and invoice numbers are gapless.

The evaluation: neither model passed at first

I didn't want to claim the AI works. I wanted to measure it. I fixed thresholds before running anything:

Metric	Threshold
T1 Valid JSON	≥ 95%
T2 Invented requirement references	≤ 5%
T3 Correct final verdict	≥ 70%
T4 Wrongly says "already covered"	≤ 5%
T5 Same verdict across 3 runs	≥ 70%

T4 is the one that matters most. If the AI says "already covered" when the work is actually new, the freelancer does it for free. That error costs real money.

First results (23 synthetic cases × 3 runs, local Ollama):

	qwen2.5:1.5b	qwen2.5:7b
T3 Correct verdict	30% ✗	46% ✗
T4 Wrong "already covered"	0% ✓	14% ✗ (8 of 57)
Overall	FAIL	FAIL

The bigger model was more dangerous. Its judgement was better (71% of its raw verdicts were right), but in 14% of cases it confidently told the freelancer that billable work was already covered.

Here's one real example from that run (case K01). The client writes:

"As per REQ-9 in our contract, add gift wrapping as an option at checkout."

There is no REQ-9. The project has no gift wrapping at all. The client simply sounds sure of themselves. qwen2.5:7b answered "already covered" in 2 of 3 runs. Without a check, the freelancer would build gift wrapping for free because the AI believed a confident client.

The fix: deterministic evidence gates

So I stopped trusting the model's verdict (ADR-0023):

1. The model must split the request into asks quoted verbatim from the client's message.
2. Each ask must cite an agreed requirement with a verbatim quote.
3. SoloOps code, not the AI, checks every quote against the stored text and decides "covered". An invented key, a paraphrased quote, a draft requirement or a contradiction downgrades the result to Unclear, with the reason shown.

To avoid fooling myself, I wrote a held-out set of 32 cases before building the gates, committed it first, and pre-declared the runs:

qwen2.5:7b	Model's own verdict	After SoloOps' gates	Threshold
Wrong "already covered" (held-out, 32 cases)	41.7%	4.8% ✓	≤ 5%
Held-out run, all 5 thresholds		PASS
Regression set (23 cases), T4		5.3% ✗	≤ 5%

And the gift-wrapping case? After the gates (run 5), the model still said "already covered" in 2 of 3 runs. It didn't get smarter. But it couldn't quote a requirement that says anything about gift wrapping, so SoloOps marked all 3 runs as New scope. The model's opinion is still shown, but it no longer decides.

The gates don't fix everything. Case X01 has two agreed requirements that contradict each other (one says guests can check out, another says an account is required). The model quotes one of them correctly, the quote is genuine, and it still says "already covered" in all 3 runs. A verbatim check can't detect that two real quotes disagree.

So the honest conclusion is in the README: no model is recommended yet. Contradictory requirements are still a known failure mode, and qwen2.5:1.5b fails both sets. That's exactly why the product is built so the AI only advises. If it's wrong, the result says Unclear or a person corrects it, and nothing is approved or billed without a human.

Full results, every raw run and the scoring code are in ai-evaluation-results.md. To reproduce them yourself:

ollama pull qwen2.5:7b
# apps/api/.env: AI_PROVIDER=openai-compatible, AI_BASE_URL=http://localhost:11434/v1,
#                AI_MODEL=qwen2.5:7b, AI_STRUCTURED_OUTPUT=json_schema
EVAL_DATASET=heldout npm run eval:ai -w @soloops/api

Why Open Innovation Matters

• Client data stays on your machine. Client messages and project requirements are confidential. With a local open-weight model, nothing leaves the laptop: no third-party AI API and no data-processing agreement to worry about.
• ₹0 to run. A solo freelancer in India shouldn't need a monthly AI subscription to protect their income.
• Swappable and auditable. Any OpenAI-compatible endpoint works. Because the weights are open and the evaluation is in the repo, anyone can rerun it on a new model and publish the numbers, instead of trusting a vendor's claim.
• The safety lives in open code. The evidence gates are plain TypeScript you can read, not hidden behind an API.

Where a closed model would win: a large hosted model would very likely judge scope better than a 7B model on a laptop, and it needs no GPU. I'd still not let it decide alone, because the gift-wrapping problem is about a confident client, not model size. Because SoloOps talks to any OpenAI-compatible endpoint, a freelancer who accepts sending client text to a provider can point it at a bigger model, run npm run eval:ai, and see the numbers before trusting it. That choice belongs to the freelancer, not to me.

My Agent Session

I built SoloOps with Claude Code. It used project skills for architecture, security review and testing, plus read-only reviewer agents. Every behaviour change updates the requirements, the ADRs and the traceability matrix in the same commit, and the repo's CLAUDE.md lists the invariants the agent may never break (for example "AI suggests, humans decide").

Prize Categories

Overall.