ScopeGhost: An Open-Weight AI Scope Auditor for Freelancers and Agencies

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle.

ScopeGhost: Building a Local-First AI Scope Auditor with Gemma 3

What if freelancers could challenge scope creep with verified evidence instead of uncomfortable arguments?

A client asks for one more feature. Then another revision. Then a native mobile application that was never included in the original agreement.

For freelance developers and small software agencies, these requests can turn a profitable project into weeks of unpaid work.

I built ScopeGhost, an open-source, local-first AI scope auditor that helps developers compare incoming client requests against an agreed project scope, verify contractual evidence, and prepare reviewable change-order drafts.

The interesting part isn't simply putting an LLM behind a web interface. It's engineering the boundaries around the model so that its conclusions can be checked.

1. What I Built

ScopeGhost is designed for freelance software engineers, independent consultants, and boutique agencies managing fixed-scope projects.

It compares project requirements and client communications against baseline contract clauses, then produces structured findings that can be reviewed by a human.

Core capabilities

• Scope classification: Categorizes requests as IN_SCOPE, POTENTIALLY_OUT_OF_SCOPE, AMBIGUOUS, or INSUFFICIENT_EVIDENCE.
• Evidence-linked findings: Connects an assessment to supplied source excerpts and contract clauses.
• Citation verification: Checks whether quoted evidence actually exists in the supplied source text.
• Topic-relevance safeguards: Helps prevent an unrelated exclusion clause from being used to justify an out-of-scope classification.
• Deterministic financial calculations: Calculates estimated costs using integer-cent arithmetic.
• Change-order workflow: Helps prepare change-order drafts for human review and client discussion.
• Local model inference: Uses Gemma 3 through Ollama instead of requiring a hosted LLM API for the core inference workflow.

Who could pay for it?

The target market is independent developers, freelance consultants, and small software agencies.

Potential business models include individual subscriptions, per-audit pricing, or agency plans. These are commercial hypotheses that need validation with actual users; I have not established customer demand or revenue.

The value proposition is straightforward: make it easier to identify potentially out-of-scope work before it becomes unpaid work.

2. Code and Project Links

GitHub repository: https://github.com/suvendukungfu/ScopeGhost

License: MIT

The project uses Next.js, React, TypeScript, Tailwind CSS, SQLite, Zod, Ollama, Gemma 3, Vitest, and Playwright.

3. Product Workflow

The application takes a project scope and new client requests, analyzes their relationship, checks the evidence, and presents findings for review.

flowchart TD
    A["Project Scope and Contract Clauses"] --> C["ScopeGhost"]
    B["Client Emails, Tickets and Notes"] --> C
    C --> D["Local AI-Assisted Analysis"]
    D --> E["Citation and Relevance Checks"]
    E --> F{"Evidence Supports Finding?"}
    F -->|Yes| G["Grounded Scope Finding"]
    F -->|Unclear or Conflicting| H["Ambiguous or Insufficient Evidence"]
    G --> I["Human Review"]
    H --> I
    I --> J["Reviewable Change-Order Draft"]

The goal is not to automate legal judgment. It is to give the developer a clearer, evidence-based starting point for a conversation about scope.

4. System Architecture

ScopeGhost separates probabilistic model interpretation from deterministic validation and persistence.

flowchart TB
    subgraph INPUT["Input Layer"]
        A["Contract Clauses"]
        B["Client Communications"]
    end

    subgraph APP["ScopeGhost Application"]
        C["Next.js Interface"]
        D["Input Parsing"]
        E["Analysis Service"]
        G["Citation Verification"]
        H["Topic-Relevance Gates"]
        I["Financial Engine"]
        J["Human Review"]
    end

    subgraph AI["Local Inference"]
        F["Ollama"]
        K["Gemma 3 4B"]
        F --> K
    end

    subgraph DATA["Persistence"]
        L["SQLite"]
        M["Analysis and Audit Records"]
        L --> M
    end

    A --> D
    B --> D
    C --> D
    D --> E
    E --> F
    K --> E
    E --> G
    G --> H
    H --> I
    I --> J
    J --> L
    J --> N["Reviewable Findings"]

This is a logical view of the architecture. The model proposes an analysis; the application remains responsible for validation and financial calculations.

5. Deep Dive: Gemma 3 4B and Local Inference

ScopeGhost uses Google Gemma 3 4B (gemma3:4b) through Ollama.

Gemma 3 is an open-weight model family. The 4B variant offers a practical starting point for developers experimenting with local inference and structured natural-language analysis.

What the model does

Gemma 3 helps interpret client requests, propose scope classifications, identify candidate evidence, and produce structured analysis for the application to validate.

It does not serve as the final authority on whether a quotation exists, whether a clause was supplied, or whether a financial calculation is correct.

Responsibility	Component
Interpret natural-language requests	Gemma 3 4B
Propose scope classifications	Gemma 3 4B
Extract candidate evidence and clause references	Gemma 3 4B
Validate structured output	Zod and application code
Verify citations against source text	Deterministic citation verifier
Check clause existence	Application validation
Apply topic-relevance safeguards	Deterministic service logic
Calculate estimated monetary values	Integer-cent financial engine
Persist findings and audit records	SQLite-backed workflow

The inference sequence

sequenceDiagram
    actor User
    participant UI as ScopeGhost UI
    participant S as Analysis Service
    participant O as Ollama
    participant G as Gemma 3 4B
    participant V as Validation Layer
    participant DB as SQLite

    User->>UI: Submit scope and client request
    UI->>S: Start analysis
    S->>S: Prepare structured evidence
    S->>O: Send local inference request
    O->>G: Generate proposed analysis
    G-->>O: Return model completion
    O-->>S: Return proposed result
    S->>V: Validate structure and citations
    V->>V: Check clause references and relevance
    V-->>S: Validated finding or error
    S->>DB: Persist records where applicable
    S-->>UI: Return result for review
    UI-->>User: Display findings and evidence

Model limitations

Actual inference latency and output quality depend on hardware, model version, prompt length, inference settings, and input complexity.

The reported integration suite tests specific scenarios using live local inference. Four integration tests cannot establish universal accuracy across contracts, languages, or adversarial inputs.

Local inference offers greater control over data handling, but it is not automatically a complete privacy guarantee. Application configuration and optional integrations also matter.

6. SOLID Principles and Engineering Decisions

ScopeGhost applies several software engineering principles to make its behavior more maintainable and testable.

Single Responsibility Principle

Analysis, citation verification, topic-relevance checks, financial calculations, and persistence have distinct responsibilities. The model should not need to calculate invoice totals, and the financial engine should not need to interpret client intent.

Separation of Concerns

Probabilistic reasoning is separated from deterministic business rules. This makes it easier to improve prompts or change the inference client without intentionally changing the rules governing citation verification or financial arithmetic.

Fail Safely and Preserve Uncertainty

Missing evidence should not automatically become an out-of-scope conclusion. Where the service cannot establish a relevant contractual exclusion, it can return an ambiguous result instead of presenting an unsupported claim with unjustified confidence.

Defense in Depth

The application uses multiple layers rather than relying on one prompt instruction:

• Untrusted evidence framing.
• Structured output validation.
• Clause-reference validation.
• Citation verification.
• Topic-relevance filtering.
• Exclusion eligibility checks.
• Human review.

These controls address different failure modes. None guarantees protection against every attack or incorrect conclusion.

Deterministic Financial Logic

Financial calculations use integer-cent arithmetic rather than relying on generated floating-point totals.

For example, at an illustrative rate of $150 per hour, a 20-hour task represents $3,000 in estimated labor. The developer remains responsible for reviewing the estimate and approving the final commercial proposal.

Testability and Reproducibility

Synthetic fixtures, unit tests, a regression harness, and opt-in local inference tests make different parts of the system easier to validate independently.

The following diagram summarizes these design relationships:

flowchart TB
    A["ScopeGhost Application"]
    A --> B["Analysis Service"]
    A --> C["Citation Verifier"]
    A --> D["Topic-Relevance Gate"]
    A --> E["Financial Engine"]
    A --> F["Persistence Layer"]
    A --> G["Inference Client"]

    B --> C
    B --> D
    B --> E
    B --> F
    B --> G

    C --> H["Verifiable Evidence"]
    E --> I["Reproducible Calculations"]
    G --> J["Replaceable Inference Boundary"]

This illustrates the intended component boundaries, rather than claiming that every SOLID principle has been formally proven across the codebase.

7. Security and Prompt-Injection Handling

Client communications are untrusted input. A message might contain text that attempts to override the model's instructions or fabricate a contractual exception.

ScopeGhost uses explicit untrusted-evidence framing and deterministic checks to help prevent such content from becoming an accepted contractual finding.

flowchart TD
    A["Untrusted Client Message"] --> B["Evidence Encapsulation"]
    B --> C["Local Model Inference"]
    C --> D["Proposed Structured Output"]
    D --> E{"Clause ID Exists?"}
    E -->|No| F["Reject Unknown Clause"]
    E -->|Yes| G["Verify Citation and Relevance"]
    G --> H{"Checks Pass?"}
    H -->|No| I["Reject or Downgrade"]
    H -->|Yes| J["Evidence-Grounded Finding"]
    F --> K["Human Review or Error Handling"]
    I --> K
    J --> K

In the reported live integration test, an adversarial prompt attempted to introduce an invented clause ID. The production validation layer rejected that unknown reference.

This demonstrates a specific safeguard working in a tested scenario. It does not mean Gemma 3 is immune to prompt injection or that every possible adversarial request will be caught.

8. Financial Calculation Workflow

The financial engine turns reviewed effort estimates into reproducible cost ranges.

flowchart TD
    A["Candidate Scope Change"] --> B["Review Estimated Hours"]
    B --> C["Set Hourly Rate"]
    C --> D["Convert Monetary Values to Integer Cents"]
    D --> E["Calculate Minimum Cost"]
    D --> F["Calculate Maximum Cost"]
    E --> G["Build Cost Range"]
    F --> G
    G --> H["Human Reviews Estimate"]
    H --> I["Change-Order Draft"]

The underlying relationship is:

[
\text{Estimated Cost}=\text{Estimated Hours}\times\text{Hourly Rate}
]

At $150 per hour, a 20-hour task produces a $3,000 labor estimate. The result is a calculation from supplied inputs, not a guarantee of actual implementation effort or the final project price.

9. SDLC: From Problem Definition to Release Review

I approached ScopeGhost through an iterative software development lifecycle rather than treating it as a one-shot AI code-generation exercise.

flowchart TD
    A["Problem Definition"] --> B["Requirements"]
    B --> C["Architecture and Threat Modeling"]
    C --> D["Incremental Implementation"]
    D --> E["Unit and Regression Tests"]
    E --> F["Live Integration Tests"]
    F --> G["Build and E2E Verification"]
    G --> H["Documentation and Release Review"]
    H --> I["User Feedback"]
    I --> B

Phase 1 — Requirements

I defined the central problem as scope creep in fixed-scope freelance engagements. The requirements included evidence-linked classifications, conservative handling of missing evidence, and reviewable change-order workflows.

Phase 2 — Architecture and threat modeling

I separated inference from deterministic validation and financial calculations. I also considered failure cases involving fabricated quotations, unknown clause IDs, irrelevant exclusions, and adversarial instructions inside client-provided text.

Phase 3 — Incremental implementation

The application was built in reviewable increments using typed interfaces, structured schemas, an inference client, analysis services, persistence, and user-facing workflows.

Phase 4 — Verification

The testing strategy includes unit tests, synthetic regression tests, opt-in live inference tests, browser tests, type checking, and production builds.

Phase 5 — Documentation and release review

The repository includes setup instructions, evaluation fixtures, a testing framework, and technical documentation. The M4.9 evaluation framework and Hacktoberfest submission materials have been committed to the repository.

Phase 6 — User validation

The next step is to obtain feedback from real freelancers and small agencies. Their experiences will help determine whether the evidence presentation and change-order workflow solve a sufficiently important problem to support a sustainable product.

This is an iterative SDLC approach, not a claim of formal certification or completion of every possible release process.

10. Testing Architecture and Evaluation Results

ScopeGhost separates deterministic safeguards, mock-based regression evaluation, and live model integration.

flowchart TB
    A["ScopeGhost Verification"]

    A --> B["Tier 1: Deterministic Safeguards"]
    A --> C["Tier 2: Synthetic Regression"]
    A --> D["Tier 3: Live Ollama Integration"]
    A --> E["Build and Browser Verification"]

    B --> B1["Schema and Citation Checks"]
    B --> B2["Clause and Topic Gates"]
    B --> B3["Financial Invariants"]

    C --> C1["36 Synthetic Cases"]
    C1 --> C2["Pre-Recorded Completions"]
    C2 --> C3["Production Analysis Service"]

    D --> D1["Gemma 3 4B"]
    D1 --> D2["Four Reported Integration Tests"]

    E --> E1["TypeScript"]
    E --> E2["Production Build"]
    E --> E3["Playwright E2E"]

Synthetic regression benchmark

The reported npm run test:eval results on the versioned corpus v2.0.0-synthetic are:

Metric	Result
Synthetic evaluation cases	36
Strict exact accuracy	97.2% (35/36)
Acceptable classification rate	100% (36/36)
False exclusion rate on in-scope fixtures	0% (0/6)

Important limitation: The benchmark uses DeterministicHarnessModel and pre-recorded completions passed through the production ScopeAnalysisService. It evaluates regression behavior and deterministic safeguards. It is not a benchmark of 36 independent live model responses or proof of real-world contract accuracy.

Live Ollama integration tests

The repository includes four opt-in tests using local gemma3:4b inference:

1. Live clause-citation extraction and verification.
2. Analysis orchestration and SQLite persistence.
3. Preservation of uncertainty when evidence is insufficient.
4. A prompt-injection scenario involving an invented contractual clause reference.

These tests exercise specific scenarios, not universal model robustness.

Reproduce the checks

# Unit test suite
npm test

# Synthetic regression evaluation
npm run test:eval

# Live Ollama integration suite
npm run test:integration

# TypeScript type checking
npm run typecheck

# Production build
npm run build

# Browser end-to-end tests
npm run test:e2e

The live integration suite requires Ollama and the gemma3:4b model. Consult the repository README for the exact opt-in requirements.

11. Local Deployment and Data Flow

The local-first design allows developers to run the core inference workflow on their own machines.

flowchart TB
    subgraph DEVICE["Developer Machine"]
        A["Browser"]
        B["Next.js Application"]
        C["Ollama Service"]
        D["Gemma 3 4B"]
        E["Local SQLite Database"]
    end

    A <--> B
    B <--> C
    C <--> D
    B <--> E

    F["Optional External Services"]
    B -. "If configured and invoked" .-> F

This diagram describes the intended local deployment. Optional integrations and application configuration can change the data-flow boundary, so local inference should not be interpreted as an automatic guarantee that every kind of data always remains on-device.

12. Why Does Open Innovation Matter?

Open-weight AI makes it possible to experiment with local inference, inspect application behavior, and adapt the model workflow without tying the entire product to one hosted API.

For ScopeGhost, three benefits stand out.

Data control: Local inference offers a way to process sensitive project evidence without sending model prompts to a hosted LLM endpoint, when the application is configured appropriately.

Cost flexibility: Local inference can reduce dependence on per-request API pricing, although hardware, electricity, and maintenance still have costs.

Freedom to experiment: Developers can test alternative models, adjust prompts, and improve the application while keeping the validation layer under their control.

Open innovation also makes the engineering more inspectable. Other developers can explore the source code, examine the evaluation fixtures, and understand what the system checks and where its limitations remain.

13. What's Next?

The next milestone is practical validation with real users.

flowchart LR
    A["Current Foundation"] --> B["Validate with Freelancers"]
    B --> C["Improve Audit Usability"]
    C --> D["Broader Contract Formats"]
    D --> E["Additional Export Workflows"]
    E --> F["Expanded Evaluation Corpus"]

These are potential next steps, not claims that all of these features are already implemented.

I want to learn whether freelancers find the audit results useful, whether evidence-backed findings make scope discussions easier, and whether the workflow saves enough time to justify a paid product.

Conclusion

ScopeGhost is my exploration of how open-weight AI can support a real business problem without asking users to trust an LLM blindly.

The model helps interpret natural language. Deterministic safeguards validate critical evidence. Financial logic remains reproducible. Human review stays part of the decision process.

The goal isn't to replace professional judgment. It's to give independent developers better evidence before they agree to do more work.

Explore the code and share your feedback:

GitHub: https://github.com/suvendukungfu/ScopeGhost

devchallenge #hf26challenge #opensource #ai