DEV Community: suvendu kumar sahoo

Never Lose Your Context Again.

suvendu kumar sahoo — Thu, 22 Oct 2026 09:43:22 +0000

🚀 Hacktoberfest 2026 — Week 4: Building ContextKeeper | Evidence-Driven AI, Local-First Recovery & Engineering Beyond the Demo

This Hacktoberfest, I've been building *ContextKeeper *, a local-first developer productivity tool designed to help developers reconstruct interrupted coding sessions from repository evidence.

The central problem is simple, but technically interesting:

What happens when your IDE crashes, your terminal closes, your AI coding-agent session disappears, or you return to a project after losing the context of what you were working on?

You might have uncommitted changes, untracked files, partially implemented functions, stale test logs, incomplete documentation, and no reliable record of the original task.

Git can show committed history and working-tree changes. Documentation can explain intended architecture. Test logs can describe earlier execution results. A local language model can interpret code and suggest possibilities.

But none of those sources, individually, necessarily tells you exactly what happened during an interrupted session.

I wanted to approach this problem from a different engineering perspective:

Instead of asking an AI to guess what happened, reconstruct the observable state first, preserve evidence, distinguish facts from assumptions, and make uncertainty explicit.

That idea became ContextKeeper.

🧠 1. The Problem ContextKeeper Is Trying to Solve

Modern development involves multiple tools, branches, terminals, coding agents, partial implementations, and context switches.

When a session is interrupted, a developer may need to reconstruct:

Which branch and commit they were working on.
Whether the working tree contains staged, unstaged, untracked, renamed, deleted, or conflicted files.
Which parts of the implementation appear to have changed recently.
Which source files, configuration files, and documents are relevant.
Whether a test log represents the current code or an earlier version.
Which conclusions are directly supported by repository artifacts.
Which likely explanations are only inferences.
Which parts of the original objective cannot be determined without clarification.

Traditional recovery based only on commit history misses uncommitted work. Persistent AI memory can retain useful context, but it may not reflect the actual current filesystem state. Automatically executing the project's tests or scripts may also be inappropriate when inspecting an unfamiliar or sensitive repository.

ContextKeeper is designed to address this gap with a deterministic, passive recovery pipeline and optional local AI interpretation.

The guiding principle is:

Reconstruct what the evidence supports. Explain what can reasonably be inferred. Preserve what remains unknown.

🎯 2. What Is ContextKeeper 2.0?

ContextKeeper is a local-first project recovery application with three main ways to interact with its capabilities:

A command-line interface for developers who prefer scripts, terminals, and structured output.
An interactive local web dashboard for exploring claims, evidence, changes, and recommendations.
An optional AI-assisted interpretation mode that uses a locally hosted Ollama model.

The recovery engine is designed to operate independently of AI. Local AI can provide a qualitative narrative, but it is not the authority for determining repository facts.

Core capabilities

Repository inspection

Git working-tree status.
Staged and unstaged diffs.
Untracked files, deletions, renames, and conflicts.
Recent commit history.
Selected source files, package manifests, and project documentation.
Explicitly supplied historical test logs.

Evidence and provenance

Structured evidence records.
Relative file paths and line locators.
SHA-256 content digests.
Stable, content-derived evidence identifiers.
Evidence freshness and redaction metadata.
Cross-references between claims and evidence records.

Deterministic recovery

FACT, INFERENCE, and UNKNOWN classification.
Defensive validation of evidence identifiers.
Explicit limitations and knowledge gaps.
State-dependent, conservative recovery recommendations.

Developer experience

Markdown recovery dossiers.
Machine-readable JSON output.
Interactive dashboard.
Claim filters and search.
Evidence inspector.
Git diff viewer.
Prioritized next steps.
Markdown and JSON export.
Optional light and dark themes.

Optional local AI

Ollama integration.
Loopback-only endpoint policy.
Bounded, redacted evidence input.
Structured model responses.
Evidence-reference validation.
Deterministic fallback when inference is unavailable, malformed, or too slow.

The intention is to make session recovery useful without requiring a cloud AI account or mandatory remote processing.

🏗️ 3. Technology Stack and Engineering Choices

ContextKeeper uses a deliberately lightweight stack.

Area	Technology	Responsibility
Language	TypeScript	Typed application logic and domain contracts
Runtime	Node.js	Application execution and native APIs
Module system	ECMAScript modules	Modular source code and compiled distribution
Filesystem	`node:fs/promises`	Bounded read-only repository inspection
Git execution	`node:child_process.execFile`	Controlled subprocess calls without shell interpolation
HTTP server	`node:http`	Local dashboard and API
Cryptography	`node:crypto`	SHA-256 content digest generation
CLI parsing	`node:util.parseArgs`	Command and option parsing
Local AI	Ollama HTTP API	Optional locally hosted inference
Tests	Node.js native test runner	Unit, integration, security, and evaluation tests
Type checking	TypeScript compiler	Static type validation
Distribution	npm executable package	CLI packaging and installation

One architectural decision was to avoid production runtime dependencies wherever practical.

The reported implementation declares zero production dependencies and primarily uses Node.js standard-library APIs. TypeScript and Node.js type declarations are development dependencies.

This reduces dependency overhead and simplifies distribution, although it does not eliminate every security risk: the application still needs careful input validation, safe subprocess handling, robust HTTP controls, and dependency/build verification.

The dashboard also uses a self-contained HTML/CSS/JavaScript interface rather than depending on a remote UI service or CDN.

📐 4. Architecture Diagram 1 — System Context

This diagram shows how the developer, CLI, dashboard, security controls, inspection subsystems, recovery engine, evidence records, and optional local AI fit together.

flowchart TB
    subgraph Users["Users and Interfaces"]
        Dev["Developer using CLI"]
        BrowserDev["Developer using Browser"]
    end

    subgraph Entrypoints["Entrypoints and Presentation"]
        CLI["CLI Entry Point<br/>bin/contextkeeper.js → src/cli.ts"]
        WebUI["Local Dashboard<br/>127.0.0.1:3333"]
        DashServer["Dashboard Server<br/>dashboardServer.ts"]
        DashHTML["Dashboard Client<br/>dashboardHtml.ts"]
        Formatters["Markdown and JSON Formatters"]
    end

    subgraph Security["Security and Boundaries"]
        PathGuard["Path and Symlink Guard"]
        Redactor["Secret Redactor"]
        GitRunner["Restricted Git Runner"]
    end

    subgraph Inspection["Passive Inspection"]
        Inspector["Inspection Orchestrator"]
        GitInspector["Git Inspector"]
        FileInspector["Filesystem Inspector"]
        Digest["SHA-256 and Evidence IDs"]
    end

    subgraph Recovery["Deterministic Recovery"]
        Engine["Recovery Engine"]
        Claims["Claim Generator"]
        Dossier[("Recovery Dossier")]
    end

    subgraph LocalAI["Optional Local AI"]
        Provider["Ollama Provider"]
        Ollama["Local Ollama Daemon"]
    end

    Repo["Selected Repository<br/>Working Tree and Git Data"]

    Dev --> CLI
    BrowserDev --> WebUI
    WebUI <--> DashServer
    DashServer --> DashHTML
    CLI --> Inspector
    DashServer --> Inspector

    CLI --> PathGuard
    DashServer --> PathGuard

    Inspector --> GitInspector
    Inspector --> FileInspector
    GitInspector --> GitRunner
    GitRunner --> Repo
    FileInspector --> Repo
    FileInspector --> PathGuard

    GitInspector --> Redactor
    FileInspector --> Redactor
    Redactor --> Digest

    Inspector --> Engine
    Engine --> Claims
    Claims --> Dossier
    Engine --> Dossier

    Dossier --> Formatters
    Formatters --> CLI
    Dossier --> DashServer

    Engine -.-> Provider
    Provider <--> Ollama
    Provider -.-> Dossier

Architectural takeaway: both user interfaces rely on the same recovery foundation. The dashboard is not intended to become a separate source of truth, and optional AI does not replace deterministic evidence collection.

🔍 5. The Evidence Pipeline

The core of ContextKeeper is not the dashboard or the model. It is the process of turning repository artifacts into traceable evidence.

The pipeline starts by validating the target repository path. The Git and filesystem inspection subsystems then gather bounded observations. Sensitive content is redacted, content is normalized, and evidence identifiers are generated.

The recovery engine consumes those records and constructs a dossier.

A typical evidence record may contain:

An evidence ID, such as ev_git_status_....
A source type, such as Git status, commit history, diff hunk, project document, or source file.
A relative path or other locator.
A content digest.
A capture timestamp and freshness classification.
Redaction status.
Bounded content or an excerpt.
Metadata about skipped files or inspection limitations.

SHA-256 helps identify the captured content. It does not prove that the content is semantically correct or that a claim derived from it is true.

Architecture Diagram 2 — Component Architecture

flowchart TD
    Bin["bin/contextkeeper.js"]

    subgraph Entry["Application Entrypoints"]
        CLI["src/cli.ts"]
        Index["src/index.ts"]
    end

    subgraph Presentation["Presentation"]
        Server["src/server/dashboardServer.ts"]
        HTML["src/server/dashboardHtml.ts"]
        Markdown["markdownFormatter.ts"]
        Text["textFormatter.ts"]
    end

    subgraph Recovery["Core Recovery"]
        Engine["recoveryEngine.ts"]
        Generator["claimsGenerator.ts"]
        Types["src/types.ts"]
    end

    subgraph Inspection["Inspection"]
        Inspector["src/inspector.ts"]
        Git["gitInspector.ts"]
        FS["fileInspector.ts"]
        Runner["gitRunner.ts"]
    end

    subgraph Security["Security and Integrity"]
        Guard["pathGuard.ts"]
        Redactor["redactor.ts"]
        Digest["evidence/digest.ts"]
    end

    AI["src/ai/localAiProvider.ts"]

    Bin --> CLI
    Index --> Types
    Index --> Inspector
    Index --> Engine
    Index --> Server

    CLI --> Inspector
    CLI --> Engine
    CLI --> Markdown
    CLI --> Text
    CLI --> Server
    CLI --> AI

    Server --> HTML
    Server --> Guard
    Server --> Inspector
    Server --> Engine
    Server --> Markdown
    Server --> AI

    Inspector --> Guard
    Inspector --> Git
    Inspector --> FS

    Git --> Runner
    Git --> Redactor
    Git --> Digest

    FS --> Guard
    FS --> Redactor
    FS --> Digest

    Engine --> Generator
    Generator --> Digest
    Generator --> Types

    Markdown --> Types
    Text --> Types
    AI --> Types

This diagram represents the documented module boundaries. The actual implementation still has some tightly coupled responsibilities, particularly in the CLI, dashboard server, and embedded dashboard asset.

🔄 6. Architecture Diagram 3 — Deterministic CLI Recovery

The CLI workflow illustrates how ContextKeeper reconstructs project state without asking an AI model to establish the facts.

sequenceDiagram
    autonumber
    actor Developer
    participant Bin as bin/contextkeeper.js
    participant CLI as src/cli.ts
    participant Guard as pathGuard.ts
    participant Inspector as inspector.ts
    participant Git as gitInspector.ts
    participant Runner as gitRunner.ts
    participant FS as fileInspector.ts
    participant Redactor as redactor.ts
    participant Digest as digest.ts
    participant Engine as recoveryEngine.ts
    participant Claims as claimsGenerator.ts
    participant Formatter as markdownFormatter.ts

    Developer->>Bin: contextkeeper recover . --report markdown
    Bin->>CLI: runCli(argv)
    CLI->>CLI: Parse and validate options
    CLI->>Guard: validateTargetRoot(".")
    Guard-->>CLI: Canonical repository root

    CLI->>Inspector: inspectRepository(root, options)

    Inspector->>Git: inspectGitRepository(root)
    Git->>Runner: Read-only Git status and diffs
    Runner-->>Git: Bounded Git output
    Git->>Redactor: Redact sensitive content
    Git->>Digest: Generate evidence digest and ID
    Git-->>Inspector: Git evidence records

    Inspector->>FS: inspectFilesystem(root, options)
    FS->>Guard: Validate contained file paths
    FS->>Redactor: Redact selected file content
    FS->>Digest: Generate evidence identifiers
    FS-->>Inspector: File evidence and warnings

    Inspector-->>CLI: InspectionResult
    CLI->>Engine: reconstructProjectState(result)
    Engine->>Claims: generateClaims(result)
    Claims->>Claims: Classify and validate claims
    Claims-->>Engine: Recovery claims
    Engine->>Engine: Generate safe recommendations
    Engine-->>CLI: Recovery dossier

    CLI->>Formatter: Format Markdown
    Formatter-->>CLI: Recovery report
    CLI-->>Developer: Write report to stdout

The key design property is the separation between collecting observable state and interpreting it. The CLI should not need to run the inspected project's tests, execute its scripts, or modify its repository to construct the recovery report.

🧾 7. The Claim Model: FACT, INFERENCE, UNKNOWN

One of ContextKeeper's central architectural decisions is to make the epistemic status of a statement explicit.

`FACT`

A directly supported observation.

Examples:

The repository is on a particular branch.
A specific file exists.
A Git status query reports uncommitted changes.
A captured diff contains particular lines.
A source file contains a particular exported declaration.

A FACT must cite one or more valid evidence records. A valid identifier is necessary, but identifier existence alone is not sufficient to prove semantic support.

`INFERENCE`

A reasoned interpretation based on evidence.

Examples:

Active edits may indicate that work was in progress when a session ended.
A TODO marker may indicate an unfinished implementation concern.
Related untracked files may be part of the same development task.

An inference must explain its rationale and limitations. It should never be presented as direct observation.

`UNKNOWN`

A meaningful limitation of the available evidence.

Examples:

The original user objective cannot be reconstructed reliably from code alone.
A test log does not establish the current working tree's test status if code changed after the log was generated.
A source declaration does not prove successful execution.
Conflicting documents do not always reveal which intended behavior is correct.

Preserving UNKNOWN is not a failure to reason. It prevents a recovery tool from sending the developer in the wrong direction with unjustified certainty.

Architecture Diagram 4 — Evidence-to-Claim Flow

flowchart LR
    subgraph Raw["Repository Artifacts"]
        Status["Git Status"]
        Diff["Git Diffs"]
        Log["Commit History"]
        Docs["Documentation"]
        Code["Source Files"]
        TestLog["Approved Historical Test Log"]
    end

    subgraph Sanitize["Sanitization and Normalization"]
        Redact["Secret Redaction"]
        Hash["Normalize Lines and Hash Content"]
        ID["Generate Evidence IDs"]
    end

    Evidence[("Evidence Records")]

    subgraph Synthesis["Claim Generation"]
        Fact["FACT Rules"]
        Infer["INFERENCE Rules"]
        Unknown["UNKNOWN Rules"]
        Verify["Citation Verification"]
    end

    subgraph Ledger["Auditable Claim Ledger"]
        Facts["Evidence-backed facts"]
        Inferences["Explicitly labeled inferences"]
        Unknowns["Documented knowledge gaps"]
    end

    Status --> Redact
    Diff --> Redact
    Log --> Redact
    Docs --> Redact
    Code --> Redact
    TestLog --> Redact

    Redact --> Hash --> ID --> Evidence

    Evidence --> Fact
    Evidence --> Infer
    Evidence --> Unknown

    Fact --> Verify
    Infer --> Verify
    Unknown --> Verify

    Verify --> Facts
    Verify --> Inferences
    Verify --> Unknowns

This workflow aims to make each recovery conclusion inspectable rather than relying on a single free-form generated summary.

Architecture Diagram 5 — Claim Classification and Validation

stateDiagram-v2
    [*] --> CandidateObservation

    state CandidateObservation {
        [*] --> CheckEvidence
        CheckEvidence --> HasEvidence: Evidence exists
        CheckEvidence --> NoEvidence: Evidence missing
    }

    HasEvidence --> EvaluateCertainty

    state EvaluateCertainty {
        DirectObservation: Direct observation
        Deduction: Deductive reasoning
        Conflict: Stale or conflicting evidence
    }

    DirectObservation --> FactCandidate
    Deduction --> InferenceCandidate
    Conflict --> UnknownCandidate
    NoEvidence --> UnknownCandidate

    state FactCandidate {
        [*] --> CitationAudit
        CitationAudit --> CitationsValid: IDs resolve
        CitationAudit --> CitationsInvalid: IDs missing
    }

    CitationsValid --> FACT
    CitationsInvalid --> UNKNOWN
    InferenceCandidate --> INFERENCE
    UnknownCandidate --> UNKNOWN

    FACT --> [*]
    INFERENCE --> [*]
    UNKNOWN --> [*]

The classification model is designed to be conservative. Its remaining challenge is semantic validation: ensuring that the wording of a claim is actually justified by the content it cites.

🌐 8. The Interactive Dashboard

The CLI is useful for automation, terminal workflows, and piping results into other tools. The dashboard makes the same information easier to explore.

The local dashboard includes:

Recovery overview

Repository path, branch, commit, and working-tree state.
Claim counts by classification.
Evidence counts and inspection statistics.
Warnings and skipped-file reasons.
Recovery recommendations.

Claim ledger

Search by claim text or ID.
Filter by FACT, INFERENCE, and UNKNOWN.
Sort and inspect claim details.
View rationale, limitations, and evidence references.
Navigate from a claim to its supporting evidence.

Evidence inspector

Evidence ID and source type.
File or Git locator.
SHA-256 digest.
Freshness and redaction status.
Bounded content inspection.

Changes viewer

Staged and unstaged diffs.
Untracked files.
Readable, line-oriented change inspection.

Report preview and export

Markdown recovery report.
Structured JSON dossier.
Downloadable outputs.

Visual experience

Light and dark themes.
Accessible status labels and visible focus states.
A responsive, self-contained interface without external fonts or CDNs.

The design goal is to make a recovery report something a developer can investigate, rather than a wall of text they have to trust blindly.

Architecture Diagram 6 — Dashboard Request Sequence

sequenceDiagram
    autonumber
    actor User as Developer
    participant Browser as Dashboard Browser
    participant Server as dashboardServer.ts
    participant Guard as pathGuard.ts
    participant Inspector as inspector.ts
    participant Engine as recoveryEngine.ts
    participant AI as localAiProvider.ts
    participant Formatter as markdownFormatter.ts

    User->>Browser: Open local dashboard
    Browser->>Server: GET /
    Server->>Server: Validate loopback client and Host
    Server-->>Browser: Self-contained dashboard HTML

    User->>Browser: Select repository and inspect
    Browser->>Server: POST /api/inspect
    Server->>Server: Validate request and Origin
    Server->>Guard: Validate target root
    Guard-->>Server: Canonical root
    Server->>Inspector: inspectRepository(root)
    Inspector-->>Server: InspectionResult
    Server->>Engine: reconstructProjectState(result)
    Engine-->>Server: RecoveryDossier

    opt AI interpretation requested
        Server->>AI: Interpret bounded redacted evidence
        AI-->>Server: Validated interpretation or fallback
    end

    Server-->>Browser: RecoveryDossier JSON
    Browser->>Browser: Render claims and evidence

    User->>Browser: Export Markdown report
    Browser->>Server: GET /api/export/markdown
    Server->>Formatter: Format validated dossier
    Formatter-->>Server: Markdown
    Server-->>Browser: Report download

The dashboard is intended to run on the local loopback interface at http://127.0.0.1:3333. Its request boundaries and the recovery engine should be tested independently of the appearance of the interface.

🤖 9. Optional Local AI — Interpretation Without Giving Up Provenance

ContextKeeper also supports optional AI-assisted interpretation using a locally hosted Ollama model.

The architectural intention is not to replace deterministic recovery with an unrestricted autonomous agent.

Instead, the model receives bounded, redacted evidence and can help produce a qualitative explanation of what may be happening in the project.

The local AI subsystem is designed around several controls:

Only local loopback inference endpoints are accepted.
Repository content is treated as untrusted input.
Prompt construction explicitly isolates repository content.
Responses are expected to conform to a structured schema.
Model-generated evidence identifiers are checked against the real evidence set.
Invalid responses, unavailable models, and timeouts trigger deterministic fallback.
AI is optional and is not required for the core CLI or dashboard.

Ollama has been exercised with a local gemma3:4b model in the reported validation. However, local inference latency can vary; a larger recovery prompt has previously exceeded the configured 25-second timeout. The fallback behavior is therefore important, and broader latency testing remains useful.

Architecture Diagram 7 — Local AI Sequence

sequenceDiagram
    autonumber
    participant Caller as CLI or Dashboard
    participant AI as localAiProvider.ts
    participant Ollama as Local Ollama
    participant Dossier as Recovery Dossier

    Caller->>AI: generateLocalAiInterpretation(dossier)
    AI->>AI: Validate loopback endpoint

    alt Non-local endpoint
        AI-->>Caller: Reject and fall back
    else Local endpoint accepted
        AI->>Ollama: GET /api/tags

        alt Daemon unavailable
            AI-->>Caller: UNAVAILABLE / FALLBACK
        else Model available
            Ollama-->>AI: Installed model list
            AI->>AI: Build bounded redacted prompt
            Note over AI: Repository text is untrusted<br/>Evidence IDs are supplied explicitly

            AI->>Ollama: POST /api/generate

            alt Timeout or HTTP failure
                AI-->>Caller: Deterministic fallback
            else Response received
                Ollama-->>AI: Structured response
                AI->>AI: Parse and validate schema
                AI->>AI: Validate cited evidence IDs

                alt Invalid response or citations
                    AI-->>Caller: Fallback with warning
                else Valid response
                    AI-->>Dossier: Attach AI interpretation
                    AI-->>Caller: SUCCESS
                end
            end
        end
    end

An important distinction: a local model producing plausible prose is not the same as a model proving a fact. The model's role is interpretation; verified evidence remains the foundation for factual statements.

🔐 10. Security Architecture: Treat Repositories as Untrusted Input

ContextKeeper may inspect repositories cloned from unfamiliar locations or containing sensitive company code. That makes its security boundaries an essential part of the architecture.

The project implements several layers of defense.

Filesystem safety

Canonical path validation.
Symlink containment checks.
File type and path exclusion policies.
Bounded file count, individual file size, cumulative bytes, directory depth, and inspection time.
Graceful handling of inaccessible or unsupported files.

Git subprocess safety

Explicitly allowlisted read-only Git subcommands.
execFile with shell: false.
Sanitized Git-related environment variables.
Defensive Git configuration overrides.
Timeouts and bounded output buffers.
Rejection of dangerous arguments outside the supported interface.

Secret handling

Pattern-based masking of common credential formats.
High-entropy token screening.
Redaction before evidence is stored or rendered.
Regression tests using synthetic credentials.

Dashboard security

Loopback-only binding.
Client and Host header validation.
Origin validation for state-changing requests.
Request-body size limits.
Content Security Policy and anti-framing headers.
Restricted target paths.

AI security

Local-only endpoint checks.
Redacted, bounded prompt payloads.
Untrusted repository-content isolation.
Structured response validation.
Evidence-ID verification.
Deterministic fallback.

These are defense-in-depth controls, not a claim that every conceivable attack or secret format can be detected. ContextKeeper still requires continuing security review, especially as new data sources and dashboard features are introduced.

Architecture Diagram 8 — Security Trust Boundaries

flowchart TD
    subgraph Untrusted["Untrusted Inputs"]
        Args["CLI Arguments and Paths"]
        Requests["Dashboard HTTP Requests"]
        Files["Repository Files and Symlinks"]
        GitConfig["Hostile Git Configuration"]
        ModelOutput["Untrusted Model Output"]
    end

    subgraph HTTPBoundary["HTTP Boundary"]
        Loopback["Loopback IP Check"]
        Host["Host Header Validation"]
        Origin["Origin / CSRF Validation"]
        Body["Request Size Limit"]
    end

    subgraph PathBoundary["Filesystem Boundary"]
        Root["Canonical Root Validation"]
        Symlinks["Symlink Containment"]
        Exclusions["Path and File Exclusions"]
        Limits["File and Time Budgets"]
    end

    subgraph ProcessBoundary["Subprocess Boundary"]
        Allowlist["Git Command Allowlist"]
        Environment["Environment Sanitization"]
        Config["Git Configuration Overrides"]
        Exec["execFile, shell:false, Timeout"]
    end

    subgraph DataBoundary["Evidence Boundary"]
        Redact["Secret Redaction"]
        Digest["SHA-256 Content Identity"]
        Provenance["Evidence Citation Validation"]
    end

    subgraph AIBoundary["AI and Output Boundary"]
        LocalOnly["Local Endpoint Enforcement"]
        PromptIsolation["Untrusted Content Isolation"]
        Schema["Structured Response Validation"]
    end

    Outputs["Recovery Dossier and Reports"]

    Requests --> Loopback --> Host --> Origin --> Body --> Root
    Args --> Root
    Files --> Symlinks
    Root --> Symlinks --> Exclusions --> Limits
    GitConfig --> Environment --> Allowlist --> Config --> Exec
    Limits --> Redact
    Exec --> Redact
    Redact --> Digest --> Provenance
    ModelOutput --> Schema
    Provenance --> LocalOnly --> PromptIsolation --> Schema
    Provenance --> Outputs
    Schema --> Outputs

📦 11. Packaging and Distribution

ContextKeeper is designed to be distributed as a lightweight npm executable.

The development workflow uses TypeScript for strict checking and compiles the source into ECMAScript modules for distribution.

The package build and packaging checks are intended to include:

The executable wrapper.
Compiled JavaScript output.
README and license.
Required runtime files.
No test fixtures or internal development reports in the distributable package.

The CLI supports recovery-report generation and starting the local dashboard. The project documentation also covers typechecking, testing, building, installation, and troubleshooting.

Architecture Diagram 9 — Build and Deployment

flowchart TD
    Source["TypeScript Source"]
    Tests["Automated Tests"]
    Config["tsconfig.json"]

    Typecheck["npm run typecheck"]
    Test["npm test"]
    Build["npm run build"]
    Dist["Compiled dist/src/*.js"]

    Prepack["npm prepack"]
    Allowlist["Package File Allowlist"]
    Tarball["npm Package Tarball"]

    Install["Install Package"]
    Binary["contextkeeper Executable"]
    Runtime["Resolve Compiled CLI"]
    CLI["Run CLI or Dashboard"]

    Source --> Typecheck
    Tests --> Test
    Config --> Typecheck
    Source --> Build
    Build --> Dist
    Typecheck --> Build
    Test --> Build
    Dist --> Prepack
    Prepack --> Allowlist
    Allowlist --> Tarball
    Tarball --> Install
    Install --> Binary
    Binary --> Runtime
    Runtime --> CLI

The reported package dry-run included 21 files and retained zero runtime production dependencies. The CLI package was also reported as successfully installed and exercised in an isolated sandbox.

One issue identified during the architecture audit was the need to ensure the package's public JavaScript import entry point matches the files actually included in the tarball. Programmatic library consumption should be verified separately from executable CLI use.

🧪 12. Testing and Evaluation

I wanted the project to go beyond a successful build and a few happy-path demos.

The reported validation suite covers several distinct categories:

Unit tests

SHA-256 digest stability.
Line-ending normalization.
Evidence ID generation.
Secret-redaction behavior.
Filesystem path validation.
Git command argument restrictions.

Integration tests

Git porcelain status parsing.
Repository inspection lifecycle.
Non-Git directory handling.
Recovery claims and recommendations.
Dashboard HTTP lifecycle.

Security tests

Hostile Git configurations.
Dangerous environment variable handling.
Command and option injection attempts.
Symlink escapes.
Secret redaction.
Invalid dashboard Host and Origin requests.
Non-destructive inspection.

Synthetic evaluation

Untracked feature files.
Exported source declarations.
Staged and unstaged changes.
Stale documentation.
Historical test logs.
Unknown project intent.
Secret-bearing content.
Resource-limit enforcement.
Malformed input.
Non-Git repositories.

Dogfooding

Self-inspection of the ContextKeeper repository.
Synthetic interrupted-session profiles.
Citation resolution and uncertainty handling.
Repository immutability.
Repeat-run determinism.

Optional local AI tests

Loopback endpoint enforcement.
Non-local endpoint rejection.
Unavailable-daemon handling.
Local model response and citation validation where the runtime is available.

Reported validation results

At the time of the supplied project reports:

60/60 automated tests passed.
TypeScript typechecking completed with zero reported errors.
The production build succeeded.
The 12-scenario synthetic corpus passed.
The dogfooding suite reported 9/9 passing scenarios.
The combined evaluation reported 48/48 eligible files inspected, 202/202 citation references resolving, 24/24 required UNKNOWN classifications preserved, zero seeded-secret exposures, and 59/59 evaluated recommendations matching expected repository state.
Package generation and an isolated CLI installation were reported successful.

These are measured results for the project's implemented test corpus. They should not be interpreted as proof of universal recovery accuracy, complete secret detection, or perfect semantic correctness across arbitrary repositories.

Architecture Diagram 10 — Testing and Validation Pipeline

flowchart TB
    Start["npm test"]

    subgraph Unit["Unit Tests"]
        Digest["Digest Tests"]
        Redactor["Redactor Tests"]
        PathGuard["Path Guard Tests"]
        GitRunner["Git Runner Tests"]
    end

    subgraph Integration["Integration Tests"]
        GitStatus["Git Status Tests"]
        Inspection["Inspection Lifecycle"]
        Recovery["Recovery Engine Tests"]
        Dashboard["Dashboard HTTP Tests"]
    end

    subgraph Security["Security Audits"]
        HostileGit["Hostile Git Config Tests"]
        Arguments["Argument and Environment Tests"]
        Immutability["Non-Destructive Invariants"]
    end

    subgraph Evaluation["Recovery Evaluation"]
        GroundTruth["12 Ground-Truth Archetypes"]
        Dogfooding["9 Dogfooding Scenarios"]
    end

    subgraph AI["Local AI Tests"]
        Locality["Loopback Enforcement"]
        Fallback["Unavailable and Timeout Fallback"]
        Citations["Model Citation Validation"]
    end

    Typecheck["npm run typecheck"]
    Build["npm run build"]
    Release["Review Results and Package"]

    Start --> Unit
    Start --> Integration
    Start --> Security
    Start --> Evaluation
    Start --> AI

    Unit --> Typecheck
    Integration --> Typecheck
    Security --> Typecheck
    Evaluation --> Typecheck
    AI --> Typecheck

    Typecheck --> Build --> Release

🧩 13. SOLID Principles and Architecture Review

A senior-level project should discuss its weaknesses as openly as its strengths.

The architecture audit found the following.

Single Responsibility Principle — Partial

The CLI handles several responsibilities, including argument parsing, command routing, recovery execution, and dashboard startup.

The dashboard server also combines transport security, routing, recovery orchestration, and export formatting. The dashboard HTML module embeds a large amount of CSS, HTML, and JavaScript in one source file.

Potential improvement: split command handlers, route handlers, and frontend assets into focused modules.

Open/Closed Principle — Partial

The claim generator contains sequential claim-generation rules. Adding another class of evidence or claim may require modifying the central generator.

The local AI provider is also closely coupled to the Ollama API.

Potential improvement: introduce a small claim-rule registry and a local AI provider interface only when extension needs justify the additional abstraction.

Liskov Substitution Principle — Not Applicable

The application is primarily functional and data-oriented. It relies on TypeScript interfaces and data structures rather than a class hierarchy.

Introducing inheritance purely to satisfy a SOLID checklist would add little value.

Interface Segregation Principle — Pass

The audit reports that the domain interfaces, including inspection options, content locators, and recommendations, have focused responsibilities.

Dependency Inversion Principle — Partial

Some high-level components import concrete filesystem operations and Git-running functions directly.

Potential improvement: use injected interfaces in the parts that would benefit from easier isolated testing or alternate implementations.

The wider engineering review also considered DRY, KISS, YAGNI, type safety, error handling, immutability, determinism, separation of concerns, testability, and maintainability.

The goal is not to maximize abstractions. It is to keep the architecture understandable, testable, and safe as the project grows.

🧱 14. Architectural Decisions and Trade-Offs

Several decisions shape the product.

1. Zero runtime dependencies

Using Node.js native APIs keeps the package lightweight. The trade-off is that routing, argument handling, and server behavior must be implemented and tested carefully.

2. Read-only repository inspection

ContextKeeper does not automatically stash, commit, clean, or modify the target repository. It also avoids executing arbitrary target-project test scripts during recovery.

This protects unfinished work but means current runtime correctness cannot be inferred from the repository alone.

3. Deterministic recovery first

The recovery pipeline remains useful without a model. AI is an optional layer rather than a required dependency.

4. Evidence-linked claims

The system requires claims to cite evidence records. This improves traceability, but semantic support still needs careful evaluation: the existence of a citation is not equivalent to proof that the claim is correct.

5. Local-only dashboard and AI

The dashboard uses loopback access controls, and the AI provider is designed for local Ollama inference. This limits the deployment model but supports a privacy-oriented experience for developers working with sensitive source code.

6. Bounded inspection

File-count, byte-size, depth, and time limits control resource usage. The trade-off is incomplete evidence when a repository exceeds those limits. Skipped files must remain visible through warnings rather than disappearing silently.

These decisions are documented in the project's architecture decision records and threat model.

⚠️ 15. What ContextKeeper Cannot Guarantee

Being explicit about limitations is part of building reliable developer tools.

ContextKeeper cannot reliably reconstruct the original task from arbitrary code changes alone. A developer may need to clarify the intended outcome.

A valid SHA-256 digest establishes content identity, not semantic truth. A source declaration does not establish runtime correctness. A historical test log does not establish that subsequently changed code passes.

Secret detection relies on patterns and heuristics, so unusual, custom, or low-entropy secrets may evade detection.

Concurrent external processes may also change a repository while it is being inspected. Read-only content operations can update operating-system access timestamps, depending on filesystem behavior.

Local model inference may time out or return invalid output; deterministic fallback remains essential.

Finally, the evaluation results are bounded by the repositories and scenarios tested so far. More diverse real-world sessions and independent evaluations are needed to establish how useful the tool is across different projects and workflows.

🗺️ 16. Development Roadmap

The next stages are about improving reliability and usability rather than adding features without evidence.

Priority 1 — Real-world recovery quality

Evaluate actual interrupted coding sessions and record missed changes, weak recommendations, ambiguous claims, and cases where UNKNOWN should be preserved.

Priority 2 — Package and public API correctness

Keep the installed CLI reliable and ensure that any advertised programmatic library interface resolves to actual packaged JavaScript and type declarations.

Priority 3 — Architecture modularity

Split the largest modules into clearer units when doing so demonstrably improves testability and maintainability.

Priority 4 — Local AI latency and resilience

Measure inference time on real hardware, minimize redundant prompt content, and improve bounded payload construction without removing provenance controls.

Priority 5 — Developer onboarding and community contribution

Maintain useful diagrams, installation instructions, architectural decision records, contributor documentation, reproducible tests, and accessible issue reports.

💻 17. Quick Start

For developers who want to explore the project, the documented commands include:

# Run in the ContextKeeper repository
npm install

# Typecheck
npm run typecheck

# Run the automated tests
npm test

# Build the distribution
npm run build

# Generate a Markdown recovery dossier
node bin/contextkeeper.js recover . --report markdown

# Generate JSON
node bin/contextkeeper.js recover . --report json

# Start the local dashboard
node bin/contextkeeper.js ui .

# Optional: request local AI interpretation
node bin/contextkeeper.js recover . --report markdown --ai --ai-model gemma3:4b

# Inspect package contents without publishing
npm pack --dry-run

The AI command requires a working local Ollama service and the selected model to be installed. Without it, deterministic recovery remains the fallback.

📚 18. Open-Source Documentation and Architecture Artifacts

The project includes a set of engineering documents designed to make the implementation inspectable and easier to contribute to:

docs/PROJECT_DEEP_DIVE.md — system architecture and workflow walkthroughs.
docs/ARCHITECTURE_DECISION_RECORDS.md — architectural choices and trade-offs.
docs/SOLID_AND_CODE_QUALITY_AUDIT.md — design-principle assessment and remediation proposals.
docs/DEVELOPER_GUIDE.md — development setup, CLI, dashboard, testing, packaging, and troubleshooting.
docs/FINAL_MVP_COMPLETION_REPORT.md — recorded acceptance and validation results.
docs/diagrams/01_system_context.mmd
docs/diagrams/02_component_architecture.mmd
docs/diagrams/03_cli_recovery_sequence.mmd
docs/diagrams/04_dashboard_request_sequence.mmd
docs/diagrams/05_evidence_claim_flow.mmd
docs/diagrams/06_local_ai_sequence.mmd
docs/diagrams/07_claim_classification_state.mmd
docs/diagrams/08_security_trust_boundaries.mmd
docs/diagrams/09_packaging_deployment_flow.mmd
docs/diagrams/10_testing_validation_pipeline.mmd

These diagrams illustrate the system context, component architecture, CLI recovery, dashboard request flow, evidence-to-claim pipeline, local AI lifecycle, claim classification, security boundaries, packaging, and testing.

The diagrams are authored as Mermaid source files. Their syntax should be validated with an actual renderer before describing them as independently render-tested.

🌱 19. What I Learned Building ContextKeeper

This project has reinforced several lessons for me as a developer working at the intersection of AI, developer tools, and open source.

First, deterministic systems still matter in AI products. Collecting reliable physical facts about a repository is often more valuable than generating an elaborate explanation without provenance.

Second, security needs to shape the architecture. Path validation, subprocess restrictions, redaction, bounded processing, and read-only behavior are core design decisions—not features that should be added only after the interface is complete.

Third, uncertainty deserves an explicit representation. Keeping UNKNOWN as a first-class outcome makes it possible to recover project state without pretending to understand a developer's intentions.

Fourth, passing tests are necessary but not sufficient. Packaging, failure handling, actual model behavior, interface usability, semantic claim support, and real-world validation all matter.

Finally, good architecture includes trade-offs. Zero runtime dependencies, embedded dashboard assets, a deterministic-first engine, and a locally hosted AI provider each offer benefits while introducing limitations that should be documented and revisited as the product matures.

🚀 20. The Vision

My goal with ContextKeeper is to make interrupted development less painful by helping developers recover their working context from observable evidence.

I want to keep improving the experience while preserving its core engineering principles:

Local-first operation.
Read-only inspection.
Bounded resource usage.
Redacted evidence.
Traceable conclusions.
Explicit uncertainty.
Optional, locally hosted AI.
Reproducible validation.
Clear architectural documentation.

ContextKeeper is not meant to pretend it knows everything about a project. It is meant to help developers establish what can be known, understand what is likely, and recognize what still requires their judgment.

The principle behind the project is simple:

Don't let AI invent your project's history. Reconstruct the evidence, preserve uncertainty, and help the developer take the next step.

I'm continuing to improve ContextKeeper and welcome constructive feedback, architectural review, security feedback, testing ideas, and contributions from the developer and open-source community.

🔗 GitHub: [https://github.com/suvendukungfu/contextkeeper-ai]

Hacktoberfest #hf26challenge #devchallenges #OpenSource

ScopeGhost: An Open-Weight AI Scope Auditor for Freelancers and Agencies

suvendu kumar sahoo — Wed, 14 Oct 2026 09:18:02 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle.

ScopeGhost: Building a Local-First AI Scope Auditor with Gemma 3

What if freelancers could challenge scope creep with verified evidence instead of uncomfortable arguments?

A client asks for one more feature. Then another revision. Then a native mobile application that was never included in the original agreement.

For freelance developers and small software agencies, these requests can turn a profitable project into weeks of unpaid work.

I built ScopeGhost, an open-source, local-first AI scope auditor that helps developers compare incoming client requests against an agreed project scope, verify contractual evidence, and prepare reviewable change-order drafts.

The interesting part isn't simply putting an LLM behind a web interface. It's engineering the boundaries around the model so that its conclusions can be checked.

1. What I Built

ScopeGhost is designed for freelance software engineers, independent consultants, and boutique agencies managing fixed-scope projects.

It compares project requirements and client communications against baseline contract clauses, then produces structured findings that can be reviewed by a human.

Core capabilities

Scope classification: Categorizes requests as IN_SCOPE, POTENTIALLY_OUT_OF_SCOPE, AMBIGUOUS, or INSUFFICIENT_EVIDENCE.
Evidence-linked findings: Connects an assessment to supplied source excerpts and contract clauses.
Citation verification: Checks whether quoted evidence actually exists in the supplied source text.
Topic-relevance safeguards: Helps prevent an unrelated exclusion clause from being used to justify an out-of-scope classification.
Deterministic financial calculations: Calculates estimated costs using integer-cent arithmetic.
Change-order workflow: Helps prepare change-order drafts for human review and client discussion.
Local model inference: Uses Gemma 3 through Ollama instead of requiring a hosted LLM API for the core inference workflow.

Who could pay for it?

The target market is independent developers, freelance consultants, and small software agencies.

Potential business models include individual subscriptions, per-audit pricing, or agency plans. These are commercial hypotheses that need validation with actual users; I have not established customer demand or revenue.

The value proposition is straightforward: make it easier to identify potentially out-of-scope work before it becomes unpaid work.

2. Code and Project Links

GitHub repository: https://github.com/suvendukungfu/ScopeGhost

License: MIT

The project uses Next.js, React, TypeScript, Tailwind CSS, SQLite, Zod, Ollama, Gemma 3, Vitest, and Playwright.

3. Product Workflow

The application takes a project scope and new client requests, analyzes their relationship, checks the evidence, and presents findings for review.

flowchart TD
    A["Project Scope and Contract Clauses"] --> C["ScopeGhost"]
    B["Client Emails, Tickets and Notes"] --> C
    C --> D["Local AI-Assisted Analysis"]
    D --> E["Citation and Relevance Checks"]
    E --> F{"Evidence Supports Finding?"}
    F -->|Yes| G["Grounded Scope Finding"]
    F -->|Unclear or Conflicting| H["Ambiguous or Insufficient Evidence"]
    G --> I["Human Review"]
    H --> I
    I --> J["Reviewable Change-Order Draft"]

The goal is not to automate legal judgment. It is to give the developer a clearer, evidence-based starting point for a conversation about scope.

4. System Architecture

ScopeGhost separates probabilistic model interpretation from deterministic validation and persistence.

flowchart TB
    subgraph INPUT["Input Layer"]
        A["Contract Clauses"]
        B["Client Communications"]
    end

    subgraph APP["ScopeGhost Application"]
        C["Next.js Interface"]
        D["Input Parsing"]
        E["Analysis Service"]
        G["Citation Verification"]
        H["Topic-Relevance Gates"]
        I["Financial Engine"]
        J["Human Review"]
    end

    subgraph AI["Local Inference"]
        F["Ollama"]
        K["Gemma 3 4B"]
        F --> K
    end

    subgraph DATA["Persistence"]
        L["SQLite"]
        M["Analysis and Audit Records"]
        L --> M
    end

    A --> D
    B --> D
    C --> D
    D --> E
    E --> F
    K --> E
    E --> G
    G --> H
    H --> I
    I --> J
    J --> L
    J --> N["Reviewable Findings"]

This is a logical view of the architecture. The model proposes an analysis; the application remains responsible for validation and financial calculations.

5. Deep Dive: Gemma 3 4B and Local Inference

ScopeGhost uses Google Gemma 3 4B (gemma3:4b) through Ollama.

Gemma 3 is an open-weight model family. The 4B variant offers a practical starting point for developers experimenting with local inference and structured natural-language analysis.

What the model does

Gemma 3 helps interpret client requests, propose scope classifications, identify candidate evidence, and produce structured analysis for the application to validate.

It does not serve as the final authority on whether a quotation exists, whether a clause was supplied, or whether a financial calculation is correct.

Responsibility	Component
Interpret natural-language requests	Gemma 3 4B
Propose scope classifications	Gemma 3 4B
Extract candidate evidence and clause references	Gemma 3 4B
Validate structured output	Zod and application code
Verify citations against source text	Deterministic citation verifier
Check clause existence	Application validation
Apply topic-relevance safeguards	Deterministic service logic
Calculate estimated monetary values	Integer-cent financial engine
Persist findings and audit records	SQLite-backed workflow

The inference sequence

sequenceDiagram
    actor User
    participant UI as ScopeGhost UI
    participant S as Analysis Service
    participant O as Ollama
    participant G as Gemma 3 4B
    participant V as Validation Layer
    participant DB as SQLite

    User->>UI: Submit scope and client request
    UI->>S: Start analysis
    S->>S: Prepare structured evidence
    S->>O: Send local inference request
    O->>G: Generate proposed analysis
    G-->>O: Return model completion
    O-->>S: Return proposed result
    S->>V: Validate structure and citations
    V->>V: Check clause references and relevance
    V-->>S: Validated finding or error
    S->>DB: Persist records where applicable
    S-->>UI: Return result for review
    UI-->>User: Display findings and evidence

Model limitations

Actual inference latency and output quality depend on hardware, model version, prompt length, inference settings, and input complexity.

The reported integration suite tests specific scenarios using live local inference. Four integration tests cannot establish universal accuracy across contracts, languages, or adversarial inputs.

Local inference offers greater control over data handling, but it is not automatically a complete privacy guarantee. Application configuration and optional integrations also matter.

6. SOLID Principles and Engineering Decisions

ScopeGhost applies several software engineering principles to make its behavior more maintainable and testable.

Single Responsibility Principle

Analysis, citation verification, topic-relevance checks, financial calculations, and persistence have distinct responsibilities. The model should not need to calculate invoice totals, and the financial engine should not need to interpret client intent.

Separation of Concerns

Probabilistic reasoning is separated from deterministic business rules. This makes it easier to improve prompts or change the inference client without intentionally changing the rules governing citation verification or financial arithmetic.

Fail Safely and Preserve Uncertainty

Missing evidence should not automatically become an out-of-scope conclusion. Where the service cannot establish a relevant contractual exclusion, it can return an ambiguous result instead of presenting an unsupported claim with unjustified confidence.

Defense in Depth

The application uses multiple layers rather than relying on one prompt instruction:

Untrusted evidence framing.
Structured output validation.
Clause-reference validation.
Citation verification.
Topic-relevance filtering.
Exclusion eligibility checks.
Human review.

These controls address different failure modes. None guarantees protection against every attack or incorrect conclusion.

Deterministic Financial Logic

Financial calculations use integer-cent arithmetic rather than relying on generated floating-point totals.

For example, at an illustrative rate of $150 per hour, a 20-hour task represents $3,000 in estimated labor. The developer remains responsible for reviewing the estimate and approving the final commercial proposal.

Testability and Reproducibility

Synthetic fixtures, unit tests, a regression harness, and opt-in local inference tests make different parts of the system easier to validate independently.

The following diagram summarizes these design relationships:

flowchart TB
    A["ScopeGhost Application"]
    A --> B["Analysis Service"]
    A --> C["Citation Verifier"]
    A --> D["Topic-Relevance Gate"]
    A --> E["Financial Engine"]
    A --> F["Persistence Layer"]
    A --> G["Inference Client"]

    B --> C
    B --> D
    B --> E
    B --> F
    B --> G

    C --> H["Verifiable Evidence"]
    E --> I["Reproducible Calculations"]
    G --> J["Replaceable Inference Boundary"]

This illustrates the intended component boundaries, rather than claiming that every SOLID principle has been formally proven across the codebase.

7. Security and Prompt-Injection Handling

Client communications are untrusted input. A message might contain text that attempts to override the model's instructions or fabricate a contractual exception.

ScopeGhost uses explicit untrusted-evidence framing and deterministic checks to help prevent such content from becoming an accepted contractual finding.

flowchart TD
    A["Untrusted Client Message"] --> B["Evidence Encapsulation"]
    B --> C["Local Model Inference"]
    C --> D["Proposed Structured Output"]
    D --> E{"Clause ID Exists?"}
    E -->|No| F["Reject Unknown Clause"]
    E -->|Yes| G["Verify Citation and Relevance"]
    G --> H{"Checks Pass?"}
    H -->|No| I["Reject or Downgrade"]
    H -->|Yes| J["Evidence-Grounded Finding"]
    F --> K["Human Review or Error Handling"]
    I --> K
    J --> K

In the reported live integration test, an adversarial prompt attempted to introduce an invented clause ID. The production validation layer rejected that unknown reference.

This demonstrates a specific safeguard working in a tested scenario. It does not mean Gemma 3 is immune to prompt injection or that every possible adversarial request will be caught.

8. Financial Calculation Workflow

The financial engine turns reviewed effort estimates into reproducible cost ranges.

flowchart TD
    A["Candidate Scope Change"] --> B["Review Estimated Hours"]
    B --> C["Set Hourly Rate"]
    C --> D["Convert Monetary Values to Integer Cents"]
    D --> E["Calculate Minimum Cost"]
    D --> F["Calculate Maximum Cost"]
    E --> G["Build Cost Range"]
    F --> G
    G --> H["Human Reviews Estimate"]
    H --> I["Change-Order Draft"]

The underlying relationship is:

[
\text{Estimated Cost}=\text{Estimated Hours}\times\text{Hourly Rate}
]

At $150 per hour, a 20-hour task produces a $3,000 labor estimate. The result is a calculation from supplied inputs, not a guarantee of actual implementation effort or the final project price.

9. SDLC: From Problem Definition to Release Review

I approached ScopeGhost through an iterative software development lifecycle rather than treating it as a one-shot AI code-generation exercise.

flowchart TD
    A["Problem Definition"] --> B["Requirements"]
    B --> C["Architecture and Threat Modeling"]
    C --> D["Incremental Implementation"]
    D --> E["Unit and Regression Tests"]
    E --> F["Live Integration Tests"]
    F --> G["Build and E2E Verification"]
    G --> H["Documentation and Release Review"]
    H --> I["User Feedback"]
    I --> B

Phase 1 — Requirements

I defined the central problem as scope creep in fixed-scope freelance engagements. The requirements included evidence-linked classifications, conservative handling of missing evidence, and reviewable change-order workflows.

Phase 2 — Architecture and threat modeling

I separated inference from deterministic validation and financial calculations. I also considered failure cases involving fabricated quotations, unknown clause IDs, irrelevant exclusions, and adversarial instructions inside client-provided text.

Phase 3 — Incremental implementation

The application was built in reviewable increments using typed interfaces, structured schemas, an inference client, analysis services, persistence, and user-facing workflows.

Phase 4 — Verification

The testing strategy includes unit tests, synthetic regression tests, opt-in live inference tests, browser tests, type checking, and production builds.

Phase 5 — Documentation and release review

The repository includes setup instructions, evaluation fixtures, a testing framework, and technical documentation. The M4.9 evaluation framework and Hacktoberfest submission materials have been committed to the repository.

Phase 6 — User validation

The next step is to obtain feedback from real freelancers and small agencies. Their experiences will help determine whether the evidence presentation and change-order workflow solve a sufficiently important problem to support a sustainable product.

This is an iterative SDLC approach, not a claim of formal certification or completion of every possible release process.

10. Testing Architecture and Evaluation Results

ScopeGhost separates deterministic safeguards, mock-based regression evaluation, and live model integration.

flowchart TB
    A["ScopeGhost Verification"]

    A --> B["Tier 1: Deterministic Safeguards"]
    A --> C["Tier 2: Synthetic Regression"]
    A --> D["Tier 3: Live Ollama Integration"]
    A --> E["Build and Browser Verification"]

    B --> B1["Schema and Citation Checks"]
    B --> B2["Clause and Topic Gates"]
    B --> B3["Financial Invariants"]

    C --> C1["36 Synthetic Cases"]
    C1 --> C2["Pre-Recorded Completions"]
    C2 --> C3["Production Analysis Service"]

    D --> D1["Gemma 3 4B"]
    D1 --> D2["Four Reported Integration Tests"]

    E --> E1["TypeScript"]
    E --> E2["Production Build"]
    E --> E3["Playwright E2E"]

Synthetic regression benchmark

The reported npm run test:eval results on the versioned corpus v2.0.0-synthetic are:

Metric	Result
Synthetic evaluation cases	36
Strict exact accuracy	97.2% (35/36)
Acceptable classification rate	100% (36/36)
False exclusion rate on in-scope fixtures	0% (0/6)

Important limitation: The benchmark uses DeterministicHarnessModel and pre-recorded completions passed through the production ScopeAnalysisService. It evaluates regression behavior and deterministic safeguards. It is not a benchmark of 36 independent live model responses or proof of real-world contract accuracy.

Live Ollama integration tests

The repository includes four opt-in tests using local gemma3:4b inference:

Live clause-citation extraction and verification.
Analysis orchestration and SQLite persistence.
Preservation of uncertainty when evidence is insufficient.
A prompt-injection scenario involving an invented contractual clause reference.

These tests exercise specific scenarios, not universal model robustness.

Reproduce the checks

# Unit test suite
npm test

# Synthetic regression evaluation
npm run test:eval

# Live Ollama integration suite
npm run test:integration

# TypeScript type checking
npm run typecheck

# Production build
npm run build

# Browser end-to-end tests
npm run test:e2e

The live integration suite requires Ollama and the gemma3:4b model. Consult the repository README for the exact opt-in requirements.

11. Local Deployment and Data Flow

The local-first design allows developers to run the core inference workflow on their own machines.

flowchart TB
    subgraph DEVICE["Developer Machine"]
        A["Browser"]
        B["Next.js Application"]
        C["Ollama Service"]
        D["Gemma 3 4B"]
        E["Local SQLite Database"]
    end

    A <--> B
    B <--> C
    C <--> D
    B <--> E

    F["Optional External Services"]
    B -. "If configured and invoked" .-> F

This diagram describes the intended local deployment. Optional integrations and application configuration can change the data-flow boundary, so local inference should not be interpreted as an automatic guarantee that every kind of data always remains on-device.

12. Why Does Open Innovation Matter?

Open-weight AI makes it possible to experiment with local inference, inspect application behavior, and adapt the model workflow without tying the entire product to one hosted API.

For ScopeGhost, three benefits stand out.

Data control: Local inference offers a way to process sensitive project evidence without sending model prompts to a hosted LLM endpoint, when the application is configured appropriately.

Cost flexibility: Local inference can reduce dependence on per-request API pricing, although hardware, electricity, and maintenance still have costs.

Freedom to experiment: Developers can test alternative models, adjust prompts, and improve the application while keeping the validation layer under their control.

Open innovation also makes the engineering more inspectable. Other developers can explore the source code, examine the evaluation fixtures, and understand what the system checks and where its limitations remain.

13. What's Next?

The next milestone is practical validation with real users.

flowchart LR
    A["Current Foundation"] --> B["Validate with Freelancers"]
    B --> C["Improve Audit Usability"]
    C --> D["Broader Contract Formats"]
    D --> E["Additional Export Workflows"]
    E --> F["Expanded Evaluation Corpus"]

These are potential next steps, not claims that all of these features are already implemented.

I want to learn whether freelancers find the audit results useful, whether evidence-backed findings make scope discussions easier, and whether the workflow saves enough time to justify a paid product.

Conclusion

ScopeGhost is my exploration of how open-weight AI can support a real business problem without asking users to trust an LLM blindly.

The model helps interpret natural language. Deterministic safeguards validate critical evidence. Financial logic remains reproducible. Human review stays part of the decision process.

The goal isn't to replace professional judgment. It's to give independent developers better evidence before they agree to do more work.

Explore the code and share your feedback:

GitHub: https://github.com/suvendukungfu/ScopeGhost

devchallenge #hf26challenge #opensource #ai

I Built an AI That Wants You to Stop Using It

suvendu kumar sahoo — Thu, 08 Oct 2026 09:06:56 +0000

I Built an AI That Wants You to Stop Using It

TrailLens — Look beyond the screen.

Most AI products are designed to keep you inside the interface.

TrailLens was designed to do the opposite.

You find something outside.

You capture it.

Local Gemma 3 4B helps understand it.

TrailLens turns that observation into a short, structured field mission.

And then it tells you:

PHONE DOWN.

The most important part of the experience begins when the screen stops being the center of attention.

🌿 What Is TrailLens?

TrailLens is a local AI field-experiment engine for outdoor exploration.

Its core loop is:

SEE
  ↓
GEMMA UNDERSTANDS
  ↓
MISSION READY
  ↓
PHONE DOWN
  ↓
EXPLORE
  ↓
RETURN
  ↓
REFLECT
  ↓
FIELD RECORD

The idea is intentionally simple:

Use AI to create curiosity, then get out of the way.

TrailLens is not trying to become another chatbot.

It is trying to give you a reason to look more closely at the physical world.

🧭 The Problem

A lot of digital products optimize for attention:

more scrolling
more notifications
more interactions
more time on the screen

But when you're outside, the interesting thing is usually already there.

A leaf.

A piece of bark.

A flower.

A stone.

A texture.

A pattern you normally walk past.

So I started with a simple question:

What if AI could help someone notice the physical world, then deliberately disappear from the experience?

That became TrailLens.

🌲 The TrailLens Experience

01 — SEE

You notice something interesting outside.

02 — CAPTURE

TrailLens uses the camera to capture the observation.

03 — GEMMA UNDERSTANDS

The image enters the local analysis pipeline.

Gemma 3 4B analyzes the visual input and produces structured information that TrailLens can turn into a field mission.

04 — MISSION READY

Instead of returning only an identification, TrailLens generates a structured FieldMission.

For example:

Find another nearby leaf with a similar shape and compare the number of lobes and the pattern of its main veins.

The important difference is:

the output becomes an action, not just an answer.

05 — PHONE DOWN

Once the mission starts:

PHONE DOWN.

Put your phone away.
Look.
Walk.
Notice.
Compare.

Pocket Mode intentionally minimizes the interface.

06 — EXPLORE

The user spends a short, bounded period investigating the physical environment.

07 — RETURN

When they come back:

What did you notice?

The reflection is written by the user.

Gemma does not invent the experience for them.

08 — FIELD RECORD

The session becomes a compact record containing the mission, session information, and the user's own reflection.

The application becomes a record of something the person actually did, rather than another chat transcript.

🧠 Why Gemma Matters

Gemma is not just an API call hidden somewhere inside the project.

It is the reasoning layer connecting:

VISUAL OBSERVATION
        ↓
LOCAL AI REASONING
        ↓
STRUCTURED FIELD MISSION

TrailLens uses Google Gemma 3 4B through Ollama for local inference.

That choice matters because the intended environment is the real world, where connectivity may be unreliable.

The local architecture allows the core AI inference path to run on the user's machine rather than depending on a remote AI inference service.

That gives TrailLens an architecture centered around:

local inference
reduced dependence on cloud AI availability for the core model step
a privacy-oriented image-processing path
an open-weight model that can run within a local stack

For this project, open innovation isn't just a licensing detail.

It changes the product architecture.

🏗️ System Architecture

flowchart LR
    U[User Outdoors]
    C[Camera]
    B[Browser]
    A[Next.js /api/analyze]
    V[Payload Validation]
    O[Local Ollama]
    G[Gemma 3 4B]
    M[FieldMission]
    S[Safety Gate]
    Q[Mission Quality]
    P[Pocket Mode]
    X[Outdoor Session]
    R[Reflection]
    F[Field Record]

    U --> C
    C --> B
    B --> A
    A --> V
    V --> O
    O --> G
    G --> M
    M --> S
    S --> Q
    Q --> B
    B --> P
    P --> X
    X --> R
    R --> F

The major boundaries are intentionally explicit:

Browser → Validation → Local Inference → Structured Mission → Safety → Quality → Outdoor Activity

🔬 What Happens After Gemma Responds?

I didn't want a naive:

MODEL OUTPUT
     ↓
SHOW IT TO USER

pipeline.

Instead:

sequenceDiagram
    participant User
    participant Browser
    participant Next as Next.js
    participant Ollama
    participant Gemma

    User->>Browser: Capture outdoor subject
    Browser->>Next: POST /api/analyze
    Next->>Next: Validate payload
    Next->>Ollama: Structured inference request
    Ollama->>Gemma: Multimodal analysis
    Gemma-->>Ollama: Structured FieldMission
    Ollama-->>Next: Model response
    Next->>Next: Safety validation
    Next->>Next: Mission quality evaluation
    Next-->>Browser: Analysis + Mission
    Browser-->>User: Mission Ready

A generated mission therefore passes through application-level validation before it reaches the user experience.

🛡️ Safety Is Part of the AI Design

An outdoor AI assistant shouldn't blindly turn every plausible model output into an activity.

TrailLens includes a deterministic safety layer that can reject unsafe challenge language and substitute a safe fallback.

The system explicitly accounts for categories such as:

wild plant or mushroom ingestion
harvesting or collecting specimens
dangerous terrain
inappropriate wildlife interaction

The principle is simple:

AI can create curiosity without creating unnecessary risk.

Safety is therefore treated as an application boundary, not as something the model is simply expected to get right.

🧪 A Plausible Mission Isn't Necessarily a Good Mission

One of the biggest lessons from building TrailLens was:

A mission sounding plausible does not mean it is a good field mission.

So I built a deterministic Mission Quality Evaluator.

Every mission can be evaluated across five dimensions:

Dimension	What it asks
Grounding	Is the mission connected to what was actually observed?
Specificity	Does it contain concrete actions and observable details?
Safety	Does it avoid known hazards?
Executability	Can a person realistically perform it?
Outdoor Value	Does it require real-world observation instead of screen-only thinking?

Each dimension is scored from 0–20, for a total of 100.

The fixture suite contains 10 deterministic scenarios.

The benchmark produced an average score of 85.4 / 100, while Specificity was the weakest dimension at 14.6 / 20 average.

That result was valuable.

I did not change the rubric to make the numbers look better.

The failures showed exactly where generated missions still needed improvement.

That is the point of an evaluator.

⚙️ Engineering the Local Model

TrailLens includes a reproducible Gemma benchmark suite covering:

cold vs. warm inference
model residency
token ceilings
prompt compression
image-resolution experiments
structured output
reproducibility

One repeated finding was the effect of keeping the local model resident:

Warm model-load overhead was in the tens of milliseconds in the tested runs, versus multi-second cold loading.

The project also makes an important distinction:

model-load latency is not the same thing as total end-to-end inference latency.

Absolute inference time varied across repeated runs on the same test machine, so the benchmark is documented as empirical testbed evidence rather than a universal performance guarantee.

✅ Verification

TrailLens is backed by a real automated verification suite.

Current project verification includes:

100 automated tests
ESLint with zero errors
successful Next.js production build
deterministic safety tests
Mission Quality fixture evaluation
structured FieldMission validation
local Gemma benchmark artifacts
reproducibility evidence

The repository deliberately separates different forms of evidence:

Functional Correctness
        ↓
Contract Validation
        ↓
Safety Validation
        ↓
Mission-Quality Evaluation
        ↓
Performance Benchmarking
        ↓
Reproducibility
        ↓
Real-World Field Validation

That distinction matters.

A passing unit test is not the same thing as proving that a person enjoyed a field mission.

🌿 Why This Fits "Touch Grass"

The challenge is about using open-source AI to create something that gets people away from the screen and into the real world.

TrailLens was built around that requirement from the beginning.

Typical AI interaction

QUESTION
   ↓
AI ANSWER
   ↓
MORE QUESTIONS
   ↓
MORE SCREEN TIME

TrailLens

OBSERVATION
   ↓
GEMMA
   ↓
FIELD MISSION
   ↓
PHONE DOWN
   ↓
REAL WORLD
   ↓
REFLECTION

The screen is supposed to be the shortest part of the experience.

That is the product.

🏕️ What Makes the Idea Different?

The interesting part isn't simply:

"Gemma can identify a leaf."

The interesting part is:

What happens after the identification?

Instead of ending with:

"This is probably an oak leaf."

TrailLens can turn that observation into:

"Find another nearby leaf and compare its lobes and main veins."

That changes AI from an answer engine into a field-experiment trigger.

And then the phone is supposed to disappear from the experience.

📖 The Field Record

The end product is not a chat history.

It is a field record.

A compact memory of:

what was investigated
what mission was attempted
session information
what the user personally noticed

The reflection is intentionally user-authored.

TrailLens doesn't ask the model to fabricate what the person experienced.

🛠️ Built With

Frontend

Next.js
React
TypeScript
Tailwind CSS

AI

Google Gemma 3 4B
Ollama
structured JSON / JSON Schema output

Engineering

Zod
Vitest
ESLint
Turbopack
deterministic safety rules
deterministic Mission Quality evaluation

📈 What I Actually Built

The project evolved through concrete engineering milestones.

M1 — Field Mission Contract

A structured contract between model output and physical-world activity.

M2 — Local Gemma Latency Engineering

Cold/warm benchmarking, model residency, prompt compression, token boundaries, resolution experiments, and structured output testing.

M2.1 — Reproducibility Evidence

A second complete benchmark run to separate reproducible engineering behavior from variable wall-clock latency.

M3 — Pocket Mode + Reflection + Field Records

The physical immersion and reflection loop.

M4 — Mission Quality + Grounding

A deterministic 5-dimension evaluator backed by a 10-fixture test suite.

Productization

An editorial field-guide interface, camera-pipeline hardening, SDLC documentation, contributor documentation, and engineering evidence.

🔍 What I Learned

The biggest lesson wasn't:

"Make the model smarter."

It was:

Design the system around what the model is good at — and explicitly constrain what it isn't.

That led to several architectural decisions:

structured model output
deterministic safety gates
deterministic mission-quality evaluation
performance benchmarking
reproducibility checks
Pocket Mode
user-authored reflection
explicit evidence boundaries

The result is less like a chatbot and more like an AI-powered field instrument.

⚠️ What TrailLens Does Not Claim

I want the project to be honest about its limits.

The Mission Quality system is a deterministic lexical and structural proxy. It is not a semantic judge of the physical world.

The 10-fixture benchmark is not a human user study.

Benchmark latency is measured on a specific local test environment and should not be interpreted as a universal hardware guarantee.

Actual outdoor engagement depends on people, terrain, weather, curiosity, and context.

Those limitations are documented deliberately.

Reproducibility is more useful than inflated claims.

🌎 Open Source

TrailLens is open source:

GitHub

https://github.com/suvendukungfu/traillens

The repository includes:

application source
architecture documentation
SDLC documentation
evaluation reports
benchmark artifacts
Mission Quality fixtures
contributor instructions
changelog

🚀 Run TrailLens Locally

Requirements

Node.js
Ollama

Install Gemma:

ollama pull gemma3:4b

Clone:

git clone https://github.com/suvendukungfu/traillens.git
cd triallens

Install dependencies:

npm install

Run:

npm run dev

Then open:

http://localhost:3000

🧭 The Philosophy

The entire project can be reduced to one sentence:

Use AI once. Then go outside.

Or even shorter:

AI is the bridge. The real product is outside.

That is why TrailLens exists.

🌱 Hacktoberfest 2026 — Touch Grass

TrailLens was built for the Hacktoberfest Open-Source AI Challenge — Week 1: Touch Grass.

The challenge asks builders to create something with open-source AI at its core that gets people away from the screen and into the world.

TrailLens was designed around exactly that constraint.

The most important screen in the product is the one that tells you:

PHONE DOWN.

Then the app gets out of the way.

🔗 Links

GitHub:

https://github.com/suvendukungfu/traillens

Hacktoberfest Week 1:

https://dev.to/challenges/hacktoberfest-week1-2026-10-05

License: MIT

Final Thought

Most AI assistants try to keep you talking to them.

TrailLens has a different goal:

Say something useful.

Give you something to investigate.

Then get out of the way.