ScopeTrace — find the clause, then question the promise

Nerion Dev — Sun, 18 Oct 2026 03:48:25 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle.

AI disclosure: AI assistance was used for development, evaluation, independent agent reviews and this write-up. All examples are fictional. The measurements come from actual local runs; the limitations and failed cases are included below.

What I Built

The brief says USD 80. The proposal says EUR 80. Both sentences are about the same thing, and an embedding model can score them as a strong match. That is exactly why a match should start a review rather than finish it.

ScopeTrace is a local second look at a freelance brief and proposal. Put one requirement on each line, paste the proposed deliverables, and retrieve up to three related clauses per requirement. Each suggestion shows a literal quote and its source line. You can inspect alternatives, select evidence, record a review decision and leave a question for the next conversation. Downloading the JSON report keeps those decisions alongside the original text; unreviewed items remain unreviewed.

This is a narrow side-hustle hypothesis: a freelancer or small software team might value finding omissions and wording differences before agreeing to a fixed scope. It is not a contract checker. No paying customer, time saving or willingness to pay has been established.

The example workflow is an import script. One proposal clause correctly paraphrases the requested error log. Another changes the currency. Another uploads records despite the request for local processing. These are different review tasks, even when all three produce related text.

Demo

Watch the local application walkthrough.

The video is a captioned sequence of screenshots of the running application, operated by an AI agent with synthetic text. It is not a user study or a claim that the human author performed those clicks. It shows the normal sample, a review note, a downloaded report and the known error-log false positive.

Code

Source, setup instructions and license.

The application code is MIT licensed. The model is downloaded separately from its pinned upstream revision; its Apache-2.0 declaration and the runtime dependencies are credited in the third-party notices. Model weights, local environments, client documents and private agent-session histories are excluded from the source package.

How I Built It

The central retrieval step runs sentence-transformers/all-MiniLM-L6-v2, an open model, through ONNX Runtime on a Windows CPU. No fine-tuning or hosted inference API is involved. The pinned revision is 1110a243fdf4706b3f48f1d95db1a4f5529b4d41.

The requirement and proposal passages are tokenized into overlapping chunks of at most 256 wordpieces, including special tokens. Masked mean pooling and L2 normalization follow the model card's embedding recipe. Cosine similarity ranks related passages. Original offsets and literal quotes make each suggestion inspectable; the browser displays the returned quote instead of applying JavaScript's different string-indexing convention to Python offsets.

An explicit rules layer adds wording cues for recognized currency codes, numeric digits, duration phrases, platform names and negation words. A bare currency symbol needs clarification. These rules can miss context or flag harmless variations: they do not associate every amount with its service, convert business days to calendar time or resolve the scope of every negation. No cue means that these rules found nothing, not that the clauses agree.

The interface uses plain HTML, CSS and JavaScript. A Python server binds only to 127.0.0.1, accepts bounded same-origin JSON requests and serves a fixed set of static files. Pasted content is rendered as text. The server does not write document history or request logs, and the interface has no analytics, cloud inference, database or localStorage. A report downloaded by the user does contain the pasted text and notes.

Compare against a small baseline before adding a polished screen

One set of 16 synthetic English cases was used for development. Another agent prepared 16 reserved cases, sealed before thresholds were chosen. The retrieval core and evaluation code, model manifest, package versions and thresholds were fixed before opening that second set. The interface and wording checks were added afterward.

Method	Relevant top result / 13 positive cases	Relevant result in first three	Correct / accepted suggestions
MiniLM	12/13	13/13	12/14
Jaccard	4/13	12/13	4/15
BM25	4/13	12/13	4/14

The ranking columns do not apply an abstention threshold. The final column uses a separate development threshold for each method. These are case-level retrieval results, not recall of every relevant clause. The lexical baselines already found a relevant passage in their top three for 12 of 13 positive cases; the larger observed improvement was which passage appeared first.

The raw reports preserve the examples and failures. The reserved set is now open and will not be presented as unseen evidence after any future tuning. Small, non-random, agent-authored English examples do not estimate commercial accuracy.

Keep the mistakes in the demo

Two accepted semantic suggestions were wrong:

For operation on a disconnected laptop, an installer shortcut outranked the actual no-internet clause, which came third. The internal cause of that order was not established.
For a timestamped error-log requirement, no candidate offered a log. The model still accepted a sentence about saving PNG/JPEG files. It correctly rejected only two of the three absent cases.

The Known failure button loads that already-open error-log case. Its purpose is to let a reviewer see the limitation directly, not to count the same case as a second independent evaluation. Contradictory sentences can also have high similarity. The application never turns a score into an automatic approval.

Modest local requirements

The pinned model files total about 90.9 MB. A separate short four-candidate harness query took 5.44 ms median over 20 warm repetitions, with about 198.4 MiB peak process working set. Those are harness measurements, not full-document application latency, concurrent-user capacity or boot-cold startup. Windows file-cache state was not controlled. The application was tested on CPU; no GPU speed claim is made.

The reviews found and corrected concrete implementation errors before preparing the public package, including malformed request lengths, unresolved currency symbols and differing duration wording. Test counts for HTTP validation and literal quotes are not extra cases of semantic accuracy.

Why Does Open Innovation Matter?

The open model makes the core operation runnable on the same computer as the documents without a per-query hosted inference bill. A pinned revision and visible evaluation also let someone inspect what produced the suggestions instead of trusting a changing remote endpoint. Replacing or fine-tuning the model is possible future work, not a feature evaluated here.

Initial package and model setup requires internet. The application loads local files and has no cloud inference call. A Python-level network guard observed no socket requests during the measured model run, but that is not operating-system or native-code network isolation. This is a local prototype, not a certified privacy guarantee; browser extensions, OS memory and exported files remain outside that narrow claim.

Upstream credit matters: MiniLM's model card documents the model and pooling recipe. ONNX Runtime, Hugging Face Tokenizers and NumPy have their own licenses. ScopeTrace does not claim to have trained the model.

My First User

No external-user study or payment has been recorded. The app has been exercised by agents with invented briefs and proposals, including deliberate failures. The next useful question is whether a freelancer finds the side-by-side evidence and clarification notes worth using on an actual scope review. That remains untested.

Prize Categories

Overall challenge only. No partner category is claimed for technology that this project does not use.

DEV Community: Nerion Dev