I stopped asking AI to count. Now it catches 95.6% of the mistakes in crochet patterns, on a laptop

Heet Ranpura — Sun, 18 Oct 2026 16:14:25 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

Round 5 says 31 stitches. You've counted three times. You keep getting 30.

Somewhere tonight, someone is pulling out an evening's work because of a number like that. They will blame their own hands. Sometimes they're right. Sometimes the pattern itself is wrong, and there's no way to tell which.

So I gave the job to an AI. I showed an open-weight model 60 crochet patterns with 45 wrong rows hidden in them and asked it to find the mistakes. It found 20. Along the way it flagged 78 rows that were perfectly fine. Then I tried a model 30 times bigger on some of the same patterns. It still missed almost half.

The problem wasn't crochet. The problem was that I was asking a language model to do arithmetic.

So I stopped. In StitchProof, the model only reads the pattern and turns each row into a tiny program. Plain Python does all the counting. With that split, and a 4B model fine-tuned on Tinker to do the reading, StitchProof caught 95.6% (43/45) of the same mistakes, runs on a laptop with the Wi-Fi off, and shows the arithmetic behind every flag:

Rnd 5 says (31), but '(sc 3, inc) x6' uses 24 of 24 stitches and makes 30.

This post is about crochet. The idea in it is for anyone who has watched an LLM confidently get a number wrong: let the model translate, and let code count.

What I Built

StitchProof is a private tech editor for crochet patterns.

Crochet is bigger than it looks. Ravelry, the main home for knitters and crocheters, had almost 9 million registered users by 2020, and independent designers sell their patterns there and on Etsy as PDFs. Before a pattern goes on sale, many designers pay a human tech editor to check every stitch count, because one wrong number in round 5 means every maker who buys it tears out their work.

StitchProof does the stitch-math part of that job:

Who	What they do	What they get
Designers (would pay)	Paste a draft, open a `.txt`, `.md` or `.pdf`, or photograph a printed page	Every row marked OK, ERROR, WARNING or CAN'T CHECK, the exact numbers behind each flag, a silhouette chart of the piece and a downloadable report
Makers (free)	Type the stitches they had and paste the row they're stuck on	"The pattern says 31. These instructions make 30 from your 24 stitches. The pattern itself doesn't add up here, so it's not you."

The side hustle: makers use it free, which is how designers hear about it. Designers pay per pattern check, or a small monthly fee if they publish often. A per-check price only works if each check costs nothing to run, which is exactly what a small local model gives me. (That's my business hypothesis, not a claim about revenue.)

Demo

The waits for the model and for Gemma are sped up 3x in the video. Everything you see runs on one MacBook.

Every screen, one at a time

1. Start. Paste a pattern, open a file, or photograph a printed page.

2. Check a pattern. The sample has two deliberate mistakes. Rows are annotated in place as the model reads them, like a spellchecker. Select a row and the panel shows the stitches available, used, made and stated.

3. The second mistake. Rnd 11 repeats (sc 2, dec) 5 times instead of 6, so it makes 15, not 18, and leaves 4 stitches unworked. StitchProof reports both problems.

The silhouette plots stitches per round. For amigurumi it traces the outline of the finished piece, so a mistake shows up as a spike or a dent, hatched and labelled.

4. Fix and check again. Edit Rnd 5 and re-run. Unchanged rows come straight from a local cache, so only the edited row goes back to the model.

5. Light or dark.

6. Printed patterns. Plenty of patterns live in books and on paper. Drop in a photo and Gemma 4 transcribes it on your computer.

7. Review before checking. The photo sits beside the editable transcription, so you can fix any misread before a single stitch is counted.

8. Stuck on a row. The maker mode: one row, one plain answer.

Code

HeetRanpura / stitchproof

A local tech editor for crochet patterns: a fine-tuned 4B model reads each row, plain code checks every stitch count.

StitchProof

A local tech editor for crochet patterns. A small fine-tuned model reads every row. Plain code checks every stitch count. Nothing leaves your computer.

Demo video · Results · How it works · Architecture · Run it

Round 5 says 31. You count 30.

You've unravelled it twice. Is the pattern wrong, or are you?

Crochet designers sell their patterns as PDFs on Etsy and Ravelry. Before publishing, many pay a human tech editor to check every single stitch count, because one wrong number in round 5 means hundreds of makers tearing out an evening's work. Errata are a fact of life.

So why not ask an AI? We tried. Given 60 test patterns and asked directly to find the rows whose counts are wrong, the open-weight Qwen3.5-4B caught 44.4% (20/45) of the mistakes. Chatbots are famously bad at crochet. They "count" stitches they never counted.

StitchProof never asks…

View on GitHub

MIT licensed. 149 Python tests, plus frontend tests, run in CI on every push. Every result in this post comes from files in eval/results/, and every rate is shown with its count.

Built on open source: Qwen3.5 (Qwen team), Gemma 4 (Google DeepMind), Tinker and tinker-cookbook (Thinking Machines), llama.cpp, Ollama, FastAPI and React. Full credits are in the README.

How I Built It

The one rule: the model never counts

Segment. The pattern is split into lines. Each line goes to the model with a short header (title, US terms, rounds or rows) and the line before it, for context.
Translate. A fine-tuned Qwen3.5-4B turns one line into one LineProgram JSON object: work, increase, decrease, skip, "to end", turning chain, repeat, plus the row label and the stated count, copied exactly as written, even when it's wrong. That last part is the whole trick: the model has to faithfully transcribe the designer's mistake so the code can catch it.
Simulate. A deterministic engine walks the rows in order: magic rings, foundation chains and skipped chains, turning chains that do or don't count, repeats that must divide "around", ranges like Rnds 6-9. It reports COUNT_MISMATCH, OVERFLOW, LEFTOVER, REPEAT_REMAINDER, NO_STATED_COUNT or CANT_CHECK.
Explain. Every verdict is a fixed template filled with numbers. Nothing the user reads about a count is generated by a model.

The model's instructions say it plainly (prompts.py):

- Transcribe exactly. Never fix or compute counts. stated_count = the count
  written ("(18)", "[18 sts]", "- 18 st", "you should have 18"), even if wrong.

And this is what "counting" looks like on the other side, straight from simulate.py. No model anywhere near it:

def op(self, op: Op, tail: int = 0) -> None:
    if isinstance(op, Work):          # "sc 3": 1 stitch into each of the next 3
        self.take(op.n)
        self.made += op.n
    elif isinstance(op, IntoOne):     # "inc": 2 stitches into 1
        self.take(1)
        self.made += op.k
    elif isinstance(op, Together):    # "dec": 2 stitches worked together into 1
        self.take(op.k)
        self.made += 1
    ...

When the code can't be sure (lace, multi-size patterns, UK terms, short rows, a line the model couldn't read), the row says CAN'T CHECK with a reason. It never guesses. After a wrong row, the engine re-syncs to the designer's stated count, so one mistake is reported once instead of cascading down the whole pattern.

Architecture at runtime

Streaming: results arrive row by row over Server-Sent Events, so the sheet fills in as the model reads. That's exact, not a trick: each row's count depends only on the rows before it.
Per-line cache: each parsed line is cached by a hash of (header, previous line, line), so re-checking an edited pattern only sends the changed rows.
Private by construction: the server binds to 127.0.0.1, fonts and icons are bundled, there's no telemetry, and uploaded photos are read from a temp file and deleted.
Stack: FastAPI + Uvicorn, React 19 + Vite + Tailwind, llama.cpp (llama-server, Metal), Ollama for Gemma 4, pypdf, Jinja2 and Matplotlib for reports.

Building the model on Tinker

No published patterns were used anywhere. Real patterns are someone's copyrighted work, so every training example is generated by my own code:

shapes.py builds pattern structures: spheres, tubes, cones, flat pieces, triangles, mixed stitches.
errors.py seeds known mistakes (wrong stated count, wrong repeat count, wrong run length, wrong foundation chain, a repeat that doesn't divide "around"). Every seeded pattern is checked: the simulator must flag exactly the seeded rows.
render.py writes the notation in many styles. Four styles and the whole cone shape are held out of training and used only for evaluation.
paraphrase.py asks two teacher models on Tinker, gpt-oss-120b and Qwen3.5-397B-A17B, to rewrite some patterns in a natural designer's voice. Code then checks that every number and row label survived the rewrite.
ood.py asks a third family, Kimi-K2.6, to write 25 patterns from structure alone, with none of my templates. That's the test for "did it learn crochet, or just my generator?"

Training was a LoRA on Qwen/Qwen3.5-4B with tinker-cookbook's supervised loop: rank 32, one epoch, 1,410 line examples, thinking disabled. Validation loss went from 0.303 to 0.0028.

The whole build cost US$1.62 of Tinker credit (costs.md):

Model on Tinker	Used for	US$
Qwen3.5-4B	LoRA training (1 epoch + validation)	1.018
Qwen3.5-4B	sampling for baselines, the fine-tuned eval, the export check	0.359
Kimi-K2.6	writing the out-of-generator test set	0.099
Qwen3.5-397B-A17B	training-data rewrites	0.084
gpt-oss-120b	training-data rewrites + the large-model baseline	0.041
storage	checkpoints	0.017
Total		1.619

Results

Four approaches, the same eval sets, the same prompt and the same parser, at temperature 0:

A: base Qwen3.5-4B, asked directly to find the rows with wrong counts
B: gpt-oss-120b asked the same way (a much bigger model, run on 15 patterns as a reference)
C: base Qwen3.5-4B reading each line + the simulator
D: StitchProof: fine-tuned Qwen3.5-4B reading each line + the simulator

	A: base, asked directly	C: base + simulator	D: StitchProof
Held-out: errors caught	44.4% (20/45)	75.6% (34/45)	95.6% (43/45)
Held-out: false alarms	13.7% (78/571)	42.6% (243/571)	22.2% (127/571)
Other model family: errors caught	61.1% (11/18)	100.0% (18/18)	100.0% (18/18)
Other model family: false alarms	11.2% (23/205)	50.7% (104/205)	8.3% (17/205)
Other model family: whole pattern right	32.0% (8/25)	16.0% (4/25)	80.0% (20/25)
Valid JSON (held-out)	86.7% (52/60)	78.0% (663/850)	100.0% (850/850)
Stated count copied faithfully (held-out)	n/a	67.4% (415/616)	100.0% (616/616)

What the table says, in plain words:

Asking a model to count doesn't work, at any size. A misses more than half the mistakes. B, roughly 30 times bigger, caught 54.5% (6/11) with 38.3% (62/162) false alarms.
The simulator alone isn't enough either. C catches more, but raises false alarms on 42.6% of clean rows, because the base model can't reliably write the program format. Only 78.0% of its outputs are even valid JSON.
Fine-tuning the reader is what makes it work. D writes valid JSON every time, copies every stated count faithfully, and on patterns written by a model family it never saw, gets the whole pattern right 80% of the time with 8.3% false alarms.
It's cheaper to run, too. On the held-out set, D needed about 40% fewer output tokens than C (32,366 vs 54,442) for the same rows, because it writes the program cleanly the first time.

Where it still fails (measured, not hidden)

Almost all of D's false alarms on the held-out set come from one notation it never saw: *sc, inc; rep from * 4 more times, which means 5 repeats in total. The model writes 4. On rows in that style it raises false alarms on 97.5% (115/118) of clean rows, and it's also why D catches only 66.7% (4/6) of the wrong-repeat-count mistakes. On rows in the other three unseen styles, D parses 99.8% (466/467) exactly, with 2.7% (12/438) false alarms.

I didn't retrain on that style, because that would be tuning on the test set. It goes into the next version's training templates, with fresh held-out styles to test against.

Getting the model out of the cloud

Patterns are a designer's unreleased work, so the fine-tuned model had to run locally. I exported the LoRA with Tinker's weights API, converted it for llama.cpp and run it on a 4-bit base:

Disk	RAM	Load time	Per row (M4 MacBook)	Agreement with Tinker
2.93 GB	3.18 GB	2.5 s	2.36 s mean	100.0% (50/50)

One gotcha worth sharing with anyone fine-tuning Qwen3.5 on Tinker:

Tinker trains the linear-attention Q, K and V projections as separate LoRAs, but Qwen3.5 stores them as one fused in_proj_qkv weight, so llama.cpp can't load the adapter as-is. An exact fused LoRA stacks the A matrices and puts the B matrices on a block diagonal (export.py):

for prefix, g in groups.items():
    a = torch.cat([g["qA"], g["kA"], g["vA"]], dim=0)            # rank r -> 3r
    b = torch.block_diag(g["qB"], g["kB"], g["vB"]) * 3.0       # llama.cpp scales by alpha / rank
    out[f"{prefix}.linear_attn.in_proj_qkv.lora_A.weight"] = a.contiguous()
    out[f"{prefix}.linear_attn.in_proj_qkv.lora_B.weight"] = b.to(g["qB"].dtype).contiguous()

The * 3.0 matters: the fused LoRA has rank 3r, and llama.cpp divides by rank, so without it every update would be a third of what Tinker trained. The equivalence check (50 validation lines, same prompt and parser, local vs Tinker) is what proved the conversion exact.

Reading paper patterns with Gemma 4

Gemma 4 (gemma4:e4b-it-qat in Ollama) has one job: turn a photo of a printed pattern into text, locally. It never counts either. On 12 held-out patterns rendered as a clean scan and as a phone-style photo (slight rotation, uneven light, noise, JPEG):

	Typed text	Scan → Gemma	Photo → Gemma
Numbers kept	n/a	98.9% (357/361)	98.1% (354/361)
Errors caught, end to end	100.0% (12/12)	91.7% (11/12)	100.0% (12/12)
False alarms, end to end	23.2% (22/95)	25.3% (24/95)	22.1% (21/95)
Lines character-perfect	n/a	50.3% (81/161)	42.2% (68/161)

Gemma keeps the numbers but likes to drop separators like " - ". That's why the app always puts the photo next to the editable transcription before anything is checked.

How the build itself ran

I wrote a spec with milestones and checkpoints, then built StitchProof with Claude Code against it, stopping at each checkpoint to review the counting rules, the baselines and the results before moving on. The spec is in the repo as SPEC.md, and the counting rules are in CONVENTIONS.md, each with its own test.

Why Does Open Innovation Matter?

Every one of these would have been impossible, or worse, with a closed API:

I could fix the model, not just the prompt. A general model, even a 120B one, can't read crochet notation reliably. A LoRA on an open 4B model, trained for about a dollar, took valid output from 78.0% to 100.0% and whole-pattern accuracy on unseen writing from 16.0% to 80.0%. You can't do that to a model you only rent.
The designer's work never leaves their machine. An unreleased pattern is a designer's product. With open weights on their own laptop, there's no "we don't train on your data" to trust. Turn the Wi-Fi off and StitchProof still works.
A check costs nothing, so the business works. Each check is electricity, not an API bill. That's what makes a cheap per-pattern price possible, and lets makers use it free.
I could open it up and measure it. Every model choice, prompt and parser is in the repo, and every claim above can be re-run from it. The weak spot is in the results because I could look inside and find it.
No lock-in. The base model, the LoRA and the GGUF files sit on disk. If a better open model comes out, swapping it in is a training run and an export, not a migration.

My First User

My first user is a family member who sells handmade hexagon cardigans and oversized patchwork sweaters on Etsy, made from her own patterns.

Her work shows why stitch math matters to a seller. A hexagon cardigan is dozens of identical motifs, each worked in rounds that have to land on the same count, and a patchwork sweater is panels that have to line up with their neighbours. One wrong number in her pattern doesn't cost one piece. It costs every motif made from it.

She ran StitchProof on one of her own patterns, on her own laptop. The pattern never left the machine, and it isn't in the training data or in any of the results above, which were all measured on generated patterns.

Her patterns also show exactly where v1 stops. Hexagons with chain-space corners and sweaters written for several sizes come back as CAN'T CHECK today, so those two are first on the list below. If you design or sell crochet pieces and have a draft you'd let me check, I'd love to hear from you in the comments.

Prize Categories

I'm entering Best Use of Tinker, Best Use of Gemma and Best Use of GitHub Copilot.

Best Use of Tinker

The category asks for a fine-tune with a clear improvement over a baseline. StitchProof's case:

The improvement is measured by code, not by opinion. Because the model only transcribes, I can score its programs against gold programs and let the simulator count. Same prompt, same parser, same eval sets for every approach.
A real baseline, not a strawman. C is the same base model with the same simulator. The only difference between C and D is the Tinker fine-tune: errors caught 75.6% → 95.6%, false alarms 42.6% → 22.2%, valid JSON 78.0% → 100.0% on held-out styles; false alarms 50.7% → 8.3% and whole-pattern accuracy 16.0% → 80.0% on another model family's writing.
Better on cost too: about 40% fewer output tokens than the base model for the same rows.
Tested against overfitting on purpose. Held-out notation styles, a held-out shape, and a test set written by a third model family. The one failure is reported with its exact numbers.
I used Tinker for the whole loop: two teacher families for training data, a third for the out-of-generator test set, every baseline, the training run, and the export check. Total: US$1.62.
The model leaves Tinker and runs anywhere. Exported, converted to GGUF, and verified at 100.0% (50/50) agreement on a 4-bit base. The fused Q/K/V LoRA fix is reusable by anyone training Qwen3.5 on Tinker who wants to run it with llama.cpp.

Best Use of Gemma

Gemma 4 runs locally and does a real job no other part can: reading paper. Plenty of patterns live in books and printouts, and without Gemma those users can't use StitchProof at all.
Measured, not just wired in: 98.1% (354/361) of numbers kept from phone-style photos, and 100.0% (12/12) of errors still caught end to end.
Designed around its weakness: only 42.2% of photo lines come back character-perfect, so the UI shows the photo beside an editable transcription before checking, instead of trusting it blindly.
Fits the privacy promise: gemma4:e4b-it-qat through Ollama, on the same laptop. The photo is read from a temp file and deleted.

Best Use of GitHub Copilot (GitHub Actions)

The category includes automating your project with GitHub Actions. StitchProof uses two workflows:

CI on every push and pull request: 149 pytest tests (simulator golden tests, one per counting rule and error code, parser robustness, generators, the API), plus frontend typecheck, Vitest and a production build. It fails if the committed UI build doesn't match the source, so the app you run is always the app in the code.
A manual eval workflow (Eval (synthetic sets)): it runs approaches C and D on Tinker with a hard spending cap (--max-usd 3 per run), writes the results table into the job summary and uploads the raw results as artifacts. It never touches the private data/real/ folder.

Limitations and What's Next

Training data is synthetic. The only real pattern checked so far is my first user's, and it isn't part of the measured results.
Notation the model hasn't seen can trip it up; "rep from * N more times" is the measured example.
v1 covers US terms, amigurumi rounds and flat rows with common stitches. Charts, lace, multi-size patterns, UK terms, short rows and Tunisian crochet are CAN'T CHECK, with a reason.
Designers don't all count the same way; StitchProof follows CONVENTIONS.md and always shows the numbers so a human can judge.
It checks stitch math only, not gauge, fit or how the finished piece looks.

Next: support chain-space corners and multi-size patterns, since both show up in my first user's work; add the missing notation styles to training and test on fresh held-out ones; check more real drafts with designers; package the model for Ollama; and try the same translate, simulate, verify design on knitting.

If you crochet, design patterns, or just have opinions about LLMs and arithmetic, I'd love to hear what you think in the comments.

DEV Community: Heet Ranpura