CSV Preflight: a catalogue check that refuses to guess your price column

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle.

What I Built

A product spreadsheet can look tidy and still be a bad import. Two rows can share the same SKU. A price can be negative. A column called value might mean a sale price, a supplier cost, or something else entirely.

CSV Preflight is a small browser tool for a shop owner or catalogue freelancer who wants to find those problems before importing a file. It uses an open embedding model to suggest four column mappings, asks the user to review them, and produces a downloadable audit with the original source lines.

The important boundary is simple: the model suggests the meaning of a heading; deterministic code checks the records. The app does not change a price or rewrite a product.

A plausible paid use would be a catalogue freelancer providing a reviewed preflight report alongside a shop's import preparation. The deliverable would be a concrete list of records to fix, not a claim that an importer will accept the file. This is a business hypothesis: there is no paying customer, sale, or demonstrated willingness to pay yet.

Demo

Open the working browser demo

1. Choose Load synthetic sample.
2. Click Suggest column mapping. The sample deliberately uses unfamiliar headings: stock reference, merchandise name, selling price, and money denomination.
3. Review the four selections and tick the confirmation box.
4. Click Check catalogue, then Download audit JSON.

The five-record sample produces six findings affecting three records: a duplicate SKU, a negative price, a missing SKU, a non-finite price, an empty title, and an unsupported currency. The source CSV remains unchanged.

The Run local model validation button reproduces the synthetic mapping experiment in the same browser. The initial model download can take a while; this is not a server-side AI call.

Code

Source, MIT application licence, and reproducible evidence

The frozen release is commit 2d1805f. Its 30 files were checked against local file sizes and Git blob hashes after publication. GitHub Pages serves the demo, model, and runtime from the project directory.

For a local copy, run node server.mjs with an existing Node installation and open http://127.0.0.1:8766/. No package installation or paid API account is required. node tests.mjs runs the deterministic core tests.

How I Built It

The new application, parser, mapping rules, interface, and fixtures were built on October 13 during this challenge's window.

The model is Xenova/all-MiniLM-L6-v2, pinned to revision 751bff37182d3f1213fa05d7196b954e230abad9, using its quantized ONNX weights. Transformers.js 3.8.1 runs it through the browser's ONNX WASM backend. The model maps text to 384-dimensional embeddings; it does not generate replacement catalogue records.

The mapping flow starts with exact aliases such as sku or currency. For unfamiliar headings, it compares their embeddings with descriptions of the four target fields. A candidate must exceed similarity 0.35 and beat the next candidate by a margin of 0.08. These are prototype thresholds, not calibrated probabilities.

Weak matches remain unselected. Column collisions remain unselected. Multiple explicit price candidates also remain unselected. A spreadsheet containing both retail price and sale price should make the operator choose; silently picking one would be the wrong kind of convenience.

That last rule came from a failed development case. The first implementation correctly handled 22 of 24 expected field decisions on six synthetic fixtures. Adding the explicit multiple-price guard raised that to 23 of 24. The initial result is retained in the repository, rather than replaced by the better run.

Four additional fixtures were then frozen separately, after the guard was implemented:

Fixture group	Expected field decisions	Exact aliases correct	Model mapping correct
Six development fixtures	24	16	23
Four frozen holdout fixtures	16	8	14

In the holdout group, two expected mappings were left unselected because the evidence was weak; they count as incorrect decisions in that table. There were no incorrectly selected mappings in those four fixtures. Correct abstention is counted only when the fixture expects an abstention.

This is a small synthetic experiment. It does not establish general accuracy, multilingual support, or a customer's saved time. The holdout was not used to tune the thresholds.

The parser and audit passed 15 core tests covering quoting, newlines, BOM handling, malformed CSV, row widths, IDs, numeric boundaries, and ambiguity. The deployed demo was also exercised: actual model suggestions, the six-finding sample report, and the real JSON download were checked. The downloaded JSON matched the displayed report.

Why Does Open Innovation Matter?

An exact-alias dictionary alone could not map any of the sample's four unfamiliar headings. The open model recovered all four. That is the specific contribution of AI here; checking whether a price is negative does not need a model.

Open weights also make this implementation practical to inspect and run. Model downloads from remote model hubs are disabled at runtime, and the pinned model and runtime files are included with the app. The deployed test's recorded model resources came from the same GitHub Pages origin. CSV records are processed in the browser, and the mapping model receives headings rather than product records.

The supplied loopback server adds a same-origin content-security policy. The public static deployment should not be mistaken for that server's security-policy guarantee. This is an implementation with inspectable data flow, not a claim of a completed privacy or security audit. A local copy can be served from the included assets without a hosted inference service; offline service-worker behaviour was not tested.

The original quantized weights are about 23 MB, so the trade-off includes an initial download and browser memory use. The first successful public model load in the test took about 34 seconds including resource loading; it was not a controlled clean-cache or mobile benchmark. Later mapping uses the model already loaded in memory.

The model and Transformers.js retain their Apache-2.0 licences; the ONNX runtime licence is included separately. Those general-purpose model/runtime assets were reused from the earlier Outside in 20 prototype, with provenance and file hashes preserved in MODEL_PROVENANCE.json. The earlier application's code was not reused as this new project's implementation. The new application code is MIT licensed.

Where This Prototype Stops

The current scope is English comma-delimited UTF-8 CSV, a 1 MB file, at most 30 columns and 5,000 records, and the four product fields. Currency checks use a displayed subset, and SKU comparisons are case-sensitive after trimming. This is not a Shopify compatibility certificate, a tax check, or an automatic repair service.

The next useful experiment would be to let a real catalogue operator try a non-sensitive file, observe where the proposed mapping needs correction, and ask whether the report removes enough work to justify paying for it. That experiment has not happened yet.

For now, the result is a working, testable starting point: use the model where language is fuzzy, keep record checks deterministic, and leave a doubtful price column blank.