DEV Community: Kamelyoul

Filtering a 2 GB CSV in the browser without uploading it: streams, a Web Worker and a 64-bit hash set

Kamelyoul — Wed, 07 Oct 2026 08:24:36 +0000

Excel stops at 1,048,576 rows. Most online CSV tools want you to upload the file first, which is a non-starter for a customer list or an accounting export. I wanted something in between: open a multi-gigabyte CSV in a normal browser tab, filter it, remove duplicates, split it into Excel-sized files, and never send a byte of it anywhere.

The result is CSV Géant, a static site with no framework and no runtime dependency. The UI is in French by default (there's an EN button in the header). This post covers the architecture, the actual code, and numbers I measured on my own machine, including the parts that don't stream.

The architecture

Main thread: UI only. It sends the File object to a module Web Worker with postMessage. A File is a handle, so this doesn't copy the content.
Worker: reads the file in 8 MB slices, decodes, parses, filters, deduplicates and writes the output. Progress messages go back to the UI.
Output sinks: either an in-memory Blob that becomes a download, or, in Chrome/Edge, files written directly into a folder the user picks (File System Access API).
Server: one Cloudflare Pages Function with three routes: anonymous counters, license check, and the price/purchase link. No CSV content, row or column name goes through it. The CSP has connect-src 'self', and once the page is loaded the tool keeps working offline.

The core (csv.js and traitement.js) has no DOM dependency, so the same code runs in Node for the tests (fs.openAsBlob gives a Blob-like file).

1. Reading: `slice()` plus a streaming `TextDecoder`

The whole read loop is short. Identifiers are in French; parcourir means "walk through":

export const TAILLE_BLOC = 8 << 20; // 8 Mo lus à la fois

export async function parcourir(fichier, { encodage, sep, surEnregistrement, surBloc }) {
  const decodeur = new TextDecoder(encodage); // retire le BOM éventuel
  let index = 0;
  const lecteur = new LecteurCSV(sep, (champs) => surEnregistrement(champs, index++));
  for (let o = 0; o < fichier.size; o += TAILLE_BLOC) {
    const tampon = await fichier.slice(o, o + TAILLE_BLOC).arrayBuffer();
    lecteur.pousser(decodeur.decode(new Uint8Array(tampon), { stream: true }));
    if (surBloc) await surBloc(Math.min(o + TAILLE_BLOC, fichier.size), index);
  }
  lecteur.pousser(decodeur.decode());
  lecteur.terminer();
  return index;
}

{ stream: true } matters: an 8 MB boundary can fall in the middle of a multi-byte UTF-8 character ("é" is two bytes). In streaming mode the decoder keeps the incomplete bytes and prepends them to the next chunk.

Encoding detection runs on the first 2 MB: BOM first, then a strict UTF-8 decode (new TextDecoder("utf-8", { fatal: true })), and Windows-1252 if that throws. Since the sample can itself end mid-character, a helper strips a truncated trailing sequence before the strict decode. Without it, a valid UTF-8 file would be misdetected as Windows-1252 whenever the 2 MB cut lands inside an accented character.

The delimiter (;, ,, tab, |) is guessed by parsing the sample with each candidate and keeping the one that gives the most regular column count (at least 2 columns), then the most columns.

2. A parser that can stop anywhere

The parser is fed arbitrary chunks of text, so a record can be cut anywhere, including inside a quoted field that contains newlines. The internal #analyser returns the index of the first character it could not consume as a complete record, and the caller keeps that tail for the next chunk:

pousser(morceau) {
  const s = this.reste ? this.reste + morceau : morceau;
  const consomme = this.#analyser(s, false);
  this.reste = consomme < s.length ? s.slice(consomme) : "";
  if (this.reste.length > LONGUEUR_MAX_ENREGISTREMENT) {
    throw new Error("ENREGISTREMENT_TROP_LONG");
  }
}

LONGUEUR_MAX_ENREGISTREMENT is 16 MB. Without it, a single unclosed quote (or a wrong delimiter/encoding choice) would make reste grow until the tab runs out of memory. With it, the user gets an error message that points at the likely cause.

The parser handles RFC 4180 quoting (doubled quotes, newlines inside quotes), \n, \r\n and \r, and tolerates text after a closing quote ("abc"def). Rows with an unexpected number of columns are counted and reported, not dropped.

3. Deduplication: 64-bit fingerprints in a `Uint32Array`

Storing every distinct row in a JS Set doesn't work at 20 million rows. Instead, each row (or the chosen column, optionally trimmed and lower-cased) is reduced to two independent 32-bit hashes, stored in a flat open-addressing table:

#inserer(h1, h2) {
  const masque = this.capacite - 1;
  let p = (h1 ^ Math.imul(h2, 0x9e3779b1)) & masque;
  const t = this.table;
  for (;;) {
    const a = t[p * 2];
    const b = t[p * 2 + 1];
    if (b === 0) {
      t[p * 2] = h1;
      t[p * 2 + 1] = h2;
      this.taille++;
      return true;
    }
    if (a === h1 && b === h2) return false;
    p = (p + 1) & masque;
  }
}

h2 is never 0, so 0 marks an empty slot. The table doubles when it would be more than half full.

Two honest consequences:

Memory is 16 to 32 bytes per distinct key, depending on where you are in the doubling cycle (8 bytes per slot, load factor between 25% and 50%). For the 21.3 million distinct rows in my test file, the final table is 512 MiB, plus the previous 256 MiB table while it's being copied.
It's probabilistic. Two different rows with the same 64-bit fingerprint would be treated as duplicates. The probability is about n²/2⁶⁵, so roughly 1 in 80,000 for 21 million distinct rows. For a deduplication tool, I decided that was acceptable; you may disagree.

4. Writing: BOM, CRLF, and two sinks

Output rows are serialized (quoted only when needed, CRLF line endings), accumulated in a string, and encoded with TextEncoder in batches of about 4 million characters. Each output file starts with a UTF-8 BOM and repeats the header row. The BOM is what makes Excel read accents correctly instead of guessing a legacy encoding. Splitting "for Excel" means 1,048,575 data rows per file, plus the header.

The download sink groups the encoded chunks into 64 MB Blobs as it goes:

async ecrire(octets) {
  const c = this.courant;
  c.morceaux.push(octets);
  c.enAttente += octets.length;
  c.taille += octets.length;
  if (c.enAttente >= 64 << 20) {
    c.blobs.push(new Blob(c.morceaux));
    c.morceaux = [];
    c.enAttente = 0;
  }
}

The whole result is still assembled before the download starts. For results over 1 GB, the UI recommends the other sink: showDirectoryPicker(), then one createWritable() stream per output file. That one only exists in Chromium browsers.

5. The part that doesn't stream: sorting

You can't sort a file you only see 8 MB at a time without an external merge sort, and I haven't built one. So sorting keeps the filtered result in memory, as compactly as I could manage: each row is stored already serialized in UTF-8 inside 16 MB blocks, numeric and date keys go into a Float64Array, and what gets sorted is a Uint32Array of indices:

const ordreIdx = new Uint32Array(this.n);
for (let i = 0; i < this.n; i++) ordreIdx[i] = i;
ordreIdx.sort((a, b) => cmp(cles[a], cles[b]) || a - b);

The || a - b makes the sort stable. Hard limits: 3 million rows or 600 million characters. Beyond that, the export stops with an explicit message (filter first, then sort).

Measured numbers

Conditions: AMD Ryzen 5 4500U (6 cores), 16 GB RAM, NVMe SSD, Windows 11. Chrome 154 headless driven by Playwright (the tests/e2e-tres-gros.mjs script from the repo), app served from localhost. Test file generated by the repo's scripts/generer-csv.mjs: 22 million rows, 9 columns, 2.22 GB, semicolon-delimited UTF-8 with accents, about 427,000 quoted fields containing line breaks, and 660,441 exact duplicates. Each result was checked against the expected counts written by the generator.

Operation (Chrome, 2.22 GB file)	Time
Preview (encoding + delimiter detection, first 100 rows)	0.3 s
Exact row count (22,000,000)	17.4 s / 17.8 s
Deduplicate everything + split into 21 Excel-sized files, written to a folder (2.18 GB out)	102.2 s / 106.9 s
Filter `pays` (country) `= FR` + deduplicate, single download (8,001,769 rows, 0.82 GB out)	71.8 s / 72.0 s

Two runs each: roughly 125 MB/s for counting, 20 to 30 MB/s once every row is hashed and re-serialized. In headless mode, the origin private file system stood in for the native folder picker.

On a second file (2 million rows, 198 MB): numeric sort of all rows, descending, in 9.8 s and 10.7 s.

The same library in Node 24 gives the same counts; full dedup peaked there at about 1 GB of resident memory, consistent with the hash table sizes above.

These are one laptop's numbers. I haven't measured anything above 2.22 GB: the reading loop has no size-dependent state, but deduplication and sorting do.

Free version and license

Free: preview, encoding/delimiter detection and row counting with no limit. Exports (filter, dedup, sort, split) write at most 100,000 rows per operation. The worker still reads the whole file, so the result screen tells you how many rows the full result would contain.
Pro: a €29 one-time license sold through Gumroad, which removes the row cap in the browser where the key is entered. The key goes to the Pages Function, which calls Gumroad's license verification API and refuses refunded or disputed purchases. The browser stores it in localStorage and re-checks it every 7 days; offline, the stored key stays valid.

Since everything runs client-side, the cap is enforced in client-side code (limite: m.pro ? Infinity : LIMITE_GRATUITE). It's a pricing boundary, not DRM.

What the server sees

Anonymous daily counters (a visit, a preview, an export, an error), the ?ref= tag of the visit if there is one, and the license key if you type one. No cookies and no third-party analytics. The easy way to verify it: load the page, cut the network, and run an export.

Limits

Tested in Chrome only. Edge uses the same engine but I haven't tested it; Firefox and Safari aren't tested. Writing to a folder requires Chrome or Edge.
Sorting is in memory: 3 million rows / 600 million characters max.
Deduplication costs 16 to 32 bytes per distinct key and is probabilistic, as described above.
One worker, one core. Parsing isn't parallelized.
CSV in, CSV out. No .xlsx output.
A single record over 16 MB stops processing (usually an unclosed quote).

Try it

csv-geant.pages.dev: no account, and it can generate sample files up to 2 million rows. I'd like to hear about timings on other hardware, and about CSV dialects that break the delimiter detection.

Building an MCP server in Python on open data: French company lookup, tested end to end

Kamelyoul — Wed, 07 Oct 2026 08:22:29 +0000

I wanted a small, real MCP server to learn the protocol properly: not a "hello world" tool, but something an assistant could use for an actual task. French public data is a good playground. Two official APIs answer most "who is this company?" questions, need no key, and are published under the Licence Ouverte 2.0 (Etalab's open licence, which allows reuse, including commercial, as long as you credit the source):

API Recherche d'entreprises (DINUM, behind annuaire-entreprises.data.gouv.fr), built on INSEE Sirene and the national business register.
BODACC, the official gazette of commercial notices (insolvency judgments, deregistrations, accounts filings), published by DILA on OpenDataSoft.

The result is mcp-french-company-data: about 460 lines of Python across two modules, three read-only tools, stdio transport, MIT licence. This post covers the choices that mattered: which tools to expose, how the schemas are built, how errors reach the model, how it's tested, and what it can't do.

Which tools, and why only three

The tempting design is one tool per API endpoint. I went the other way and started from the questions a user actually asks:

"What's the SIREN of X?" → search_companies
"Tell me everything public about this company" → get_company
"Is it in trouble?" → get_bodacc_announcements

Three tools with clear, non-overlapping names are easier for a model to choose between than ten thin wrappers. The server also ships an instructions string that tells the model how they chain together:

mcp = MCPServer(
    name="french-company-data",
    version=__version__,
    instructions=(
        "Look up French companies in official open data. Use search_companies to find a "
        "SIREN from a name, get_company for the full profile, and get_bodacc_announcements "
        "for legal notices (insolvency, deregistration, accounts filing, changes). "
        "Data is public and may lag the registries by a few days; always cite the source URL."
    ),
)

The other big choice was what not to return. The search API can include company officers (names, birth month and year). The server never asks for them:

params: dict[str, Any] = {
    "q": q,
    "per_page": per_page,
    "page": 1,
    # Leave out `dirigeants` (names and birth months of individuals).
    "minimal": "true",
    "include": "siege,complements,finances,tva",
}

Responses are also reshaped. The raw API uses French field names and INSEE codes (5599, "21" for a headcount bracket). The server maps the most common ones to readable labels and keeps the code next to them, e.g. "SA with board of directors (5599)". Unknown codes are returned as-is rather than guessed.

Every tool is annotated read-only, so clients can treat it accordingly:

READ_ONLY = ToolAnnotations(read_only_hint=True, destructive_hint=False,
                            idempotent_hint=True, open_world_hint=True)

Schemas: let the type hints do the work

This uses the v2 Python SDK (mcp 2.3.0 when I ran the tests). Most tutorials you'll find use the v1 FastMCP class; the decorator pattern is the same. Each parameter is an Annotated type with a Pydantic Field, and the SDK turns that into the JSON Schema the client sees:

@mcp.tool(annotations=READ_ONLY)
async def get_bodacc_announcements(
    siren: Annotated[str, Field(description="SIREN (9 digits) or SIRET (14 digits)")],
    limit: Annotated[int, Field(ge=1, le=20, description="Max announcements, newest first (1-20)")] = 10,
    category: Annotated[BodaccCategory | None, Field(
        description="Optional filter: collective (insolvency), radiation, dpc (accounts), "
                    "modification, creation, immatriculation, vente, conciliation, "
                    "retablissement_professionnel, divers")] = None,
) -> dict[str, Any]:

BodaccCategory is a Literal[...] of the ten BODACC families, so the generated schema contains a real enum, and the ge/le bounds become minimum: 1 / maximum: 20. The model sees the allowed values instead of having to guess French category codes. On search_companies, query has min_length=2 and the optional postal code has pattern=r"^\d{5}$".

The docstring becomes the tool description, so I wrote those for the model, not for humans reading the code: what it returns, and what it deliberately doesn't ("Personal data of company officers is intentionally not returned").

Because the functions return a dict, the SDK also publishes an output schema and returns structured content, which is what the tests assert on.

Error handling: three layers

A tool that raises a random exception gives the model nothing to work with. Here every failure ends up as a tool result with isError: true and a sentence the model can act on.

1. Schema validation, done by the SDK before my code runs. Calling search_companies with query: "d" returns (real output):

Error executing tool search_companies: 1 validation error for search_companiesArguments
query
  String should have at least 2 characters [type=string_too_short, ...]

2. Domain validation, before any network call. A SIREN has a Luhn check digit, so a typo can be caught locally:

def _siren_or_fail(raw: str) -> str:
    siren = normalize_siren(raw)
    if not siren:
        raise ToolError(f"'{raw}' is not a valid SIREN (9 digits) or SIRET (14 digits).")
    return siren

normalize_siren strips spaces, truncates a 14-digit SIRET to its SIREN, and has one hard-coded exception: La Poste (356000000), whose SIREN doesn't pass the Luhn key.

3. Upstream failures. The HTTP layer only raises one exception type, SourceError, with a message meant to be shown as-is. Tools convert it to ToolError:

try:
    raw = await data.bodacc(s, limit=limit, famille=category)
except SourceError as exc:
    raise ToolError(str(exc)) from exc

A "not found" is also an explicit error, with the most likely reason. Some companies opt out of public listing ("non-diffusible"), and the API simply doesn't return them:

raise ToolError(f"No public record for SIREN {s}. It may not exist, or be "
                "'non-diffusible' (opted out of public listing).")

Being a polite API client

Recherche d'entreprises documents a limit of 7 requests per second per IP. BODACC on OpenDataSoft exposes an anonymous daily quota in its X-RateLimit-* headers (20,000 calls per day when I looked). An assistant looping over a list of suppliers can hit those fast, so the client:

spaces requests (at most 4/s for the search API, 2/s for BODACC) with a small async limiter;
caches identical requests in memory for 5 minutes;
sends a descriptive User-Agent with the repo URL;
on HTTP 429, waits once if Retry-After is 10 seconds or less, otherwise gives up with a clear message instead of blocking the conversation:

if resp.status_code == 429 and attempt == 0:
    delay = _retry_after(resp.headers.get("Retry-After"))
    if delay is not None and delay <= MAX_RETRY_AFTER:
        await self._sleep(delay)
        continue
    raise SourceError(f"{source}: rate limit reached, retry later "
                      f"(Retry-After: {resp.headers.get('Retry-After', 'n/a')})")

The limiter takes clock and sleep as parameters, which makes it testable without waiting.

Tests: three levels, no network by default

The suite has 27 offline tests plus one live test, opt-in. I re-ran them while writing this (Windows, Python 3.13.15, mcp 2.3.0, pytest 9.1.1):

$ pytest
collected 28 items
tests\test_live.py s
tests\test_server.py ............
tests\test_sources.py ...............
======================== 27 passed, 1 skipped in 4.58s ========================

$ LIVE=1 pytest -m live
====================== 1 passed, 27 deselected in 2.80s =======================

HTTP is simulated with httpx.MockTransport, serving JSON fixtures recorded from the real APIs and logging every request. That lets tests assert on what is sent, not only on what comes back:

async def test_get_company_invalid_siren_is_tool_error(server, fake):
    async with Client(server) as c:
        res = await c.call_tool("get_company", {"siren": "123"})
    assert res.is_error
    assert "not a valid SIREN" in res.content[0].text
    assert fake.requests == []  # rejected before any HTTP call

The MCP layer is tested through a real client. Client(server) connects in-process, so list_tools and call_tool go through the SDK exactly as a host would: schema generation, validation, structured output, error wrapping.

The stdio launch is tested twice: once by spawning python -m french_company_data with the SDK's stdio client, and once with a hand-written JSON-RPC exchange (initialize, notifications/initialized, tools/list) over the process pipes. That second test catches a classic stdio bug: anything printed to stdout that isn't protocol. It's also why main() turns down httpx logging before starting.

One fixture needed care: the insolvency example comes from a real BODACC notice about a small company. The recording script keeps the structure and wording but replaces the identity with a fictitious one (SAMPLE FLOORING SARL, SIREN 732829320) and redacts the name of the court-appointed person. The search fixtures also drop sole proprietorships, which are named after private individuals.

Plugging it into Claude Desktop or another client

It's a stdio server, so any MCP host that can launch a command works. For Claude Desktop, add this to claude_desktop_config.json (Settings > Developer > Edit Config) and restart:

{
  "mcpServers": {
    "french-company-data": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/Kamelyoul/mcp-french-company-data", "mcp-french-company-data"]
    }
  }
}

The same block works in ~/.cursor/mcp.json for Cursor; the README has the Claude Code one-liner.

To be precise about what's verified: the tests launch the server over stdio the way these hosts do, and examples/demo_client.py runs all three tools against the live APIs. No GUI client is part of the test suite.

A prompt like "Check the BODACC for SIREN 552032534: any insolvency proceedings?" gives the model get_bodacc_announcements, which returns the notices newest first, an insolvency_in_results flag, and each notice's bodacc.fr URL.

Limits (the real ones)

Not the full Sirene database. Companies that opted out of public listing are not returned by the API.
Freshness. Data can lag the official registries by a few days. BODACC notices for sole proprietors are sometimes not linked to a SIREN and can be missed.
Per-process state. The rate limiter and the cache live in memory, in one process. Two clients running two copies of the server each get their own budget.
Stdio only. No HTTP transport, no authentication: it runs locally, next to the assistant.
Partial code tables. Only 10 legal forms get a readable label; the rest come back as raw INSEE codes.
Endpoint risk. The BODACC OpenDataSoft URL is the one documented on data.gouv.fr today. If DILA moves it, it's one constant to change.
Not legal or financial advice. For a credit decision or a claim deadline, open the official notice; every result links to it.

Attribution

Data: API Recherche d'entreprises (DINUM, INSEE Sirene, RNE) and BODACC (DILA), both under Licence Ouverte 2.0. Every tool response carries that attribution in a source field. The project isn't affiliated with these administrations.

Code: github.com/Kamelyoul/mcp-french-company-data. If you need a custom MCP server for your own API or data, I take on small projects on Fiverr.

How I built in-browser animated subtitles with Whisper on WebGPU (no upload)

Kamelyoul — Wed, 07 Oct 2026 07:37:41 +0000

Most "auto captions" tools start the same way: upload your video, wait, download the result. I wanted the opposite for short vertical videos (TikTok, Reels, Shorts): the file never leaves the browser. Transcription, editing, rendering and export all run on the user's machine.

The result is a small static site: sous-titres.pages.dev (the UI is in French, but it transcribes 8 languages). This post walks through the architecture, the real code, and the limits I hit.

The pipeline

Decode the audio track in the page (AudioContext at 16 kHz, mixed down to mono).
Send the samples to a Web Worker running Whisper through transformers.js, on WebGPU if available, WASM otherwise.
Turn word-level timestamps into short caption segments (pure functions, unit-tested in Node).
Draw every frame on a <canvas>: source video + captions where the current word lights up.
Record the canvas plus the original audio with MediaRecorder and hand the user a file.

No backend touches the media. The only server code is a tiny Cloudflare Pages Function for anonymous counters and license checks.

1. Whisper in a worker, WebGPU first

The worker picks a device and a precision. WebGPU gets fp16 for the encoder when the adapter supports shader-f16, and a 4-bit decoder; the WASM fallback uses 8-bit weights for both:

async function detecterWebGPU() {
  try {
    if (!self.navigator?.gpu) return { ok: false };
    const adaptateur = await navigator.gpu.requestAdapter();
    if (!adaptateur) return { ok: false };
    return { ok: true, fp16: adaptateur.features.has("shader-f16") };
  } catch {
    return { ok: false };
  }
}

const device = gpu.ok ? "webgpu" : "wasm";
const dtype = gpu.ok
  ? { encoder_model: gpu.fp16 ? "fp16" : "fp32", decoder_model_merged: "q4" }
  : { encoder_model: "q8", decoder_model_merged: "q8" };

Two models are offered: onnx-community/whisper-base_timestamped (~90 MB) and whisper-small_timestamped (~250 MB). The _timestamped exports matter: they expose the cross-attention needed for return_timestamps: "word".

navigator.gpu existing is not enough: an adapter can be returned and the pipeline can still fail to build on it. So creation is wrapped and retried in WASM:

try {
  ({ transcripteur: t, device } = await obtenirTranscripteur(modele, false));
} catch (e) {
  self.postMessage({ type: "info", message: `WebGPU indisponible (${e.message || e}), passage en mode compatible.` });
  ({ transcripteur: t, device } = await obtenirTranscripteur(modele, true));
}

There is a second fallback: if word-level timestamps throw, the worker asks for sentence-level timestamps and the UI spreads the words across each sentence, weighted by length.

2. From Whisper chunks to captions

Whisper's word output is messier than it looks. Tokens sometimes arrive split (" micro" + "-entrepreneur."), and noise markers like [Musique] show up. The rule I settled on: a chunk that does not start with a space belongs to the previous word.

const colle = precedent && !/^\s/.test(brut) && !/^[\[(]/.test(texte);
if (colle) {
  precedent.texte += texte;
  precedent.fin = Math.max(precedent.fin, fin);
} else {
  mots.push({ texte, debut, fin });
}

Segmentation is a greedy pass with a few cut conditions: max words per line (user setting, 1 to 5), max 22 characters, a pause longer than 0.6 s, end of sentence, or a comma once the line has two words. Each segment stays on screen 0.3 s after its last word, without ever overlapping the next one. All of this lives in a DOM-free module, so it runs under node --test and also generates the .srt and the .ass karaoke (\k tags) exports.

When the user fixes a typo, the segment keeps its start and end times and the new words are spread inside them. Small, but it means editing never breaks the sync.

3. Rendering: one function, three styles

The same dessinerSousTitres(ctx, segments, t, settings) draws the live preview and the export. It measures each word, wraps at 86% of the 1080 px width, then draws word by word with a thick black stroke, a soft shadow, and a short "pop" on the active word:

const age = t - seg.mots[i].debut;
const pop = estActif ? 1 + 0.1 * Math.max(0, 1 - age / 0.18) : 1;
ctx.save();
ctx.translate(x + w / 2, y);
ctx.scale(pop, pop);

Three styles are just branches in that loop: highlight the active word's color, draw a rounded box behind it, or reveal words only once they've been spoken. The font (Montserrat 900) is self-hosted and preloaded, and the app awaits document.fonts.load() before the first draw. Otherwise the first frames get measured with a fallback font.

Because this module only needs a 2D context, I reused it outside the browser: the promo clip for the tool was rendered in Node with @napi-rs/canvas, the same rendu.js, and transformers.js on CPU.

4. Export: canvas + MediaRecorder

The export plays the video in a hidden <video>, redraws each frame on requestAnimationFrame, and records canvas.captureStream(30) plus the original audio routed through createMediaElementSource:

const candidats = [
  "video/mp4;codecs=avc1.640028,mp4a.40.2",
  "video/mp4;codecs=avc1,mp4a.40.2",
  "video/mp4",
  "video/webm;codecs=vp9,opus",
  "video/webm;codecs=vp8,opus",
  "video/webm",
];
return candidats.find((m) => MediaRecorder.isTypeSupported(m)) || null;

Recent Chrome, Edge and Safari produce MP4; Firefox produces WebM. The honest trade-off: export runs in real time (60 s of video takes about 60 s) and the tab must stay visible, because background tabs throttle rAF. WebCodecs plus a JS muxer would allow faster-than-real-time export and more consistent output. That's the next step, and it isn't shipped yet.

5. The deployment surprise: Hugging Face and `*.workers.dev`

I first served the app from a *.workers.dev subdomain, and the model never loaded there: model file requests answered 404 whenever the Referer was a workers.dev origin. Same files, same code, different referrer.

Rather than fight it, I moved the static app to Cloudflare Pages (*.pages.dev), where the model loads normally, and left a one-line Worker on the old address that answers 301 to the new one:

const CIBLE = "https://sous-titres.pages.dev";
export default {
  fetch(requete) {
    const url = new URL(requete.url);
    return Response.redirect(CIBLE + url.pathname + url.search, 301);
  },
};

Lesson: if your app fetches third-party assets client-side, test from the real production origin early. The Referer your users send is part of your dependency surface.

Privacy, verifiable

"No upload" is easy to claim, so the FAQ tells users how to check it: open DevTools → Network, run a transcription, and you'll see only the model (Hugging Face), the library (jsDelivr) and anonymous event counters (a visit, an export). No cookies, no third-party analytics.

Limits (the real ones)

Desktop Chrome/Edge recommended (that's where I tested it). Firefox and Safari aren't tested yet; without WebGPU the app falls back to WASM, which is slower. Phones can run out of memory.
First run downloads the model (~90 MB or ~250 MB), then it's cached.
Speed depends on hardware. With WebGPU, a one-minute clip usually transcribes in under a minute. Without it, expect several minutes.
Whisper makes mistakes on names, numbers and acronyms. The editor exists for that.
Short-form only. Beyond 3–5 minutes, browser memory becomes the bottleneck.
Real-time export, as described above.

Try it

sous-titres.pages.dev: free, no account, with a small watermark on exported videos. A one-time paid license removes it. Feedback on the WebGPU path, especially from AMD and Intel GPUs, is very welcome in the comments.

Look Up: an open-weight model on your phone reads the clouds, then tells you to put the phone away

Kamelyoul — Tue, 06 Oct 2026 23:13:36 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Look Up is a tiny web app for cloud spotting.

You point your phone at the sky and take one photo. An open-weight vision model, running on the phone itself, tells you which of the ten cloud families you're looking at, what that cloud usually means for the next few hours, and gives you one small thing to watch for. Then it asks you to put the phone away.

That last part is the whole point. The best thing about clouds is that they move: a cumulus grows or evaporates in five minutes, a contrail vanishes or spreads, a halo appears around the sun. You can only see that if you stop looking at a screen. So every reading ends with a mission:

Cumulus: Pick one cloud and watch it for five minutes. Is it growing upward, or fraying at the edges and evaporating?

Cirrus: Pick one wisp and watch which way its tail is combed. That is the wind direction kilometres above you.

You tap Put the phone away. The screen goes black and counts the time until you come back (you can lock the phone). When you return, you tap what the cloud did ("It grew", "It faded", "It moved on"), and the app keeps a small sky log with one number I care about: how many seconds you spent looking up for every second you spent looking at the screen.

It's for anyone who has ever looked up and wondered "what is that?", but especially for people who reach for the phone out of habit. This time, the phone sends you back to the sky.

And when it sees a cumulonimbus, there is no timer. The mission is "go indoors".

Demo

👉 https://look-up-sky-smoky.vercel.app

Open it on your phone, tap Get ready for no signal once on Wi-Fi (it downloads the model, about 90 MB), then go outside and tap Read the sky. No account and no install (you can add it to your home screen if you like).

Code

Kamelyoul / look-up

Look Up — point your phone at the sky, learn the cloud, then put the phone away. On-device CLIP, works offline.

Look Up

Point your phone at the sky. An open-weight model running on your phone names the clouds, tells you what they usually mean, and gives you a small thing to watch for. Then you put the phone away.

Built for the DEV Hacktoberfest Open-Source AI Challenge, Week 1: Touch Grass (October 5 to 11, 2026). The project and this repository were created during the challenge window.

What it does

Read the sky: take a photo of the sky (or pick one you already took).
CLIP ViT-B/16 (open weights, MIT) runs in the browser through Transformers.js and ONNX Runtime Web. It classifies the photo zero-shot into the 10 WMO cloud genera, plus contrail, clear sky and not the sky.
You get a card with what to check with your own eyes, what that cloud usually means for the next hours (hedged weather lore), and a mission…

View on GitHub

Vanilla TypeScript and Vite, with no framework and no backend. MIT licensed. npm test runs the 34 unit tests; npm run eval reproduces the accuracy numbers below.

How I Built It

The model: CLIP, zero-shot, in the browser

The core is OpenAI's CLIP ViT-B/16, an open-weight (MIT) image–text model. I run it with Transformers.js on top of ONNX Runtime Web (WebAssembly), using Xenova's ONNX export at 8-bit.

CLIP was never trained to classify clouds. It doesn't need to be: it maps images and sentences into the same space, so I describe each class in plain English and pick the closest description. Each of the 13 classes (the ten WMO cloud genera, plus contrail, clear sky and not the sky) gets 3 to 6 prompts like:

prompts: [
  'a photo of cumulus clouds',
  'fluffy white cotton-ball cumulus clouds with flat bases in a blue sky',
  'puffy cauliflower-shaped white clouds on a sunny day',
],

The descriptive prompts matter. CLIP has seen "fluffy white clouds" millions of times and "stratocumulus" far less.

The trick: the text half never ships

CLIP has two towers: one for text and one for images. The prompts are fixed, so I run the text tower once, at build time, in Node, and ship only the 43 resulting vectors: a 120 KB JSON file. The phone downloads only the vision tower. That saves a download of tens of MB and makes classification a dot product:

// score of a class = its best-matching prompt (max, not mean)
for (let r = 0; r < bank.classes.length; r++) {
  let dot = 0;
  for (let i = 0; i < dim; i++) dot += img[i] * bank.matrix[r * dim + i];
  best.set(id, Math.max(best.get(id) ?? -1, dot));
}
// then softmax(100 * score) over the 13 classes

A few design decisions:

Max, not mean. The "not the sky" class mixes very different things (grass, a room, a screen, a street). Averaging over them gives a blurry "average non-sky" that matches nothing well. So a class scores as its best-matching prompt. On my test set, that rejected 22 of 24 non-sky photos, against 18 with averaging. Point it at your feet and it tells you to point it up.
It's allowed to be unsure. If the top class is under 45%, the card says "Altocumulus, or maybe Cirrocumulus — I'm torn". Real skies often contain several genera at once, and I'd rather the app say so than bluff.
Storms get special treatment. If "cumulonimbus" is the top guess, the runner-up, or simply above 10%, the card adds a red line: "This could also be a thunderstorm cloud. If it is tall and dark, or you hear thunder, go indoors instead."

Did it actually work? I measured it.

Zero-shot demos are easy to cherry-pick, so I built a small, reproducible test set from Wikimedia Commons: the first 12 JPEG photos in each cloud category whose file name names only that genus (132 sky photos), plus 24 photos of lawns and living rooms that the app should refuse. The evaluation script uses the exact code path the browser uses: same weights, same prompts, same scoring.

Metric (CLIP ViT-B/16, 8-bit)	Result
Top-1 accuracy, 13 classes (chance ≈ 7.7%)	43% (57/132)
Correct class in the top 3	68% (90/132)
Accuracy when the app says it's confident	69% (25/36)
Contrails	11/12
Thunderstorm photos that get the storm warning	9/12 (10 false alarms on 120 other skies)
Sky photos wrongly rejected as "not the sky"	0/132
Lawns and living rooms correctly rejected	22/24

It's good at the distinctive skies (contrails, cumulus, the altocumulus "sheep"). It's honestly bad at the subtle ones: cirrostratus (2/12) and stratocumulus (1/12) mostly come back as something else. That's why the app shows its top guesses and phrases everything as "check with your own eyes". It's a field guide that nudges you to look, not an oracle. (Caveat: Commons labels are crowd-sourced, and many photos contain more than one genus.)

Choosing the model with the numbers. I also ran the smaller-patch sibling, CLIP ViT-B/32. It's twice as fast and more precise when confident (84% vs 69%). But it caught only 5 of 12 thunderstorm photos where B/16 caught 9, and it raised more false alarms (16 vs 10). For an app that tells people when to go indoors, that settled it. I shipped B/16 and accepted the extra ~200 ms. The full per-image tables are in the repo (eval/).

The rest

Offline: a small service worker caches the app shell. Transformers.js stores the weights and the WASM runtime in the browser's Cache Storage, so after the first load it works in airplane mode.
"Phone away" screen: it's black, has a big counter and nothing else. Time is measured from timestamps, so it keeps counting if you lock the phone.
Sky log: localStorage only. Nothing leaves the device.
Tests: 34 unit tests (Vitest) cover the scoring and the base64 embedding format. A test fails if someone edits the prompts without regenerating the embeddings. Another checks that the thunderstorm card can never offer a "watch it" timer. A GitHub Actions workflow runs the tests and the build on every push.

Why Does Open Innovation Matter?

For this app, open weights aren't a nice-to-have. Three things only work because of them.

1. The sky is where the signal isn't. Hills, beaches, fields and trails are exactly where you have one bar or none. A cloud app that calls a hosted vision API fails in the very place it's meant to be used. With open weights, the model is a file. The browser caches it and the app works in airplane mode.

2. Your photos stay yours. Sky photos contain more than sky: rooftops, your street, sometimes your kids at the edge of the frame. Here, nothing is uploaded, because there's no server to upload to. I didn't have to write a privacy policy for a server I'd have to trust; there just isn't one.

3. It costs nothing to run, so it can stay free. There is no API key, no per-request bill and no rate limit. The whole thing is a static site. If a thousand people use it on a sunny Saturday, my cost doesn't change.

There's a fourth, more personal reason: I could open it up and change it. Because the model is open, I could split it in two, run half of it at build time, and ship only the half the phone needs. You can't do that with a closed endpoint. The classes are just sentences in a TypeScript file too. Want to add mammatus or noctilucent clouds? Add a line, run npm run embeddings, and redeploy. No retraining, no permission needed.

Where closed models would win: a large hosted multimodal model would likely be more accurate on the hard pairs (altocumulus vs. cirrocumulus). But it would need a signal, see your photos, and cost money per look. For an app whose job is to get you off the screen and out where there's no Wi-Fi, a small model you can carry beats a big one you can't.

My Agent Session

Full disclosure: I built this with an AI coding agent (Claude Code). It wrote most of the code, the tests and the evaluation scripts, and helped draft this post. Every number above comes from scripts in the repo that you can rerun (npm test, npm run eval), not from the agent's say-so. I don't have a DevRelay recording of the session.

DEV Community: Kamelyoul

Filtering a 2 GB CSV in the browser without uploading it: streams, a Web Worker and a 64-bit hash set

The architecture

1. Reading: slice() plus a streaming TextDecoder

2. A parser that can stop anywhere

3. Deduplication: 64-bit fingerprints in a Uint32Array

4. Writing: BOM, CRLF, and two sinks

5. The part that doesn't stream: sorting

Measured numbers

Free version and license

What the server sees

Limits

Try it

Building an MCP server in Python on open data: French company lookup, tested end to end

Which tools, and why only three

Schemas: let the type hints do the work

Error handling: three layers

Being a polite API client

Tests: three levels, no network by default

Plugging it into Claude Desktop or another client

Limits (the real ones)

Attribution

How I built in-browser animated subtitles with Whisper on WebGPU (no upload)

The pipeline

1. Whisper in a worker, WebGPU first

2. From Whisper chunks to captions

3. Rendering: one function, three styles

4. Export: canvas + MediaRecorder

5. The deployment surprise: Hugging Face and *.workers.dev

Privacy, verifiable

Limits (the real ones)

Try it

Look Up: an open-weight model on your phone reads the clouds, then tells you to put the phone away

What I Built

Demo

Code

Kamelyoul / look-up

Look Up — point your phone at the sky, learn the cloud, then put the phone away. On-device CLIP, works offline.

Look Up

What it does

How I Built It

The model: CLIP, zero-shot, in the browser

The trick: the text half never ships

Did it actually work? I measured it.

The rest

Why Does Open Innovation Matter?

My Agent Session

1. Reading: `slice()` plus a streaming `TextDecoder`

3. Deduplication: 64-bit fingerprints in a `Uint32Array`

5. The deployment surprise: Hugging Face and `*.workers.dev`