The three directory sites I launched in April all run on auto-generated entries. Top AI Tools has about 1,500 HuggingFace model entries. Find Games Like has 120 Steam game entries. Open Alternative To has 80 SaaS product entries. The initial ETL pipeline filled all of them from API data using a template system I called "fallback content" — pre-written sentence structures populated with extracted metadata.
Fallback content isn't thin in a word-count sense. A typical model entry has 200–300 words covering what the model does, its architecture, use cases, and limitations. But it is thin in a differentiation sense: across ~1,500 models, the same sentence structures start repeating. "This model handles instruction prompts, multi-turn dialogue, and open-ended text generation" appears verbatim across dozens of entries. "OSS alternatives provide" opens half the comparison notes in the OSS directory.
That kind of clustering is what AdSense reviewers flag as low-quality content. It's also bad for readers. The three-tier content quality ladder I designed earlier distinguishes fallback entries (tier 1), programmatic-but-varied entries (tier 2), and hand-curated entries (tier 3). Getting from tier 1 to tier 2 at scale requires breaking the clustering — without making hundreds of individual edits by hand, and without paying LLM costs on the ~1,700 entries across the three datasets.
The answer is scripts/polish.py: a deterministic upgrade pass that runs in CI with no external API calls.
What the script actually does
polish.py has three layers.
A content extraction layer reads structured metadata already present in each entry's JSON: for models, that means pipeline_tag, tags, model_id, and library name; for games, it's Steam genre data and mechanic tags; for OSS tools, it's the product category and tier. No network requests — the ETL pipeline populated all of this.
A template selection layer picks which prose variant to use for each entry. There are between 3 and 8 pre-written variants per content slot — summary, best_for, avoid_if, and so on. Selection is seeded by the entry's slug so the same slug always picks the same templates.
A coordination layer called process_app() reads a JSON file, identifies entries whose model_used field is "", None, or "fallback-template", processes up to CAP of them, and writes the file back. The CAP defaults to 500 per app, so a full run across all three apps can touch up to 1,500 entries. It can be overridden with the POLISH_CAP environment variable — I ran a targeted pass of 37 entries this week while testing a new template pool.
The result is entries upgraded from "basic template filled with category nouns" to "metadata-informed prose with architecture-aware sentences" — without touching the database, making network requests, or paying API costs.
The determinism design choice
The most important decision I made: seeded hash instead of random selection.
The first version picked template variants randomly. The problem showed up almost immediately in CI: two consecutive runs on the same data produced different outputs. The git diff showed 400 changed entries with semantically identical but textually different content. Commits that should show no change showed noise across the whole file.
The fix is the pick() helper:
def _seed(text: str) -> int:
return int(hashlib.md5(text.encode()).hexdigest(), 16)
def pick(lst: list, seed_str: str):
return lst[_seed(seed_str) % len(lst)]
Every content slot uses the entry's slug as the seed string. The integer produced by MD5 mod the template list length is stable across environments, Python versions, and run order. Run the script on the same input twice: byte-identical output. Run it on a partially-upgraded file: skips already-upgraded entries without touching them.
This matters for ETL upsert patterns: when the daily content refresh updates an entry's metadata — new star count, updated description — the polished prose rides along through the upsert, and the next polish run skips that entry because its model_used is no longer a fallback value. If an entry ever does fall back and gets re-polished, the slug hasn't changed, so it lands on the same template choice as before.
Per-app upgrade differences
Each app has a different schema and content needs, so there are three separate upgrade functions.
polish_model (ai-tools): HuggingFace model tags are rich and structured — license, architecture family, quantization formats, supported languages, framework compatibility. The upgrade extracts these into variables and builds metadata-informed prose:
arch = _arch(entry["model_id"], entry.get("tags", []))
frameworks = _frameworks(entry.get("tags", []))
langs = _langs(entry.get("tags", []))
license_str = _license(entry.get("tags", []))
Then pick() selects one of 3 summary templates for that pipeline_tag, fills in the extracted variables, and populates best_for, limitations, pros, and cons arrays.
| Field | Before | After |
|---|---|---|
summary |
Generic pipeline description | Architecture + language + format details |
best_for |
One vague sentence or missing | 3–4 seeded specific use cases from pool |
limitations |
Missing | 1–2 extracted from task type |
model_used |
"" or null
|
"claude-routine-polish" |
polish_game (indie-games): Games have different signals — Steam genre tags, similar game references, review sentiment patterns. The upgrade builds avoid_if and good_for sentences from the genre taxonomy. A platformer with high-difficulty review mentions gets different avoid_if text than a platformer without them.
polish_oss (oss-alternatives): OSS tool entries get comparison_notes (how the tool compares to the paid SaaS original) and migration_tips (specific technical steps to switch). An error-monitoring alternative gets different migration notes than a data visualization alternative, drawn from a category-specific template pool.
The model_used field acts as the upgrade tracker across all three apps. model_used == "claude-routine-polish" means this pass ran. model_used == null means the entry is still fallback-template and queued for the next run.
The companion humanize pass
polish.py breaks the obvious boilerplate: entries that all used the same template text now have varied, metadata-informed prose. But the template pool only has 3–8 variants per field. Across ~1,500 model entries, even with seeded distribution, the most common patterns repeat 150–400 times.
That's where scripts/humanize-aiappdex.mjs comes in. While polish.py upgrades at the entry level (metadata → prose), the humanizer works at the corpus level: it detects which sentence patterns appear too many times across the full dataset and regenerates them using a finer-grained seeding scheme based on FNV-1a hash plus a per-facet salt.
The companion scripts/lint-humanization.mjs runs as part of the content quality gate and flags stock phrases above a threshold. Specific phrases that proved problematic — "handles instruction prompts, multi-turn dialogue, and open-ended text generation" — are flagged at max: 0, meaning they must never appear verbatim in the final dataset. Other phrases like "vendor lock-in" are allowed up to 24 occurrences, because they're genuinely descriptive at reasonable scale.
This two-stage approach — entry-level upgrade via polish.py, then corpus-level deduplication via humanize-aiappdex.mjs — handles the scale problem without LLM calls at publish time. The EEAT transparency work is the longer-term complement: the polish pass makes every entry readable, the transparency pages explain the generation process at the site level.
What I'd do differently
More template variants from day one. Starting with 3 variants per field meant that even with seeded selection, clusters formed quickly once the dataset grew past a few hundred entries. Ten variants per field would reduce visible repetition by roughly 3×. The templates are just Python strings in a dict — the only reason I started with so few is that fewer templates means less writing up front.
Separate template data from upgrade code. polish.py is a single 1,000-line file with both the selection logic and all the template strings. The templates should live in separate JSON files: easier to extend, easier to version independently, easier to audit without reading Python. Right now reviewing what prose is possible means reading interleaved code.
Track upgrade coverage per field, not per entry. The model_used flag marks whether an entry has been through any upgrade pass. It doesn't record which fields were upgraded. If I add a new field to the schema six months in, I can't easily query "show me all entries where best_for is still at fallback quality." A _field_versions dict per entry would make this queryable without a full re-scan.
I don't know yet how the upgraded entries will affect Search Console impression counts. The noindex gate is still blocking the lowest-quality pages from indexing. I'll publish the first comparison once the upgrade pass has been running for 30 days.
FAQ
Does polish.py use Claude or any LLM?
No. Every upgrade comes from pre-written template strings combined with metadata extracted from existing JSON fields. No API calls, no external network requests, no token cost. The point is a pass that runs in CI every week at zero marginal cost — if it required an LLM call per entry, the economics break down across ~1,700 entries and a growing model directory.
How do you prevent the same sentence from appearing too many times?
Two mechanisms: seeded template selection (same slug → same template, distributed by hash modulo) and lint-humanization.mjs, which flags phrases appearing above a corpus-level threshold. If a phrase hits the threshold, the template pool needs more variants.
When does an entry graduate from this pass to hand-curated?
polish.py handles tier 1 → tier 2. Moving to tier 3 (hand-curated) requires a human deciding an entry is worth individual attention — usually a high-traffic model, a well-known game that appears often in "similar to X" queries, or an OSS tool where I have first-hand experience. The model_used field tracks tier: null or "fallback-template" is tier 1, and "claude-routine-polish" is tier 2 — as is "metadata-derived", the tag left by the humanize pass on 251 model rows it rebuilt from metadata. Nothing in the three datasets is marked tier 3 yet.
Why not run a single LLM pass for everything?
Cost, reproducibility, and auditability. LLM output for ~1,700 entries at a meaningful model tier costs real money per run and is non-deterministic — two runs of the same prompt produce different text. For a site that needs to satisfy AdSense quality requirements, being able to audit the exact text produced matters: I can read the template pool and know precisely what prose variants are possible.
How long does a full run take?
The current default is 500 per app. Each polish_model(), polish_game(), or polish_oss() call is a few microseconds — pure Python string operations. Processing 1,500 entries across all three apps takes about 2 seconds. The CI step is dominated by file I/O, not compute.
Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.
Top comments (0)