DEV Community

Cover image for Upgrading fallback AI model entries to curated quality with a deterministic hash pool
MORINAGA
MORINAGA

Posted on Edited on

Upgrading fallback AI model entries to curated quality with a deterministic hash pool

#opensource #webdev #programming #indiehackers

By late May the AI tools directory had 380 model entries sitting on model_used = 'fallback-template'. That tier generates a bare-bones summary — "qwen2-7b is an open-source text-generation model available on HuggingFace" — which is accurate but gives users nothing to act on.

Calling Claude Haiku for all 380 at once would cost a few dollars and flood the API — and the daily job can't do it anyway. As covered in the three-tier content quality ladder, refresh-content.yml deliberately runs without ANTHROPIC_API_KEY, so every scheduled run takes the fallback-template path and the editorial upgrade happens in a separate weekly pass.

But there's another option for entries where AI-generated prose isn't strictly necessary: deterministic template enrichment. The scripts/polish.py script upgrades fallback entries using structured metadata already present in the model's HuggingFace tags — no API call, no AI involved.

What the Enrichment Adds

The HuggingFace model registry stores several fields the basic fallback ignores:

  • tags: a freeform list that often includes license:apache-2.0, pytorch, en, safetensors, gguf — the full schema is in the HuggingFace Hub docs
  • modelId: the full path like Qwen/Qwen2-7B, which encodes architecture

polish.py extracts structured facts from these fields at upgrade time:

def _license(tags: list) -> str | None:
    MAP = {"apache-2.0": "Apache 2.0", "mit": "MIT", "gpl-3.0": "GPL-3.0", ...}
    for t in tags:
        if t.startswith("license:"):
            return MAP.get(t[8:], t[8:])
    return None

def _frameworks(tags: list) -> list:
    MAP = {"pytorch": "PyTorch", "onnx": "ONNX", "gguf": "GGUF", "safetensors": "safetensors", ...}
    return [MAP[t] for t in tags if t in MAP]

def _langs(tags: list) -> list:
    MAP = {"en": "English", "zh": "Chinese", "ja": "Japanese", "multilingual": "multilingual", ...}
    return [MAP[t] for t in tags if t in MAP]
Enter fullscreen mode Exit fullscreen mode

Architecture gets inferred from substring matching against the model ID. meta-llama/Meta-Llama-3-8B gets labeled "Llama"; sentence-transformers/all-MiniLM-L6-v2 gets "MiniLM"; openai/whisper-large-v3 gets "Whisper". The match list covers about 30 common architecture families — it's family-level, not version-level — and falls back to a generic "transformer" label for anything unrecognized.

The output for a polished entry looks like: "Qwen2-7B-Instruct handles instruction prompts, multi-turn dialogue, and open-ended text generation. It follows chat template conventions and supports system-level role instructions." The license and framework facts don't go in the summary — they land in the pros list, as lines like "Apache 2.0 license permits unrestricted commercial use" and "Optimized safetensors weights available for direct inference". Between the two, that's a page worth rendering — specific enough that a user can evaluate it, distinct enough that two different models don't return identical copy.

Why Deterministic Instead of Random

Pool selection uses MD5 hash of the model name rather than a random seed:

def _seed(text: str) -> int:
    return int(hashlib.md5(text.encode()).hexdigest(), 16)

def pick(lst: list, seed_str: str):
    return lst[_seed(seed_str) % len(lst)]
Enter fullscreen mode Exit fullscreen mode

The same model always gets the same template selection. Rerunning the script produces identical output. This matters for audits: if a summary changes between two runs without any template edits, something is wrong.

Random selection would produce different outputs on every run, making it impossible to distinguish "the content changed because we edited the template pool" from "the content changed because the seed landed differently." Deterministic selection removes that ambiguity entirely.

The Per-Run Cap 

CAP = int(os.environ.get("POLISH_CAP", "500"))

UNPOLISHED = ("", None, "fallback-template")
fallback_idxs = [i for i, e in enumerate(data) if e.get("model_used") in UNPOLISHED]
to_process = fallback_idxs[:CAP]
Enter fullscreen mode Exit fullscreen mode

Only entries whose model_used is empty, null, or fallback-template are eligible — anything already polished or Claude-generated is skipped. The slice caps how many get rewritten in one pass, so a single run never churns thousands of records into one commit. The most recent run upgraded 98 entries; 282 remain. This is a weekly pass rather than part of the nightly refresh workflow, so the backlog clears over the following runs, at which point pipeline-aware content variants and Claude Haiku upgrades take entries the rest of the way.

What This Doesn't Replace

Hash-pool enrichment produces summaries that are accurate and specific. It does not produce summaries that explain what a model is distinctly good at compared to alternatives, or which deployment scenarios favor it. For that you need editorial judgment — which means Claude, not template expansion.

The isTemplateContent check in the model detail pages noindexes entries still on bare fallback content. Polish-upgraded entries have specific pros/cons populated, so they pass the check and become visible to Google. That's the practical value of this bridge step: moving pages into indexable territory without requiring Claude spend on every single entry.

Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.

Top comments (0)