My AI model directory stores 1,449 entries across two tables in Turso libSQL: models for HuggingFace metadata and model_content for AI-generated or metadata-derived prose. The ETL runs on a GitHub Actions cron and is silent on success. Without diagnostic queries, gaps accumulate invisibly — entries with no generated content, stale prose from models that have changed, boilerplate clustering that triggers the lint:humanization --strict check.
These four queries are the ones I run when something looks off, and occasionally on a schedule to catch drift before it matters.
1. Find models with no generated content
SELECT m.id, m.slug, m.pipeline_tag, m.downloads
FROM models m
LEFT JOIN model_content c ON c.model_id = m.id
WHERE c.model_id IS NULL
ORDER BY m.downloads DESC
LIMIT 50;
The LEFT JOIN combined with WHERE c.model_id IS NULL returns every row in models that has no corresponding row in model_content. Those entries render as bare stubs on the site — model name, download count, tags, but no summary or pros/cons.
Sorting by downloads DESC surfaces the gap entries that matter most. A missing entry for a model with 2 million downloads is more urgent than one with 300. After the three-tier content quality ladder runs, I expect this list to shrink by roughly 100 entries per day. If it's not shrinking, the upgrade step in the ETL is silently failing.
2. Find content that's gone stale
SELECT m.slug, m.name, c.generated_at, c.model_used,
julianday('now') - julianday(c.generated_at) AS days_old
FROM models m
JOIN model_content c ON c.model_id = m.id
WHERE days_old > 30
AND c.model_used NOT LIKE 'metadata-derived%'
ORDER BY days_old DESC
LIMIT 30;
The julianday arithmetic is SQLite-native and works in libSQL without modification. I exclude metadata-derived entries because those are rebuilt deterministically from HuggingFace metadata, not from an LLM run — their "age" doesn't indicate drift in the same way.
The 30-day threshold isn't about content expiry in any absolute sense. It's about model churn on HuggingFace: popular models release new versions frequently, and a summary written for qwen2-7b-instruct six weeks ago may not reflect current quantization formats or license updates. When this query returns more than 50 entries, I queue a partial regen run targeting the oldest content for the highest-download models first.
Knowing model_used per row also matters here. If stale entries are concentrated on claude-haiku-4-5 versus an earlier model, that flags a generation epoch boundary — the older entries were generated before a prompt change and may follow a deprecated template.
3. Detect boilerplate clustering before it accumulates
SELECT c.summary, COUNT(*) AS occurrences
FROM model_content c
WHERE c.model_used NOT LIKE 'metadata-derived%'
GROUP BY c.summary
HAVING COUNT(*) > 3
ORDER BY occurrences DESC
LIMIT 20;
This is the query that would have caught the 467-instance repetition earlier if I'd run it regularly. It groups on exact summary text and returns any string that appears more than three times across the directory.
A count above 3 for any non-trivial sentence is a signal that a generation prompt is producing clustering. The HAVING COUNT(*) > 3 threshold is loose by design — some shared language is fine — but anything above 10 almost certainly needs to be split with the humanize-aiappdex.mjs pass or flagged for a prompt revision.
You can run the same query against the pros and cons fields by replacing c.summary. In practice I run all three and compare the worst-offender counts. The pros field tends to cluster most aggressively because LLM outputs converge on positive framing for common attributes (licensing, framework compatibility, inference speed) more readily than they converge on neutral summaries.
4. Check pipeline coverage by generation tier
SELECT
CASE
WHEN c.model_used IS NULL THEN 'no-content'
WHEN c.model_used LIKE 'metadata-derived%' THEN 'metadata-derived'
WHEN c.model_used LIKE 'fallback%' THEN 'fallback-template'
ELSE 'llm-generated'
END AS tier,
COUNT(*) AS count,
ROUND(COUNT(*) * 100.0 / (SELECT COUNT(*) FROM models), 1) AS pct
FROM models m
LEFT JOIN model_content c ON c.model_id = m.id
GROUP BY tier
ORDER BY count DESC;
This returns a coverage breakdown across the four content states the directory can be in:
| tier | count | pct |
|---|---|---|
| llm-generated | 1021 | 70.4% |
| metadata-derived | 251 | 17.3% |
| fallback-template | 130 | 9.0% |
| no-content | 47 | 3.2% |
The numbers shift as the daily upgrade cron runs. What I'm watching: fallback-template should be shrinking toward zero (the deterministic hash pool upgrade handles most of these), no-content should stay near zero after initial seeding, and metadata-derived should remain stable unless the humanization script runs again or the lint check finds new patterns.
If no-content starts growing, a fetch step in the ETL is writing to models without triggering the content generation step. If fallback-template stops shrinking, the upgrade queue has stalled.
Running these in a CI health check
I wrap all four in a small Node script that connects via the Turso @libsql/client package and logs the results to a GitHub Actions step summary:
import { createClient } from "@libsql/client";
const db = createClient({
url: process.env.TURSO_URL!,
authToken: process.env.TURSO_TOKEN!,
});
const missing = await db.execute(`
SELECT COUNT(*) AS n FROM models m
LEFT JOIN model_content c ON c.model_id = m.id
WHERE c.model_id IS NULL
`);
console.log(`Missing content: ${missing.rows[0].n}`);
The check runs at the end of the daily ETL workflow and fails the step if missing > 100 — a loose threshold that alerts before gaps compound across multiple failed runs without being noisy on normal days when a few entries are expected to be in flight.
The GitHub Actions cron patterns for this job sit at 0 2 * * * UTC, an hour after the main ETL runs, so the health check always sees the freshest state.
Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.
Top comments (0)