I built a Rust CLI that filters datasets using typed AI decisions instead of LLM judges

Cleaning a large synthetic or pretraining dataset is a pain nobody talks about enough. You end up with three bad options.

Regex and keyword filters are fast but blind. They catch a malformed JSON row. They don't catch a math derivation that's circular, or a response that's sycophantic filler dressed up as reasoning.

Running a full LLM as a judge catches the subtle stuff. It also costs real money at scale and crawls compared to what you actually need.

I wanted a third option, so I built jev-curate around TypeSafe's Jev model. Jev doesn't generate text. You send it a state plus a typed question, it sends back a typed answer: a category choice, a yes/no probability, an ordinal score. No output to parse, no risk of it rewriting your data on you.

Quick start

\`bash
cargo install jev-curate
export TYPESAFE_API_KEY="your-api-key"

jev-curate filter train.parquet \
--preset reasoning-math \
--out ./output/ \
--concurrency 32
`\

What ships in v0.1.0

Three presets:

• reasoning-math — checks for circular logic and invalid reasoning steps, scores depth 1 to 5
• anti-sycophancy — flags "as an AI..." filler and ungrounded flattery
• code-correctness — catches unclosed code fences and stub placeholders

Under the hood

Rust core (Cargo, tokio, arrow, parquet) with PyO3 bindings for Python. Streams rows through an adaptive rate limiter so you don't eat 429s.

Where it stands

v0.1.0, single-node bench measured at 24 rows/sec so far, working on getting that higher with better batching. Tests run against a mock server so CI doesn't burn API credits.

Live site with benchmarks and docs: https://jev-curate.vercel.app

AkashPriyadarshii / jev-curate

High-throughput synthetic and pretraining dataset sifter for TypeSafe Jev. Rust streaming core, Parquet and JSONL I/O, typed Choice/Score/Noul judgments, speculative fan-out, 24.0 rows/sec measured.

jev-curate

High-Throughput Synthetic & Pretraining Dataset Sifter Powered by TypeSafe AI (Jev)

Live: jev-curate.vercel.app (measured 24.0 rows/sec single-node on local mock bench examples/bench_mock.rs, 1,500+ cluster target)

By Akash Priyadarshi

Why jev-curate • Quickstart • CLI Reference • Python API • Architecture • Non-Goals • Ecosystem

---

Why jev-curate?

Cleaning 10M to 1B rows of synthetic reasoning data, instruction tuning pairs, or web-scraped corpora is an economic and technical nightmare:

• Generative LLMs are too slow and expensive: Running Claude 3.5 Sonnet or GPT-4o to judge synthetic rows costs $15,000–$50,000 per billion tokens and crawls at a painful 30–50 rows/sec.
• Regex heuristics are blind to reasoning flaws: Keyword and regex filters can check syntax, but fail to detect circular reasoning, hallucinated derivation steps, or robotic sycophancy.
• Context rot from uncompressed inputs: Naively feeding raw data into LLMs causes decision accuracy to crater while burning money on boilerplate text.

jev-curate solves this…

View on GitHub

If you're curating LLM-generated data and tired of the regex-versus-full-judge tradeoff, take a look. Feedback and issues welcome, I'm building this solo out of Patna.