Benchgen - The learning infrastructure for AI agents.

Benchgen Benchgen — Wed, 19 Aug 2026 08:55:56 +0000

BenchGen is the learning infrastructure for AI agents — an open platform where developers discover benchmarks and RL environments, connect and evaluate their complete agent systems, and continuously improve them against verifiable rewards.

Most evaluation tools only trace what already happened. BenchGen closes the loop: benchmark → evaluate → fine-tune → re-evaluate, in one place. And unlike model-only leaderboards, BenchGen evaluates the whole agent — Model + Harness (memory, skills, tools, orchestration) — because most real-world agent failures live in the harness, not the model weights.

Every benchmark on BenchGen carries a verifiable reward: a machine-checkable pass condition, not a subjective score. This is Test-Driven Development applied to agents — define the benchmark and the verifiable reward first, then build or fine-tune the agent until it consistently beats it.

BenchGen ingests full agent decision trajectories (State → Action → Tool Response → Outcome → Reward), scores them across five dimensions — tool-call accuracy, skill coverage, goal completion, memory utilisation, and regression stability — and exports the clean, filtered data needed for LoRA fine-tuning on open-weight models (Llama 4, Qwen 3 72B, Gemma 4).

BenchGen is open and community-first for discovery — browse benchmarks, models, and leaderboards free, no login required — the same hub model Hugging Face built for models and datasets, applied here to agent benchmarks and RL environments. Usage-based pricing applies only to the compute-heavy work of running, training, and fine-tuning agents at scale.

Unlike LangSmith, Arize Phoenix, or Braintrust — observability tools that show you what your agent did — BenchGen tells you whether it's good enough, and helps you make it better. Unlike environment infrastructure providers, BenchGen connects environments directly to diagnosis, benchmarking, and improvement, not just execution.

The 88% of AI agent pilots that never reach production (Forrester 2026) fail primarily because teams cannot evaluate agent quality reliably. BenchGen is the infrastructure that makes that evaluation systematic — and turns it into the training data that closes the gap.

Check out Benchgen Docs

DEV Community: Benchgen Benchgen

Benchgen - The learning infrastructure for AI agents.