Open-source agents fail on long-horizon tasks because they lack two things: training data with verifiable intermediate states, and a reward signal that guides multi-hour execution without leaking the final goal. Tev-Gym and its progress-potential reward shaping (PPRS) recipe address both. This is infrastructure for training agents on economically valuable workflows like portfolio rebalancing, earnings analysis, or recursive self-improvement loops that span days.
The Sparse Reward Problem
Most agent benchmarks provide binary feedback: task complete or task failed. If a trading agent must gather market data, run a valuation model, check compliance rules, and execute trades across four exchanges, you get zero signal until the final step. That makes credit assignment nearly impossible.
Traditional RL uses dense rewards, but hand-crafting intermediate rewards for every possible agent task is not scalable. You need a systematic way to measure progress without encoding the solution path.
Latent-World-First Construction
Tev-Gym builds 1,170 verified task environments using a three-layer approach:
- Latent world specification: A complete hidden model of the task domain, workflow, and expected state after each step. This is the ground truth.
- Progressive rendering: The environment is generated from the spec, so complexity is controllable and every intermediate state is verifiable.
- Progress metric extraction: Because the spec knows the full workflow, it can measure how far an agent has advanced without revealing the solution.
This inverts the usual benchmark design. Instead of collecting tasks and reverse-engineering evaluation criteria, you start with a formal task model and render environments from it.
Progress-Potential Reward Shaping
PPRS converts the progress metric into a per-step reward. The key insight: reward the agent for reaching states with higher potential to complete the task, not for specific actions.
The reward function looks like:
def pprs_reward(state, next_state, task_spec):
progress_current = task_spec.measure_progress(state)
progress_next = task_spec.measure_progress(next_state)
potential_delta = progress_next - progress_current
return potential_delta + sparse_terminal_reward(next_state)
This satisfies the reward shaping invariance property: optimal policies under the dense reward are also optimal under the sparse terminal reward. You are not changing what the agent should do, just making the gradient less noisy.
Checkpoint Granularity Trade-Offs
Long-horizon tasks require checkpointing, but storage and evaluation costs scale with checkpoint frequency. Tev uses task-aware checkpointing based on the latent world model.
| Checkpoint Strategy | Storage Cost | Evaluation Overhead | Catastrophic Forgetting Risk |
|---|---|---|---|
| Every step | High (100% of states) | High (constant validation) | Low |
| Fixed interval (every N steps) | Medium | Medium | Medium |
| Task-aware (workflow milestones) | Low (5-10% of states) | Low (only at decision points) | Low |
| Terminal only | Minimal | Minimal | High |
Task-aware checkpointing saves states only when the latent world model indicates a workflow boundary: data gathered, model trained, compliance check passed. This reduces storage by 90% while preserving the ability to resume from meaningful intermediate states.
Training Recipe and Results
Tev-27B was trained exclusively on ALE (Agents' Last Exam) computing and math science domains using PPRS. The model matches GPT-6 Astra and Claude Opus 5 at Pass@5 on ALE-CM, despite being open-source and smaller.
Generalization is the real test. Without seeing Terminal-Bench data during training, Tev-27B surpasses GLM 5.2 and Gemini 3.7 Flash on TB 3.0 and 4.0 subsets. On middle-horizon tasks (TB 2.1), it reaches Claude Opus 4.8 performance.
This suggests the PPRS recipe teaches transferable task decomposition, not just domain-specific heuristics.
Deployment Shape for Financial Workflows
A portfolio rebalancing agent using this approach would:
- Define the latent world: Asset universe, rebalancing rules, compliance constraints, execution venues.
- Render the environment: Market data feeds, order management system, compliance API.
- Instrument progress: Checkpoints at data collection, valuation completion, compliance approval, execution start.
- Train with PPRS: Reward progress through each workflow stage without hardcoding trade logic.
- Deploy with state persistence: Save checkpoints at workflow boundaries so the agent can resume after interruptions.
The latent world model becomes your observability layer. You can inspect whether the agent is stuck, regressing, or making genuine progress toward the terminal goal.
Likely Failure Modes
Reward hacking: If the progress metric is poorly designed, agents will find shortcuts. A trading agent might mark compliance as complete without actually checking rules if the progress function only counts API calls.
Checkpoint bloat: Even task-aware checkpointing can explode storage if workflows have many decision points. You need retention policies and compression.
Generalization boundaries: Tev-27B generalizes across task lengths but stays within similar domains (computing, math). Jumping to entirely new domains (legal research, hardware design) likely requires retraining.
Latent world drift: If the real environment changes (new compliance rules, API deprecation), the latent world model becomes stale. You need a process to update the spec and re-render environments.
Technical Verdict
Use Tev-style PPRS when:
- Your task has clear intermediate milestones (data gathering, model training, approval gates).
- You need open-source models to handle multi-hour workflows.
- You can invest in building a latent world model for your domain.
- Storage and compute budgets allow task-aware checkpointing.
Avoid when:
- Tasks are truly unstructured with no natural progress metric.
- You need real-time execution with sub-second latency (checkpointing overhead is too high).
- The environment changes faster than you can update the latent world spec.
- You already have a proprietary model that handles long-horizon tasks and do not need training infrastructure.
The real contribution is not the specific model but the recipe: latent-world-first design plus progress-potential reward shaping. That is reusable infrastructure for any domain where you need agents to learn multi-day workflows.
Top comments (0)