EnvACE proves that internal world rehearsal can replace costly external interactions while still delivering strong transfer on tool‑use benchmarks. The twist is that the policy learns to simulate environment responses itself, reducing the need for large numbers of external API calls by using inexpensive forward passes through its own parameters.
Before EnvACE, training LLM agents for long‑horizon tool use relied on either hand‑crafted simulators or live API calls, both of which inflate compute costs and complicate reproducibility. Approaches such as simulator‑based reinforcement learning kept the agent’s policy tethered to an external environment loop, limiting scalability across diverse tasks.
EnvACE improves the average score from 31.2 % to 36.7 %, a gain of 5.5 % [1]. The method interleaves an action generation step with a self‑play rehearsal where the same model produces the expected tool response, and both roles are optimized jointly under task‑success rewards. This joint training internalizes the action–response mapping directly in the policy’s weights.
Parallel rehearsal at test time pushes the Overall score to 40.9 %, improving the Non‑TTS result of 36.7 % by 4.2 % [1]. After a primary decision, the agent can privately rehearse alternative tool calls and select the most promising one before committing, all without additional external environment queries.
The paper’s limitations include dependence on the fidelity of the internally learned world model; if the rehearsal dynamics diverge from real‑world APIs, performance may degrade. Moreover, the benchmarks focus on well‑structured tool interactions, leaving open whether the same gains hold for chaotic or highly stochastic environments.
If internal rehearsal scales as reported, developers should replace most external interaction loops with a modest private rehearsal budget during both training and inference. Re‑running existing tool‑use suites under this regime could slash compute expenses by orders of magnitude while preserving—or even improving—benchmark performance.
Top comments (0)