If you train or evaluate web agents on WebArena, a lot of the cost is the environment rather than the agent: five heavy stacks (Magento, Postmill, GitLab, Kiwix, OpenStreetMap) and resets that mean restarting containers.
I rebuilt the five WebArena sites as lightweight, self-hostable versions: a store with its back office, a forum, a code-hosting site, a wiki and a map. Each site is one SQLite database plus one small Node server. It boots in under a second and resets to its initial state in milliseconds (a rollback of a transaction that is never committed), so one machine can run dozens of independent copies of every site.
What is included:
- All 812 WebArena tasks converted to these sites: 807 verified by browser runs, 5 dropped with the reasons documented.
- 12,283 generated tasks from 217 templates, each template checked with UI runs.
- Template-disjoint train and held-out splits (10,452 and 2,638 tasks), so no held-out template is seen in training.
- Graders that check answers, URLs and site state.
The data (about 21 GB, the same underlying data as WebArena) is a Hugging Face dataset, and the code is Apache-2.0.
Code: https://github.com/junfengzh/websandbox
Data: https://huggingface.co/datasets/junfengzhou/websandbox
Demo, one copy of each site reset hourly: https://websandbox.pystem.com
Feedback is welcome, especially from anyone running agent RL or evals on WebArena-style tasks.
Top comments (0)