An agent that learns the world it acts in: System 1, System 2, and knowledge you can retract

Most LLM agents start every task from zero. They explore, they find the way, and the next day they explore again. Some keep a memory, but it is usually text the model wrote about itself, which nobody can check or take back. This last part of the series shows the other end of solvi 1.0: an agent that keeps verified knowledge about the world it acts in, acts fast where that knowledge predicts the action works, deliberates where it does not, and keeps written rules as hard checks in both. I wrote solvi; part 5 of 5 (overview).

The two systems

Each step is one decision with two possible answerers:

• System 1 takes an action the knowledge predicts will work and that advances an open goal: a goal's skill (the action after which its done check turned true before) or the next step of a confirmed route to where the skill worked.
• System 2 is a search over the offered actions when System 1 has nothing it is sure of: an open goal's action first, then what was never tried here, then the way to the nearest place with untried actions.
• Hard checks hold in both: the agenda's gates, a hard refusal of the action model, the failure memory.

After every step the knowledge is written back: the action model learns from the outcome, a goal done right after an action makes a skill, and where each action led becomes a map fact. Every decision is stored with what it was given and agent.replay() re-checks it without the environment.

The world and the knowledge

Example 25 in the repo is a toy crafting world: 8 places joined by exits, resources (tree, stone, iron, water), six operators with the environment's own checks (collect_stone needs stone here and a pickaxe, and so on), six goals. The agent knows none of those rules. It gets a vocabulary — the conditions its action model may learn from — and goals whose done checks are code, never a model's claim. Excerpt from examples/25_environment_agent.py:

VOCABULARY = {"here": lambda s, a: s["here"], "wood": lambda s, a: s["wood"], "stone": lambda s, a: s["stone"],
              "pickaxe": lambda s, a: s["pickaxe"], "stone_pickaxe": lambda s, a: s["stone_pickaxe"],
              "table_here": lambda s, a: s["place"] in s["tables"]}


def place(state):                                                # the map's keys: the places
    return state["place"]


def goals(km):
    """The six goals: done when the operator has worked once (code over the state, never a model's claim)."""
    for op, needs in zip(OPS, [None] + OPS[:-1]):
        km.goal(op, done=lambda s, op=op: op in s["achieved"], requires=[needs] if needs else [])
    return km

The agent itself is three lines (excerpt, same file):

    km = goals(solvi.Knowledge(vocabulary=VOCABULARY))
    agent = solvi.Agent(Crafting(), knowledge=km, key=place)
    runs = [("world 7, run 1", agent.run(seed=7, steps=150)), ("world 7, run 2", agent.run(seed=7, steps=150)),
            ("world 8 (new)", agent.run(seed=8, steps=150))]

Crafting is any object with reset(seed), actions(state) and step(action) → Outcome — the Environment protocol; solvi.testing.conformance.check_environment checks one of your own.

The same world runs in your browser, with no install: the arcade Space's "Agent and knowledge (1.0)" tab (https://huggingface.co/spaces/solvi-ai/arcade).

The same world, met again

The whole output of the file, verbatim:

Part 1 — the same world met again needs fewer slow decisions
  world 7, run 1: 61 steps, 6 of 6 goals, System 1 0, System 2 61, fallback 0, refused 23
  world 7, run 2: 12 steps, 6 of 6 goals, System 1 11, System 2 1, fallback 0, refused 0
  world 8 (new): 17 steps, 6 of 6 goals, System 1 4, System 2 13, fallback 0, refused 3
  every decision replays: 90 of 90; the knowledge journal verifies: True
  learned: collect_stone needs ["'stone'"] here and a pickaxe ['True']

Part 2 — protection vs justified risk (30 episodes on a world with a breaking bridge to the iron)
  protect : iron in 2 of 30 episodes, fell 1 times, 0 risky crossings, 5.07 goals per episode
  risk    : iron in 24 of 30 episodes, fell 6 times, 27 risky crossings, 5.80 goals per episode
  the gate 'no bridge without the stone pickaxe' held in both: True

The first run explores: 61 steps, every one decided by System 2, 23 actions refused by the world. The second run in the same world takes 12 steps, 11 of them by System 1, nothing refused. In a new world the agent keeps the rules (what an operator needs) and re-learns the map: 17 steps. The last line of part 1 is what the action model learned, readable: collect_stone works with stone here and a pickaxe.

Map facts and rules are kept apart on purpose. Rules are carried across episodes; map facts belong to the map they were seen on, and the first arrival that contradicts a carried map fact drops the carried map — its facts become hints until confirmed again.

Protection by default, risk as an option

Knowledge used as hard checks removes the failures it predicts. It can also cost a goal that needs a risk. Part 2 of the example has a world where the iron lies only across a rope bridge that breaks one time in three (the episode ends), and a written gate: no bridge without the stone pickaxe. Excerpt:

    for name, risk in (("protect", None), ("risk", RiskBudget(max_risk_per_episode=1.0, min_gain_ratio=1.0,
                                                              min_support=1))):
        km = goals(solvi.Knowledge(actions=OutcomeRates(VOCABULARY, only={"bridge": []})))
        km.agenda.gate("bridge_needs_the_tool", lambda s: s["stone_pickaxe"], blocks=["bridge"])
        agent = solvi.Agent(Crafting(bridge=True), knowledge=km, key=place, risk=risk, gain=bridge_gain)

(OutcomeRates is a 20-line action model of the example's own: the outcome rate of each action per situation — any object with observe, predict and fingerprint plugs in.)

With protection (the default) the agent fell once, predicted "refuse" for the bridge from then on, and never crossed again: iron in 2 of 30 episodes, 5.07 goals of 6. With RiskBudget it crossed 27 times against the prediction, when the expected gain outweighed the estimated risk within the episode's budget: it fell 6 times and got the iron in 24 episodes, 5.80 goals. The written gate held in both; gates, hard predictions and the failure memory are never traded.

The honest reading: justified risk lowers the cost of protection. It does not promise to do as well as an agent with no knowledge at all. In a harder dungeon game it won back part of what protection cost, not all of it. Measure it on your own metric before turning it on.

The same idea on a real game map

Example 23 walks the world map of Pokémon Red — 190 places and 447 exits, recorded from a real playthrough (place names and exits only; no ROM bytes, no graphics) — through the game's first fifteen goals, twice. System 1 is two rules over remembered routes; System 2 searches the world map the player builds as it goes; after each goal, what System 2 found becomes System 1's routes. From its output (the per-goal table cut to its totals):

  goal             run 1: moves   S1   S2   run 2: moves   S1   S2   shortest
…
  all goals                 183   36  147             62   60    2         62
…
  → run 2 needs 2 slow decisions instead of 147, and 62 moves instead of 183.

The second run takes the fewest moves possible (62, the "shortest" column), with 2 slow decisions out of 62. And at the end: "245 of 245 stored decisions replay". The recording lives in the repo's spaces/pokemon/, so run it from a clone.

What it does not do

• Growth was shown only in environments met again — this crafting toy and the Pokémon map. It was not shown on streams of one kind of decision (classification, matching) or for a support agent with tools (parts 1, 2 and 4 of this series say what the store gives there instead: accountability, not growth).
• It does not shorten the first exploration. Run 1 is a search; knowledge pays off on the second visit.
• The action model learns only what the environment checks, over your vocabulary. A rule the world does not enforce must be written as a gate.
• Rules learned in one world can be over-cautious in another. A condition that held at every success by accident does not transfer; the model then says "unknown", not "wrong".
• The knowledge store is measured to 10,000 items, not beyond.

What every item does have: a source (a person, an outcome, a written spec, or a verified System 2 answer — never the agent's own guess), and an exact retraction that takes back everything derived from it.

The series

• Part 1: routing tickets with a promise
• Part 2: product matching with a rule across answers
• Part 3: contract clauses with quotes you can check
• Part 4: guarding a support agent's tool calls
• Docs: Using solvi: agents and knowledge, mission and honest limits
• Full code and outputs: solvi-ai/recipes: 05-environment-agent
• Code: github.com/solvi-ai/solvi · pip install solvi

Where does your agent meet the same world again — the same codebase, the same internal tools, the same warehouse? That is where this kind of memory should pay off. And when your agent's knowledge turns out wrong, what would you need to see to trust a retraction of it?