Active Inference, The Learn Arc — Part 23: Session §4.1 — States, Observations, Actions — The Three Lists

#activeinference #pomdp #ai #elixir

Series: The Learn Arc — 50 posts teaching Active Inference through a live BEAM-native workbench. ← Part 22: Session 3.4. This is Part 23.

The session

Chapter 4, §1. Session title: Setup (states / observations / actions). Route: /learn/session/4/s1_setup.

Chapter 3 told you what an Active Inference agent is. Chapter 4 tells you how to specify one concretely. The first three lists you write down — hidden states, observations, actions — are the skeleton of every subsequent matrix. Get these wrong and nothing downstream can save you.

The three lists

1. Hidden states (s). The unobservable variables the agent will infer. These are the latent facts about the world that the agent tracks across time.

Tiny corridor: s ∈ {L, M1, M2, R} — four positions.
Coin toss: s ∈ {fair, biased} — two hypotheses.
Robot arm: s = (joint_angle, joint_velocity) — continuous, or discretized.

2. Observations (o). The sensory channels the agent receives. These are all the evidence Q(s) can use.

Tiny corridor: o ∈ {wall_north, wall_south, wall_east, wall_west} — one channel indicating visible walls.
Coin: o ∈ {heads, tails}.
Robot arm: o = (tactile_pressure, proprioception) — typically vector-valued.

3. Actions (a). The controls the agent can emit. The action space determines the shape of B.

Tiny corridor: a ∈ {left, right} — move cardinally.
Coin: a ∈ {flip, wait} — flip or don't.
Robot arm: a = torque_per_joint — continuous.

Three lists. Finite (mostly). Every matrix in Chapter 4 is indexed over at least two of them.

How to pick them well

The three lists are choices, not facts. Different choices give different agents, with different tractability, different interpretability, different failure modes. The session walks through the four design heuristics that keep you out of trouble:

H1: States should be things the agent needs to remember across ticks. If the agent needs to know position next tick to plan, position is a hidden state. If it doesn't, it isn't.

H2: Observations should be things the world actually emits. Not summaries the agent computes. A wall_north observation is fine; a "distance_to_goal" observation is a cheat — the agent is supposed to compute that from its belief state.

H3: Actions should be things the agent actually emits to the world. Not internal reasoning steps. "Decide to plan deeper" is not an action; moving a joint is.

H4: Keep the cardinality of each list small until forced otherwise. A 3×3 maze with 9 states converges in seconds; a 50×50 grid in minutes; a continuous state space needs generalised coordinates (Chapter 8). Start small.

The Workbench's shortcut

Every recipe in /cookbook has its three lists pre-written in the spec. Open any recipe detail page and the Runtime block tells you the state cardinality, observation structure, and action vocabulary. You can read off the three lists without writing any code.

For example, /cookbook/pomdp-tiny-corridor implicitly defines:

|s| = 4 — four corridor cells.
|o| = 4 — one wall-observation channel with 4 values.
|a| = 2 — left and right.

That's the recipe's spine.

The blanket link

Session 4.1 also introduces the concept that ties states/observations/actions to the three-plane architecture from Chapter 1: the Markov blanket.

The blanket is the contract. Observations flow in through the blanket (world → agent). Actions flow out (agent → world). Hidden states live inside the agent. Nothing else crosses.

In the Workbench that's literally how the code is separated:

world_plane — emits ObservationPackets.
agent_plane — emits ActionPackets.
shared_contracts — the only types that cross.

A plane_separation_test.exs enforces this at CI. Session 4.1 is where a learner first sees the blanket as a design tool, not just a mathematical convenience.

The four-modality expansion

One subtlety that matters: observations don't have to be scalars. Most Active Inference models use factored observation spaces — multiple independent modalities, each with its own A matrix.

Tiny corridor: one modality (wall_signature) with 4 values.
Multimodal frog: two modalities (visual, acoustic), each with its own A.
Robot arm: three modalities (vision, touch, proprioception).

Factoring lets you scale observations without blowing up A's row count. The /cookbook/multimodal-* recipes exercise this explicitly.

The concepts this session surfaces

Hidden state — unobservable latent variable.
Observation modality — one independent channel of evidence.
Action vocabulary — the discrete set of emissions.
Markov blanket — the three-list contract.

The quiz

Q: You're designing an agent that navigates a 4×4 grid with walls. What's a reasonable hidden-state list?

☐ The grid coordinates (x, y) — 16 states. ✓

☐ The distance to the goal — 1 continuous state.

☐ The observation last tick — 4 states.

☐ The action last tick — 4 states.

Why: Position is what the agent needs to remember to plan. "Distance to goal" is a derived quantity the agent should compute from its belief, not something the world stores. The state list should capture what persists across ticks — grid coordinates do; last-tick observations/actions don't.

Run it yourself

/learn/session/4/s1_setup — session page.
/cookbook/pomdp-tiny-corridor — four-state worked example.
/cookbook/multimodal-two-channel-frog — two observation modalities.
/cookbook/perception-blanket-channel-choice — what happens when you pick the wrong channel.
/builder/new — define your own three lists on the canvas.

The mental move

The three lists are the smallest unit of commitment in Active Inference. Everything — the matrices, the equations, the quizzes, the neuroscience mappings — rides on the three lists you picked. Spend disproportionate time on them. The hardest bugs in any Active Inference project trace back to Session 4.1-level decisions.

Part 24: Session §4.2 — The A matrix. P(o | s) gets a shape and a purpose. How to fill an A matrix when the sensor is deterministic, noisy, or learned. The bridge from "list of observation modalities" to "computable generative model."

⭐ Repo: github.com/TMDLRG/TheORCHESTRATEActiveInferenceWorkbench · MIT license

📖 Active Inference, Parr, Pezzulo, Friston — MIT Press 2022, CC BY-NC-ND: mitpress.mit.edu/9780262045353/active-inference

← Part 22: Session 3.4 · Part 23: Session 4.1 (this post) · Part 24: Session 4.2 → coming soon