I beat CartPole's random baseline with 5-line memory queries, no training, no download

No dataset to download for this one — just pip install gymnasium and a deterministic simulation you can run yourself in a few minutes. I wanted a test of vaas-x's outcome-grounded retrieval that had nothing to do with sensors or IoT, so I picked the most over-used toy environment in RL: CartPole.

The question isn't "can this beat a trained RL policy" — it can't, and that's not the point. The question is whether biasing action choice toward what previously kept the pole up, using nothing but similarity retrieval over logged episodes, beats picking actions at random.

Baseline first

import gymnasium as gym

env = gym.make("CartPole-v1")

def random_episode():
    obs, _ = env.reset()
    steps = 0
    while True:
        action = env.action_space.sample()
        obs, _, terminated, truncated, _ = env.step(action)
        steps += 1
        if terminated or truncated:
            return steps

baseline = [random_episode() for _ in range(50)]
print("random policy, mean steps:", sum(baseline) / len(baseline))

Log episodes as memory, not training data

Each observation gets bucketed by sign into a short state description, and the action taken goes into source_text so it comes back out of retrieval later:

from vaasx import Bootstrap

brain = Bootstrap(api_key="YOUR_API_KEY", device_id="cartpole")

def bucket(obs):
    labels = ["cart_left", "cart_right"], ["cart_still", "cart_moving_right"], \
              ["pole_left", "pole_right"], ["pole_falling_left", "pole_falling_right"]
    return " ".join(labels[i][int(v > 0)] for i, v in enumerate(obs))

log = []  # kept locally too, for the independent check later
for _ in range(200):
    obs, _ = env.reset()
    ep_items, ep_states, ep_actions = [], [], []
    while True:
        action = env.action_space.sample()
        state_text = bucket(obs)
        ep_items.append({
            "payload": {f"obs_{i}": float(v) for i, v in enumerate(obs)},
            "source_text": f"{state_text}; action_taken={action}",
        })
        ep_states.append(state_text)
        ep_actions.append(action)
        obs, _, terminated, truncated, _ = env.step(action)
        if terminated or truncated:
            break
    resp = brain.ingest(ep_items)
    for i, episode_id in enumerate(resp["episode_ids"]):
        if episode_id is None:
            continue
        # every step kept the pole up except the last one in the episode
        success = i < len(resp["episode_ids"]) - 1
        brain.outcome(episode_id, success=success)
        log.append({"state": ep_states[i], "action": ep_actions[i], "success": success})

200 episodes of a random policy, logged with the one honest signal available: did this step precede the pole falling, or not.

Recall an action instead of choosing one blind

import re

def recall_action(obs, k=5):
    hits = brain.query(bucket(obs), k=k, prefer_success=True)
    votes = [int(m.group(1)) for h in hits if (m := re.search(r"action_taken=(\d)", h["text"]))]
    if len(votes) < 2:
        return env.action_space.sample()
    return max(set(votes), key=votes.count)

Five lines. Query for similar past states, weight toward the ones that succeeded, take a majority vote over what action was taken in those.

Does it actually help?

def memory_episode():
    obs, _ = env.reset()
    steps = 0
    while True:
        action = recall_action(obs)
        obs, _, terminated, truncated, _ = env.step(action)
        steps += 1
        if terminated or truncated:
            return steps

memory_run = [memory_episode() for _ in range(50)]
print("memory-augmented, mean steps:", sum(memory_run) / len(memory_run))

Being honest about this: CartPole is noisy over 50 episodes. The gap over random should be visible, but if it isn't obvious on your run, bump the episode count before concluding anything either way — I'd rather say that up front than let a lucky/unlucky seed do the talking.

Check retrieval against a plain nearest-neighbor computed locally

import pandas as pd

local = pd.DataFrame(log)
local_pref = local.groupby("state").apply(lambda g: g.loc[g["success"], "action"].mode())
print(local_pref)

For a handful of states, compare local_pref — a majority vote computed with nothing but pandas over your own local log — against what recall_action actually returns from the API. They're answering the same question two different ways; they should agree.

Run it yourself

Full guide here — zero downloads, free-tier key at vaasx.com/pricing. If you try it, I'm curious whether the gap over random held up on your seed, and by how much.