DEV Community: heavy shield

I Made Spider-Man Swing Without Animating a Single Frame

heavy shield — Wed, 07 Oct 2026 02:42:06 +0000

I built a Spider-Man swing effect that runs entirely in the browser, with no pre-rendered animation, no sprite sheets, and no video files. Instead, a small AI agent loop generates motion parameters in real time. Here’s how it works, with runnable Python code you can adapt.

Introduction

Animation is expensive. Traditional 2D or 3D animation requires keyframes, interpolation curves, and often a rendering pipeline. For a simple web demo, that’s overkill. I wanted Spider-Man to swing across a city skyline using only procedural motion driven by an agent. The agent doesn’t generate pixels; it generates physics parameters (rope length, swing angle, release timing) that a canvas renderer uses to draw a stick-figure Spider-Man.

The result: a 60 FPS swing that feels dynamic, with zero pre-baked frames. The agent runs in Python (using a lightweight loop) and communicates with the browser via WebSocket. This post explains the problem, the solution, and the implementation.

Problem

Swinging is a physics problem: a pendulum with a changing pivot point. You could hardcode a sine wave, but it looks robotic. You could use a full physics engine, but that adds dependencies and complexity. I wanted an agent that decides when to release the web and where to attach the next one, based on the current state. The agent must:

Observe the current position and velocity.
Choose an anchor point (a building) within range.
Compute the swing arc.
Decide when to release to maintain forward momentum.
Repeat until the character reaches the right edge of the screen.

This is a classic control loop. The challenge is making it robust and fast enough for real-time.

Solution

I built a simple agent loop in Python that runs at 30 Hz. It uses a state machine with explicit termination conditions:

Observe: Get current position (x, y) and velocity (vx, vy) from the browser.
Decide: If no web is attached, find the nearest building anchor within a cone ahead. If a web is attached, compute the pendulum angle and check if the character is at the bottom of the swing and moving forward. If so, release.
Act: Send the anchor point or release command to the browser.
Terminate: Stop when x > screen width or after 500 iterations (safety).

The agent doesn’t use machine learning; it’s a rule-based controller. But it’s an “agent” in the sense of perceiving, deciding, and acting in a loop. I call it an “agent-style” loop because it’s inspired by AI agent architectures (observe-decide-act) but doesn’t use an LLM.

Implementation

Python Agent (server.py)

This code runs a WebSocket server that sends commands to the browser. It uses websockets and asyncio. Install with pip install websockets.

import asyncio
import json
import math
import websockets

# Configuration
SCREEN_WIDTH = 800
SCREEN_HEIGHT = 600
GRAVITY = 0.5
SWING_SPEED = 0.1
MAX_ITERATIONS = 500

# Building anchors (x, y) - top of buildings
ANCHORS = [(100, 100), (250, 80), (400, 120), (550, 90), (700, 110)]

class SwingAgent:
    def __init__(self):
        self.x = 50.0
        self.y = 300.0
        self.vx = 5.0
        self.vy = 0.0
        self.anchor = None
        self.angle = 0.0
        self.angular_velocity = 0.0
        self.iterations = 0

    def observe(self, state):
        self.x = state['x']
        self.y = state['y']
        self.vx = state['vx']
        self.vy = state['vy']

    def decide(self):
        if self.anchor is None:
            # Find nearest anchor ahead within range
            best = None
            best_dist = float('inf')
            for ax, ay in ANCHORS:
                if ax > self.x and ax - self.x < 300:
                    dist = math.hypot(ax - self.x, ay - self.y)
                    if dist < best_dist:
                        best_dist = dist
                        best = (ax, ay)
            if best:
                self.anchor = best
                # Initialize angle from vertical
                dx = self.x - best[0]
                dy = self.y - best[1]
                self.angle = math.atan2(dx, dy)
                self.angular_velocity = 0.0
                return {'action': 'attach', 'anchor': best}
        else:
            # Pendulum physics
            length = math.hypot(self.x - self.anchor[0], self.y - self.anchor[1])
            if length < 1:
                length = 1
            alpha = -GRAVITY / length * math.sin(self.angle)
            self.angular_velocity += alpha * 0.1
            self.angle += self.angular_velocity * 0.1
            # Check release condition: at bottom and moving forward
            if abs(self.angle) < 0.1 and self.angular_velocity > 0:
                # Release
                self.anchor = None
                return {'action': 'release', 'vx': self.vx, 'vy': self.vy}
            else:
                # Update position from pendulum
                new_x = self.anchor[0] + length * math.sin(self.angle)
                new_y = self.anchor[1] + length * math.cos(self.angle)
                self.vx = (new_x - self.x) / 0.1
                self.vy = (new_y - self.y) / 0.1
                self.x = new_x
                self.y = new_y
                return {'action': 'swing', 'x': self.x, 'y': self.y}
        return {'action': 'none'}

    def should_terminate(self):
        return self.x > SCREEN_WIDTH or self.iterations > MAX_ITERATIONS

async def handler(websocket):
    agent = SwingAgent()
    async for message in websocket:
        state = json.loads(message)
        agent.observe(state)
        if agent.should_terminate():
            await websocket.send(json.dumps({'action': 'stop'}))
            break
        command = agent.decide()
        agent.iterations += 1
        await websocket.send(json.dumps(command))

async def main():
    async with websockets.serve(handler, "localhost", 8765):
        await asyncio.Future()  # run forever

if __name__ == "__main__":
    asyncio.run(main())

Browser Client (index.html)

The client renders a stick figure and sends state updates. It uses the Canvas API.

<!DOCTYPE html>
<html>
<head><title>Spider-Man Swing</title></head>
<body>
<canvas id="canvas" width="800" height="600"></canvas>
<script>
const canvas = document.getElementById('canvas');
const ctx = canvas.getContext('2d');
const ws = new WebSocket('ws://localhost:8765');

let state = { x: 50, y: 300, vx: 5, vy: 0 };
let anchor = null;

ws.onmessage = (event) => {
    const cmd = JSON.parse(event.data);
    if (cmd.action === 'attach') {
        anchor = cmd.anchor;
    } else if (cmd.action === 'release') {
        anchor = null;
    } else if (cmd.action === 'swing') {
        state.x = cmd.x;
        state.y = cmd.y;
    } else if (cmd.action === 'stop') {
        ws.close();
    }
};

function draw() {
    ctx.clearRect(0, 0, 800, 600);
    // Draw buildings
    ctx.fillStyle = '#333';
    [[100, 100], [250, 80], [400, 120], [550, 90], [700, 110]].forEach(([x, y]) => {
        ctx.fillRect(x - 20, y, 40, 600 - y);
    });
    // Draw web
    if (anchor) {
        ctx.beginPath();
        ctx.moveTo(state.x, state.y);
        ctx.lineTo(anchor[0], anchor[1]);
        ctx.strokeStyle = 'white';
        ctx.stroke();
    }
    // Draw Spider-Man (stick figure)
    ctx.beginPath();
    ctx.arc(state.x, state.y, 10, 0, Math.PI * 2);
    ctx.fillStyle = 'red';
    ctx.fill();
    ctx.beginPath();
    ctx.moveTo(state.x, state.y + 10);
    ctx.lineTo(state.x, state.y + 30);
    ctx.strokeStyle = 'red';
    ctx.stroke();
    // Send state
    ws.send(JSON.stringify(state));
    requestAnimationFrame(draw);
}

draw();
</script>
</body>
</html>

Running It

Save server.py and run python server.py.
Open index.html in a browser.
Watch Spider-Man swing. The agent loop terminates when he reaches the right edge or after 500 iterations.

Security warning: This code uses eval? No. It doesn’t. But if you extend it to execute arbitrary code from the browser, never use eval or exec on untrusted input. Always validate and sanitize. The WebSocket server here only accepts JSON with numeric fields; it does not execute code.

Key Takeaways

Agent loops can drive animation: You don’t need an LLM to have an observe-decide-act loop. A simple rule-based controller can produce emergent motion.
Explicit termination is critical: Without a stop condition, the agent would run forever. I used x > SCREEN_WIDTH and a max iteration cap.
Physics can be simple: A pendulum model with release timing is enough for a convincing swing.
Real-time constraints: The loop runs at 30 Hz; the browser interpolates. This keeps CPU usage low.
No frames were animated: The motion is generated procedurally from parameters, not from pre-rendered images.

This approach is extensible: you could swap the rule-based agent for a reinforcement learning policy, or add more complex environments. But even this minimal version shows how an agent-style loop can create dynamic, interactive experiences without traditional animation.

Your API's Newest Users Are Agents: Designing for Non-Human Clients

heavy shield — Wed, 07 Oct 2026 02:41:25 +0000

Introduction

Your API's newest users are agents. They don't read docs, they don't browse dashboards, and they don't file support tickets. They parse OpenAPI specs, call endpoints in loops, and expect deterministic, machine-readable responses. If your API was designed for humans clicking buttons, it's already failing this new class of client.

In this post, I'll walk through a concrete example: building a small "tool API" that an LLM-based agent can call. We'll cover the problem, a solution, and a runnable Python implementation. No hand-waving about "agentic workflows" — just code you can run and a loop with explicit termination conditions.

Problem

Human-facing APIs optimize for discoverability and forgiveness. Agents optimize for determinism and low token cost. Three specific failure modes show up when agents hit a human-designed API:

Ambiguous errors. A 400 with {"error": "bad request"} forces the agent to guess. The agent will retry, hallucinate a fix, or give up.
Unbounded responses. Returning 50KB of nested JSON burns context window and increases the chance the agent truncates the useful part.
No idempotency. Agents retry on timeouts. Without idempotency keys, you get duplicate side effects.

Here's a minimal example of the kind of handler that causes these problems:

# bad_handler.py
from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route("/create_ticket", methods=["POST"])
def create_ticket_bad():
    data = request.get_json(silent=True) or {}
    if "title" not in data:
        return jsonify({"error": "bad request"}), 400
    # ... create ticket, no idempotency, returns everything
    return jsonify({"ticket": {"id": 1, "title": data["title"], "history": [...]}})

An agent calling this has no way to distinguish "missing field" from "malformed JSON" from "server error," and no safe retry path.

Solution

Design the API for a client that is literal, stateless between calls, and token-budgeted. Concretely:

Structured error codes. Every error response includes a stable code string the agent can branch on.
Idempotency keys. Accept an Idempotency-Key header; store the result keyed by it.
Bounded payloads. Return only fields the agent needs; paginate lists with explicit cursors.
A tool schema. Publish a JSON Schema your agent can load to construct calls. This is OpenAI-function-calling-style, not tied to any specific vendor.

Implementation

We'll build a small in-memory ticket API with the properties above, then write an agent loop that uses it. The agent loop is deliberately simple: it's a plan-act-observe loop with a hard step limit and a success predicate. No framework required.

The API

# agent_api.py
import json
import uuid
from dataclasses import dataclass, field, asdict
from typing import Optional

from flask import Flask, request, jsonify

app = Flask(__name__)

@dataclass
class Ticket:
    id: str
    title: str
    status: str = "open"

tickets: dict[str, Ticket] = {}
idempotency_store: dict[str, dict] = {}

TOOL_SCHEMA = {
    "name": "create_ticket",
    "description": "Create a support ticket. Idempotent on Idempotency-Key header.",
    "parameters": {
        "type": "object",
        "properties": {
            "title": {"type": "string", "minLength": 1, "maxLength": 200},
        },
        "required": ["title"],
        "additionalProperties": False,
    },
}

@app.route("/tools", methods=["GET"])
def list_tools():
    return jsonify({"tools": [TOOL_SCHEMA]})

@app.route("/create_ticket", methods=["POST"])
def create_ticket():
    key = request.headers.get("Idempotency-Key")
    if not key:
        return jsonify({"code": "missing_idempotency_key",
                        "message": "Provide Idempotency-Key header."}), 400

    if key in idempotency_store:
        return jsonify(idempotency_store[key]), 200

    data = request.get_json(silent=True)
    if not isinstance(data, dict):
        return jsonify({"code": "invalid_json",
                        "message": "Body must be a JSON object."}), 400

    title = data.get("title")
    if not isinstance(title, str) or not title.strip():
        return jsonify({"code": "invalid_title",
                        "message": "'title' must be a non-empty string."}), 422

    ticket = Ticket(id=str(uuid.uuid4()), title=title.strip())
    tickets[ticket.id] = ticket
    payload = {"ticket": asdict(ticket)}
    idempotency_store[key] = payload
    return jsonify(payload), 201

@app.route("/tickets/<ticket_id>", methods=["GET"])
def get_ticket(ticket_id: str):
    ticket = tickets.get(ticket_id)
    if ticket is None:
        return jsonify({"code": "not_found",
                        "message": f"No ticket {ticket_id}."}), 404
    return jsonify({"ticket": asdict(ticket)})

Key details: every error has a stable code; the response body is small; idempotency is enforced via header, not body, so retries are safe.

The Agent Loop

The agent loop below is deliberately framework-free. It defines explicit termination conditions: it stops when the goal is met, when the model returns no tool call, or when max_steps is reached. I'm using an OpenAI-chat-completions-style interface here as a stand-in; swap in whichever client you use.

# agent_loop.py
import json
import uuid
from typing import Any

import requests

API = "http://localhost:5000"
MAX_STEPS = 6


def call_model(messages: list[dict]) -> dict:
    """Return a message dict. Replace with your model client."""
    # Placeholder: in production, call your LLM here.
    raise NotImplementedError("Wire up your model client.")


def call_tool(name: str, args: dict) -> dict:
    if name == "create_ticket":
        r = requests.post(
            f"{API}/create_ticket",
            json=args,
            headers={"Idempotency-Key": str(uuid.uuid4())},
            timeout=10,
        )
        return {"status": r.status_code, "body": r.json()}
    return {"status": 400, "body": {"code": "unknown_tool", "message": name}}


def run_agent(goal: str) -> dict:
    tools = requests.get(f"{API}/tools", timeout=10).json()["tools"]
    messages = [
        {"role": "system", "content": "You are an agent. Use tools to satisfy the goal."},
        {"role": "user", "content": goal},
    ]

    for step in range(MAX_STEPS):
        msg = call_model(messages)
        messages.append(msg)

        # Termination condition 1: model produced a final answer.
        if not msg.get("tool_calls"):
            return {"status": "done", "steps": step, "answer": msg.get("content")}

        for call in msg["tool_calls"]:
            name = call["function"]["name"]
            args = json.loads(call["function"]["arguments"])
            result = call_tool(name, args)
            messages.append({
                "role": "tool",
                "tool_call_id": call["id"],
                "content": json.dumps(result),
            })

    # Termination condition 2: step budget exhausted.
    return {"status": "max_steps_reached", "steps": MAX_STEPS}

The loop has exactly two exit paths, both explicit. There's no "keep trying until it works" branch, which is where most agent bugs live.

Security Notes

If you extend this to execute code or shell commands, treat the tool boundary as a trust boundary. Never pass model output to eval, exec, or subprocess with shell=True. If you must run generated code, isolate it in a sandbox (a container with no network, a read-only filesystem, and a hard timeout) and validate arguments against the JSON Schema before execution. The example above avoids this entirely by only allowing a single typed tool with a bounded string parameter.

Key Takeaways

Agents are literal clients. Stable error code fields beat prose messages.
Idempotency keys belong in headers and should be required for any write endpoint an agent can retry.
Publish a JSON Schema for your tools so the agent can construct calls without guessing.
Define agent loops with explicit termination conditions — a step limit and a success predicate. "Loop until done" is not a termination condition.
Keep payloads small. Every byte you return is a token the agent pays for.
If you add code-execution tools, sandbox them. eval on model output is a remote code execution vulnerability with extra steps.

The code above is intentionally minimal so you can fork it and add your own tools. Start by auditing your existing API for the three failure modes in the Problem section — that's where the real work is.

What If Your AI Agent Never Had to Leave the Browser?

heavy shield — Tue, 29 Sep 2026 05:30:38 +0000

Introduction

Most AI agents today run in a Python process on a server or your laptop. They call APIs, maybe execute shell commands, and return text. But what if the agent's entire runtime lived inside a browser tab? No backend, no container, no SSH. Just JavaScript and WebAssembly, with a Python kernel compiled to WASM.

This post walks through a minimal agent loop that runs entirely client-side using Pyodide — Python in the browser via WebAssembly. We'll build a tool-using agent that can do arithmetic, read from a virtual filesystem, and stop under explicit conditions. All code is runnable in a modern browser.

Problem

Server-side agents have friction:

Deployment: You need a host, secrets management, and network egress.
Latency: Every tool call is a round trip.
Security surface: Arbitrary code execution on a server is a serious risk.
Demo friction: Sharing a working agent means sharing infrastructure.

A browser-native agent flips this. The sandbox is the browser tab. The runtime is WebAssembly. The only network call is loading the Python runtime itself.

Solution

We'll use Pyodide to run CPython in the browser. The agent loop is a simple ReAct-style loop: the model proposes a tool call, we execute it in Python, append the result, and repeat until a termination condition is met.

We won't call a real LLM here — instead we use a deterministic policy function so the demo is reproducible and offline. Swap the policy for a fetch to an LLM API and the loop is unchanged. We'll call this a "ReAct-style" loop because it follows the observe-think-act pattern, not because it implements the exact paper.

Termination conditions (explicit):

The policy returns a final action.
The step count exceeds MAX_STEPS (default 8).
A tool raises an exception that the policy cannot recover from.

Implementation

1. HTML shell

<!doctype html>
<html>
<head><meta charset="utf-8"><title>Browser Agent</title></head>
<body>
  <pre id="log"></pre>
  <script src="https://cdn.jsdelivr.net/pyodide/v0.26.2/full/pyodide.js"></script>
  <script type="module">
    const log = (m) => document.getElementById('log').textContent += m + '\n';
    const pyodide = await loadPyodide();
    await pyodide.runPythonAsync(await (await fetch('agent.py')).text());
    const result = await pyodide.runPythonAsync('run_agent("What is 21 * 2 plus 8?")');
    log(result);
  </script>
</body>
</html>

2. The agent module (`agent.py`)

import json
import re

MAX_STEPS = 8

# --- Tools -------------------------------------------------------------

def tool_calc(expr: str) -> str:
    """Evaluate a pure arithmetic expression.

    SECURITY WARNING: eval() executes arbitrary Python. This implementation
    restricts input to digits and operators via a regex whitelist. Do not
    remove the whitelist. Do not pass user-controlled strings from an
    untrusted source. For production, use ast.literal_eval or a parser.
    """
    if not re.fullmatch(r"[0-9+\-*/(). ]+", expr):
        raise ValueError(f"unsafe expression: {expr!r}")
    return str(eval(expr, {"__builtins__": {}}, {}))


_FS = {"notes.txt": "remember: 42"}

def tool_read_file(path: str) -> str:
    if path not in _FS:
        raise FileNotFoundError(path)
    return _FS[path]


TOOLS = {
    "calc": tool_calc,
    "read_file": tool_read_file,
}

# --- Policy (replace with an LLM call) ---------------------------------

def policy(question: str, history: list) -> dict:
    """Deterministic stand-in for an LLM. Returns a dict action.

    Swap this for a fetch() to your LLM of choice. The loop below does not
    care where the action came from.
    """
    if not history:
        # First turn: extract a math expression from the question.
        m = re.search(r"([0-9+\-*/(). ]+)", question)
        if m:
            return {"type": "tool", "name": "calc", "args": {"expr": m.group(1).strip()}}
        return {"type": "final", "content": "no expression found"}

    last = history[-1]
    if last["role"] == "tool" and last["name"] == "calc":
        # Second turn: add 8 as required by the question.
        return {"type": "tool", "name": "calc", "args": {"expr": f"{last['result']} + 8"}}
    if last["role"] == "tool" and last["name"] == "calc":
        return {"type": "final", "content": last["result"]}
    return {"type": "final", "content": "done"}

# --- Agent loop --------------------------------------------------------

def run_agent(question: str) -> str:
    history = []
    for step in range(MAX_STEPS):
        action = policy(question, history)
        history.append({"role": "assistant", "action": action})

        if action["type"] == "final":
            return f"[step {step}] {action['content']}"

        if action["type"] == "tool":
            fn = TOOLS.get(action["name"])
            if fn is None:
                history.append({"role": "tool", "name": action["name"],
                                "error": "unknown tool"})
                continue
            try:
                result = fn(**action["args"])
                history.append({"role": "tool", "name": action["name"],
                                "result": result})
            except Exception as e:
                history.append({"role": "tool", "name": action["name"],
                                "error": str(e)})

    return f"[halted: exceeded MAX_STEPS={MAX_STEPS}]"

3. What actually happens

The browser loads Pyodide (about 6 MB gzipped).
agent.py is fetched and executed in the WASM interpreter.
run_agent runs the loop. Step 0 calls calc("21 * 2") → 42. Step 1 calls calc("42 + 8") → 50. Step 2 returns final.
The result is printed to the page.

No server. No API key. The entire agent state lives in the tab and is discarded when you close it.

4. Swapping in a real LLM

Replace policy with something like:

import json
from js import fetch  # Pyodide exposes the browser fetch API

async def policy_llm(question, history):
    resp = await fetch(
        "https://api.example.com/v1/chat",
        {"method": "POST",
         "headers": {"Content-Type": "application/json"},
         "body": json.dumps({"q": question, "history": history})}
    )
    data = await resp.json()
    return json.loads(data.action)

You'll need to make run_agent async and await the policy. The rest of the loop is identical. Note that calling an LLM from the browser exposes your API key to the user — use a short-lived token or a proxy you control.

Key Takeaways

Browser-native agents are real. Pyodide gives you CPython in a tab. That's enough to run a tool-using loop, a virtual filesystem, and a policy function.
Explicit termination matters. Define MAX_STEPS, a final action, and error handling. Without them, loops run forever and burn tokens.
The sandbox is the feature. No server means no remote code execution surface, but it also means no secrets. Design accordingly.
Never eval untrusted input. The tool_calc example uses a regex whitelist and an empty __builtins__ map. That is a mitigation, not a guarantee. For anything real, use ast.literal_eval or a dedicated parser.
The policy is swappable. The loop is agnostic to whether the action came from a regex, a local model, or a remote LLM. That separation is what makes the pattern portable.

Try it: drop the two files in a folder, serve with python -m http.server, and open the page. You'll have an agent that never leaves the browser.

Running an AI Agent Inside the Browser with Pyodide and Ollama

heavy shield — Mon, 28 Sep 2026 00:10:35 +0000

Introduction

Browser automation is the default answer when an AI agent needs to use a web app: launch headless Chrome, drive it with Playwright, ferry state back and forth. It works, but the agent ends up living outside the browser, re-implementing sessions and rendering in parallel.

There's a less obvious option: run the agent inside the browser tab. Pyodide gives us CPython in WebAssembly, the page's DOM and network are directly reachable, and a local model such as Ollama can act as the reasoning backend. This post walks through a small working experiment along those lines, inspired by the tool-call pattern of MCP (Model Context Protocol).

One clarification up front, because it matters: what we build here is an MCP-style tool layer, not a full MCP protocol implementation. The official MCP SDK defines tools, resources, and prompts over standard transports (stdio, Streamable HTTP, SSE). We borrow the "model calls named tools with structured arguments" idea and implement it with a simple JSON message shape. If you want wire-level MCP in the browser, that's a follow-up project, not this one.

The Problem

Driving a browser from Python looks like this:

from playwright.sync_api import sync_playwright

def get_page_text(url):
    with sync_playwright() as p:
        browser = p.chromium.launch()
        page = browser.new_page()
        page.goto(url)
        text = page.inner_text("body")
        browser.close()
        return text

Every interaction is a round-trip between two processes. Session state (logins, cookies, rendered state) lives in the browser your user is actually looking at, while the agent operates on a separate, synthetic copy. Keeping those two worlds in sync is where the bugs and the latency come from.

The Solution

Flip the arrangement: the agent loop runs in the tab, and the page's own APIs become its tools. The architecture:

Browser tab runs a Python agent loop on Pyodide (CPython compiled to WebAssembly).
A small tool registry in the same tab exposes DOM operations (read_dom, click_element) as named tools the model can call with JSON arguments — the same shape MCP popularized.
A local model (Ollama) receives each tool call and returns the next action.
Results are executed directly against the live page.

Model inference does not happen in the browser — the agent loop does. The division of labor is: reasoning in Ollama, perception and action in the tab.

Implementation

Step 1: A tool registry in the browser

Create tools.py:

import json
import js

class ToolRegistry:
    """MCP-style named tools, callable via a simple JSON protocol."""

    def __init__(self):
        self.tools = {
            "read_dom": self.read_dom,
            "click_element": self.click_element,
            "evaluate_js": self.evaluate_js,
            "done": self.done,
        }

    def read_dom(self, selector="body"):
        element = js.document.querySelector(selector)
        if not element:
            return f"No element found for selector: {selector}"
        return element.innerText

    def click_element(self, selector):
        element = js.document.querySelector(selector)
        if not element:
            return f"No element found for selector: {selector}"
        element.click()
        return f"Clicked {selector}"

    def done(self, message=""):
        """Signal that the task is complete."""
        return f"TASK_DONE: {message}"

    def evaluate_js(self, code):
        try:
            return str(js.eval(code))
        except Exception as e:
            return f"JS error: {e}"

    def handle_request(self, request_json):
        request = json.loads(request_json)
        name, args = request.get("tool"), request.get("args", {})
        if name not in self.tools:
            return json.dumps({"error": f"Unknown tool: {name}"})
        try:
            return json.dumps({"result": self.tools[name](**args)})
        except Exception as e:
            return json.dumps({"error": str(e)})

registry = ToolRegistry()
js.window.toolRegistry = registry

Security note: evaluate_js deliberately exposes arbitrary JavaScript execution, including access to the page's session and credentials context. That's acceptable for a local experiment on pages you own. A real deployment should restrict the operation set (whitelist specific actions) and never expose unrestricted eval — the same way a production MCP server validates and sanitizes every tool input.

Step 2: Agent loop in Python

Create agent.py:

import json
import js
from pyodide.http import pyfetch

async def call_ollama(prompt):
    response = await pyfetch(
        "http://localhost:11434/api/generate",
        method="POST",
        headers={"Content-Type": "application/json"},
        body=json.dumps({
            "model": "llama3.2",
            "prompt": prompt,
            "stream": False,
            "format": "json",
        }),
    )
    data = await response.json()
    return data["response"]

async def agent_step(task):
    dom_text = js.window.toolRegistry.read_dom("body")
    prompt = f"""You are a browser agent. Task: {task}
Current page content:
{dom_text[:2000]}

Respond with JSON: {{"tool": "read_dom"|"click_element"|"evaluate_js"|"done", "args": {{...}}}}
Use "done" when the task is complete.
"""
    action = json.loads(await call_ollama(prompt))
    return js.window.toolRegistry.handle_request(json.dumps(action))

async def run_agent(task, max_steps=5):
    for _ in range(max_steps):
        result = await agent_step(task)
        print("Result:", result)
        if "TASK_DONE" in result:
            break

Two things have to be true for this to work:

Ollama must accept cross-origin requests. Browser code calling http://localhost:11434 is a cross-origin request, and Ollama rejects those by default. Start it with an allowlist: OLLAMA_ORIGINS="http://localhost:8000" ollama serve (use your page's actual origin; * works for local experiments only).
The page must reach Ollama over HTTP, not mixed content. Serve your page from http://localhost rather than opening it as file://, otherwise some browsers block the call.

Step 3: Load Pyodide and run

In your HTML page:

<script src="https://cdn.jsdelivr.net/pyodide/v0.26.2/full/pyodide.js"></script>
<script>
async function main() {
  const pyodide = await loadPyodide();
  await pyodide.runPythonAsync(await (await fetch('tools.py')).text());
  await pyodide.runPythonAsync(await (await fetch('agent.py')).text());
  await pyodide.runPythonAsync("import asyncio; asyncio.ensure_future(run_agent('find the pricing link'))");
}
main();
</script>

Run any static file server (python -m http.server 8000), start Ollama with the origins allowlist, open the page, and watch the tab operate on itself.

Key Takeaways

Running the agent loop in the tab removes the browser-automation glue layer: no subprocess management, no session replication.
The tool-call pattern (model emits structured tool invocations, host executes them) travels well beyond MCP's wire protocol — a plain JSON message shape is enough for experiments.
The division of labor matters: agent loop in WebAssembly, inference on a local Ollama. Claiming "everything runs in the browser" would ignore the CORS and mixed-content constraints above.
Unrestricted eval as a model-callable tool is a local-experiment-only feature. Production versions whitelist operations and validate arguments.

DEV Community: heavy shield

I Made Spider-Man Swing Without Animating a Single Frame

Introduction

Problem

Solution

Implementation

Python Agent (server.py)

Browser Client (index.html)

Running It

Key Takeaways

Your API's Newest Users Are Agents: Designing for Non-Human Clients

Introduction

Problem

Solution

Implementation

The API

The Agent Loop

Security Notes

Key Takeaways

What If Your AI Agent Never Had to Leave the Browser?

Introduction

Problem

Solution

Implementation

1. HTML shell

2. The agent module (agent.py)

3. What actually happens

4. Swapping in a real LLM

Key Takeaways

Running an AI Agent Inside the Browser with Pyodide and Ollama

Introduction

The Problem

The Solution

Implementation

Step 1: A tool registry in the browser

Step 2: Agent loop in Python

Step 3: Load Pyodide and run

Key Takeaways

2. The agent module (`agent.py`)