DEV Community: Mikhail Angelov

Under the Hood: Teaching an LLM to Draw in Excalidraw — an Agent Loop in 100 Lines

Mikhail Angelov — Wed, 08 Jul 2026 07:48:30 +0000

The Setup

One day I asked ChatGPT to draw an org chart. Do you know what it did? It gave me ASCII art:

        CEO
       /   \
   VP Eng  VP Prod
    /  \      |
  FE   BE    QA

Cute, but not what I wanted.

I wanted a real diagram: rectangles, arrows, colors, zoom, and the ability to move elements around manually. Not an image embedded in a chat response, not ASCII art, but a proper interactive canvas.

I could have reached for LangChain, MCP, a ready-made agent framework, and hidden everything behind several layers of abstractions. But I was interested in a different question: what is the minimum needed to turn an LLM into an agent that does not just answer with text, but changes the state of the outside world?

In my case, that “outside world” was an Excalidraw canvas.

I wrote a small agent loop from scratch. Important clarification: not the whole project, of course, but the agent core itself — roughly 100 lines. Around it there is a server, a model layer, WebSocket sync, a UI, and a lot of small engineering details. But the actual “agentic” part lives in that loop. No LangChain, no MCP, no special agent libraries.

Spoiler: the agent did not just draw the org chart. In one run, it drew the chart, inspected the result, noticed that the arrows were wrong, deleted them, recreated them with proper bindings, and only then said “done”.

Here is how it works under the hood.

The Idea: Don’t Ask for an Image, Give the Model Tools

A normal chat with an LLM looks like this:

user prompt -> LLM -> text answer

An agent loop adds one important ingredient: tools.

In other words, we do not ask the model “draw me a diagram” and hope that it somehow magically creates an image. We give it functions. For example:

get_scene — inspect what is already on the canvas;
batch_create_elements — create a batch of elements at once;
update_element — fix or adjust the result;
done — finish the task.

This is not the full list, just an illustration of the idea. I will show the complete tool set later.

The model does not draw by itself. It says: “call batch_create_elements with these arguments”. My code executes that call and changes the canvas state.

The Agent Core

In simplified form, the whole agent loop looks like this:

import { runTool, TOOLS } from './tools.js'

export async function agent(prompt: string, sessionId = 'default') {
  const log: string[] = []
  const messages: any[] = [
    { role: 'system', content: systemPrompt },
    { role: 'user', content: prompt },
  ]

  for (let turn = 0; turn < 15; turn++) {
    const response = await fetch(LLM_URL, {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${LLM_KEY}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({
        model: LLM_MODEL,
        messages,
        tools: TOOLS.map((tool) => ({
          type: 'function',
          function: {
            name: tool.name,
            description: tool.description,
            parameters: tool.parameters,
          },
        })),
        tool_choice: 'auto',
      }),
    })

    const data = await response.json()
    const msg = data.choices[0].message
    // msg may contain plain text, tool_calls, or both
    messages.push(msg)

    if (isDone(msg)) {
      return { turns: turn, log }
    }

    for (const toolCall of msg.tool_calls || []) {
      const args = JSON.parse(toolCall.function.arguments || '{}')
      const result = await runTool(toolCall.function.name, args, sessionId)

      messages.push({
        role: 'tool',
        tool_call_id: toolCall.id,
        content: JSON.stringify(result),
      })
    }
  }

  return { turns: 15, log }
}

This is not the production-ready version. The real code also has a timeout, HTTP status checks, LLM error handling, and safe JSON parsing. But the essence is exactly this.

There are three steps in the loop:

Send the LLM the conversation history and the tool descriptions.
Receive the model’s response: plain text, tool calls, or both.
Execute the tool calls and add their results back into the message history.

After that, the model receives a new context and can take the next step.

That is almost all the “magic” of agentic behavior.

How the Model Knows Which Tools Exist

Each tool is described in a format similar to JSON Schema. For example, creating an element:

{
  name: 'create_element',
  description: 'Create a new Excalidraw element. For arrows, use startElementId/endElementId to bind to shapes.',
  parameters: {
    type: 'object',
    properties: {
      type: {
        enum: ['rectangle', 'ellipse', 'diamond', 'arrow', 'text', 'line']
      },
      x: { type: 'number' },
      y: { type: 'number' },
      width: { type: 'number' },
      height: { type: 'number' },
      text: { type: 'string' },
      backgroundColor: { type: 'string' },
      strokeColor: { type: 'string' },
      startElementId: { type: 'string' },
      endElementId: { type: 'string' },
    },
    required: ['type', 'x', 'y'],
  },
}

The LLM sees this description and understands which arguments it can pass. For example:

{
  "type": "rectangle",
  "x": 320,
  "y": 80,
  "width": 160,
  "height": 70,
  "text": "CEO",
  "backgroundColor": "#a5d8ff"
}

The model does not need to know the full internal Excalidraw format. It works through a more convenient tool layer, and the server turns those parameters into real canvas elements.

How the Model “Sees” the Canvas

The important part: the model does not see the image.

When I write that “the model sees” or “the model understands”, that is, of course, a readability shortcut. It does not look at the canvas with eyes. It receives a textual description of the scene:

[rectangle] "CEO" at (350, 50)
[rectangle] "VP Engineering" at (130, 230)
[rectangle] "VP Product" at (570, 230)
[arrow] from CEO to VP Engineering
[arrow] from CEO to VP Product

This is what the get_scene tool is for. When the model calls it, the server walks through the elements in the current session and returns a compact description.

This gives us a simple feedback loop:

draw -> get_scene -> analyze -> fix -> get_scene -> done

This is the step that turns a JSON generator into an agent. Without feedback, the model just fires once with a set of elements. With feedback, it can inspect the result and adjust it.

Why `batch_create_elements` Matters More Than It Seems

At first I thought a single create_element tool would be enough. The model would create elements one by one: a rectangle, some text, an arrow, another rectangle, another arrow.

That quickly becomes inconvenient:

too many round trips to the LLM;
more chances to make a mistake halfway through;
harder to link elements together;
diagrams take longer to create.

So I added batch_create_elements: the model can send the whole set of elements in a single tool call.

For example:

{
  "elements": [
    { "id": "ceo", "type": "rectangle", "x": 350, "y": 50, "text": "CEO" },
    { "id": "eng", "type": "rectangle", "x": 150, "y": 220, "text": "VP Engineering" },
    { "id": "prod", "type": "rectangle", "x": 550, "y": 220, "text": "VP Product" },
    { "type": "arrow", "startElementId": "ceo", "endElementId": "eng" },
    { "type": "arrow", "startElementId": "ceo", "endElementId": "prod" }
  ]
}

This improves the quality of the result a lot. The model stops thinking in individual shapes and starts operating on the diagram as a whole: structure, connections, and relative layout all at once.

The Most Annoying Detail: Arrows

Creating a rectangle is easy. Creating text is easy. Even the background color is easy.

Arrows were the annoying part.

If you simply create a line from point A to point B, it may look fine visually. But it is not a real Excalidraw arrow bound to shapes. If you later drag a rectangle, the arrow stays where it was.

I wanted the agent to create diagrams that could still be edited manually. So arrows had to be bound to elements.

For that, the model passes not start and end coordinates, but startElementId and endElementId:

{
  "type": "arrow",
  "startElementId": "ceo",
  "endElementId": "vp-eng"
}

The server then calculates points on the edges of the shapes and fills in Excalidraw bindings:

function edgePoint(el, targetX, targetY, gap) {
  const cx = el.x + el.width / 2
  const cy = el.y + el.height / 2
  const dx = targetX - cx
  const dy = targetY - cy

  const hw = el.width / 2
  const hh = el.height / 2

  const scale = Math.abs(dx) * hh > Math.abs(dy) * hw
    ? hw / Math.abs(dx)
    : hh / Math.abs(dy)

  return {
    x: cx + dx * scale + (dx > 0 ? gap : -gap),
    y: cy + dy * scale + (dy > 0 ? gap : -gap),
  }
}

For a rectangle, this finds the intersection of the line “shape center -> target” with the rectangle edge. For ellipses and diamonds, the formulas are slightly different; I omitted those branches here for brevity. The idea is the same: do not force the model to calculate geometry. Hide that inside the tool.

This is an important principle: a good tool should be convenient for the model, not for the internal format of the library.

What the Agent Can Do

In the current version, the agent has around two dozen tools. It can:

create elements;
create elements in batches;
read the scene;
update elements;
delete elements;
group and ungroup;
align and distribute;
lock elements;
create a scene snapshot;
restore a snapshot;
control the viewport;
read a small design guide;
finish the task with done.

But in practice, it mostly uses four or five tools:

get_scene — inspect what is already on the canvas;
batch_create_elements — draw the main diagram;
create_element — add a single element or arrow;
update_element — fix the result;
done — finish.

That is also a useful lesson: the model does not need a huge API. It needs a small set of well-described actions.

The Agent That Fixed Itself

The most interesting moment happened while testing a three-level org chart.

I asked the agent to draw a company structure. In that run, the signal came from the textual scene: after calling get_scene, the model saw that arrows existed as standalone elements, but did not look like bound relationships between shapes via startElementId and endElementId. So it decided to delete them and recreate them as properly bound arrows.

The log looked roughly like this:

🤖 Let me check the current scene
  🛠 get_scene({})

🤖 Canvas is empty. Creating org chart...
  🛠 batch_create_elements({ 6 rectangles, 5 arrows })

🤖 The arrows are wrong — they're not bound to shapes. Let me fix that.
  🛠 delete_element({ id: "arrow-1" })
  🛠 delete_element({ id: "arrow-2" })
  🛠 delete_element({ id: "arrow-3" })
  🛠 delete_element({ id: "arrow-4" })
  🛠 delete_element({ id: "arrow-5" })

🤖 Recreating arrows with proper bindings
  🛠 create_element({ type: "arrow", startElementId: "ceo", endElementId: "vp-eng" })
  🛠 create_element({ type: "arrow", startElementId: "ceo", endElementId: "vp-prod" })
  🛠 create_element({ type: "arrow", ... })
  🛠 create_element({ type: "arrow", ... })
  🛠 create_element({ type: "arrow", ... })

🤖 Let me verify the result
  🛠 get_scene({})

🤖 Done! Here's your org chart.

Of course, this does not mean the model will always do this. It is not a formal guarantee, and it is not magic.

But it shows an important thing: if you give the model tools, execution results, and a way to inspect state, it can sometimes behave like a simple autonomous worker:

inspect -> act -> verify -> fix -> finish

To me, this is where the line between “an LLM with functions” and “an agent” starts to appear. Not in the number of frameworks, but in the presence of a feedback loop.

Where the Real Problems Started

The agent core is small. That does not mean the whole project is simple.

The complexity just moves out of the agent loop and into other places.

1. Scene Format Matters More Than It Seems

The model does not see the canvas. It only sees what get_scene returns. If the description is too poor, the model cannot understand what is wrong. If it is too verbose, the context gets polluted quickly.

You need a format that is informative enough without turning into a full Excalidraw JSON dump.

2. Tool Descriptions Are Part of the UX

Tools are not only for your code. They are also for the model.

A bad tool description is like a bad UI for a human. The model will call the function incorrectly, confuse arguments, or forget important fields.

For example, for arrows I had to explicitly write: “use startElementId/endElementId to bind to shapes”. Without that hint, the model often tried to draw arrows using coordinates.

3. Batch Operations Are Almost Mandatory

If the agent draws a large diagram one element at a time, it burns too many iterations. That is why batch_create_elements became one of the most important tools.

4. Real State Must Live on the Server

If the canvas is open in the browser, the agent runs on the server, and the elements exist only in frontend state, things break quickly.

That is why the project has a simple model layer: sessions, elements, snapshots, and WebSocket notifications. This is no longer “an agent in 100 lines”, but without it the agent has nothing stable to work with.

5. A Public Demo Is Dangerous Without Limits

If /api/agent is exposed to the internet, anyone can start burning your LLM API key. I will not go deep into security here — that is a separate article — but the minimum checklist is:

token guard;
rate limit;
prompt length limit;
timeout on LLM requests;
careful error handling.

This part is boring, but you cannot safely put this kind of project on a VPS without it.

Bonus: What If the Agent Looks at Itself?

I tried adding this line to the system prompt: “You are an AI agent that draws diagrams. Your TypeScript code lives in src/agent.ts.”

The response was funny. The agent answered in English; the meaning was roughly:

I am an AI agent running in a loop. I have a set of tools, and I call functions to draw diagrams. My code is in agent.ts: I could read it and explain how I work, but I would rather just draw the diagram.

It did not start philosophizing about itself. It kept working on the task.

In a way, that is the right behavior. A good agent does not need to reflect on its own nature. It should move the state toward the goal.

The Takeaway

This project is not really about Excalidraw.

Excalidraw is just a convenient sandbox where you can immediately see the result of the agent’s actions. The same approach could be used for GitHub issues, the filesystem, CI/CD, a CRM, a calendar, or almost anything else.

Here are my main takeaways.

1. The Agent Loop Is Simpler Than It Seems

In minimal form, it is this:

There is no mandatory hidden magic inside.

2. Tools Matter More Than the Framework

LangChain, MCP, and other tools can be useful. But agentic behavior does not appear because of them. It appears when the model has clear actions and feedback.

3. The Model Can Fix Mistakes If It Can See the Result

Not always. Not perfectly. But it can.

If the agent can inspect the state after its own actions, it gets a chance to notice a problem and make the next iteration better.

4. “100 Lines” Is Only the Core

Yes, the agent loop really is small. But the whole project is larger: model layer, tools, WebSocket, UI, Excalidraw element format, arrow binding, deployment, and safety limits.

And that is fine. The agent loop is small. The useful environment around it is not.

Code

The project is open source:

https://github.com/mikhail-angelov/excalidraw-agent

It is based on mcp_excalidraw by yctimlin: I reused ideas around the canvas, the UI, and the element format. I wrote the agent loop from scratch to understand what is really happening under the hood.

React Under the Hood: Building a Tiny Virtual DOM Renderer

Mikhail Angelov — Tue, 07 Jul 2026 13:30:29 +0000

React looks magical until you remove the parts that make it production-ready.

JSX is just function calls. Components are just functions. Virtual DOM nodes are just plain JavaScript objects. Rendering is "just" comparing two trees and patching the DOM.

Of course, real React is much more than that: hooks, scheduling, Fiber, context, refs, synthetic events, hydration, concurrent rendering, and many other things. This article is not about cloning React. The goal is much smaller: to build a tiny React-like renderer from scratch and understand what happens under the hood.

A while ago, I started wondering: why are there so many UI frameworks for the web?

I've been in IT for a long time, and I don't remember UI libraries on other platforms appearing and disappearing as quickly as they do on the web. Desktop libraries like MFC, Qt, and WPF were huge systems that evolved over many years and had relatively few alternatives. On the web, frameworks appear constantly, and the leaders keep changing.

Why?

I think one reason is that the barrier to writing a UI library has dropped dramatically.

Building a library that many people will actually use still takes significant time and expertise. But writing a prototype that is usable for simple demos, especially if it has a decent API, can take surprisingly little time.

If you're curious how, read on.

Why This Article?

There used to be a series on Habr called "Write X in 30 lines of JavaScript." I wondered: can you write React in 30 lines?

I couldn't quite make it in 30, but the final result is close enough.

The goal here is purely educational. This article may help you understand how virtual DOM-based UI frameworks work at a deeper level. I want to show that the basic idea behind another React-like UI renderer is surprisingly accessible.

Before we start, let me clarify what I mean by "UI framework." Opinions vary. Some people consider Angular and Ember to be frameworks, while React is "just a library" for the view layer.

Let's use a simple definition:

A UI framework is a library that helps create, update, and delete pages or individual page elements.

In this sense, many wrappers over the DOM API can qualify. The differences come down to the abstraction they provide for DOM manipulation and how efficient those manipulations are.

By that definition, React is definitely a UI framework.

So let's build a tiny React-like renderer.

Virtual DOM: The Core Idea

React popularized the concept of a virtual DOM. In simplified terms:

Nodes of the real DOM are built in correspondence with nodes of a pre-built virtual DOM tree.

Direct manipulation of the real DOM is discouraged. When changes are needed:

You update the virtual DOM.
The new virtual DOM is compared with the old one.
The differences are collected.
Only the necessary changes are applied to the real DOM.

This can reduce unnecessary DOM operations and makes updates easier to reason about.

The main benefit is not magic performance. The useful idea is the rendering model itself: describe the UI as data, compare two trees, and apply the required DOM changes.

Since the virtual DOM tree is just a regular JavaScript object, it is easy to manipulate: you can create nodes, change them, compare trees, and generate parts of the structure automatically from a declarative syntax such as JSX.

Starting with JSX

Here's what JSX code looks like:

const Component = () => (
  <div className="main">
    <input />
    <button onClick={() => console.log("yo")}>Submit</button>
  </div>
)

export default Component

We want calling Component() to produce a virtual DOM like this:

const vdom = {
  type: "div",
  props: { className: "main" },
  children: [
    { type: "input", props: {}, children: [] },
    {
      type: "button",
      props: { onClick: () => console.log("yo") },
      children: ["Submit"]
    }
  ]
}

Of course, we don't want to write this transformation manually. We can use the jsx-transform library, which converts JSX like this:

jsx.fromString("<h1>Hello World</h1>", {
  factory: "h"
})
// => 'h("h1", null, ["Hello World"])'

All we need is to implement the virtual DOM node constructor h: a function that recursively creates virtual DOM nodes.

In React, the equivalent idea is React.createElement.

Here's a minimal implementation:

export function h(type, props, ...stack) {
  const children = (stack || []).reduce(addChild, [])
  props = props || {}

  return typeof type === "string"
    ? { type, props, children }
    : type(props, children)
}

function addChild(acc, node) {
  if (Array.isArray(node)) {
    acc = node.reduce(addChild, acc)
  } else if (node == null || node === true || node === false) {
    // skip null/true/false — they render nothing
  } else {
    acc.push(typeof node === "number" ? String(node) : node)
  }

  return acc
}

The recursion makes the function a little dense, but the idea is simple:

flatten nested children;
ignore null, true, and false;
convert numbers to strings;
return either a DOM-like virtual node or a component result.

With this function:

h("h1", null, ["Hello World"])
// => { type: "h1", props: {}, children: ["Hello World"] }

The same works for nested nodes of any depth.

Now our Component function returns a virtual DOM node.

On to the "hard" part.

The Patch Function: Diffing Virtual DOM

We need a function patch that takes:

a root DOM element where the app is mounted;
the old virtual DOM tree;
the new virtual DOM tree.

It then updates the real DOM to match the new virtual DOM.

I based this on code from picodom:

export function patch(parent, oldNode, newNode) {
  return patchElement(parent, parent.children[0], oldNode, newNode)
}

function patchElement(parent, element, oldNode, node, isSVG, nextSibling) {
  if (oldNode == null) {
    // No old node — create a new element.
    element = parent.insertBefore(createElement(node, isSVG), element)
  } else if (node.type !== oldNode.type) {
    // Node type changed — replace it entirely.
    const oldElement = element
    element = parent.insertBefore(createElement(node, isSVG), oldElement)
    removeElement(parent, oldElement, oldNode)
  } else {
    // Same type — update properties.
    updateElement(element, oldNode.props, node.props)

    isSVG = isSVG || node.type === "svg"

    // Collect current child nodes.
    const childNodes = []
    ;(element.childNodes || []).forEach(el => childNodes.push(el))

    let oldNodeIndex = 0

    // Process children.
    if (node.children && node.children.length > 0) {
      for (let i = 0; i < node.children.length; i++) {
        if (
          oldNode.children &&
          oldNodeIndex <= oldNode.children.length &&
          (
            (
              node.children[i].type &&
              node.children[i].type === oldNode.children[oldNodeIndex].type
            ) ||
            (
              !node.children[i].type &&
              node.children[i] === oldNode.children[oldNodeIndex]
            )
          )
        ) {
          // Child exists in the old tree — patch it.
          patchElement(
            element,
            childNodes[oldNodeIndex],
            oldNode.children[oldNodeIndex],
            node.children[i],
            isSVG
          )

          oldNodeIndex++
        } else {
          // New child — insert it.
          const newChild = element.insertBefore(
            createElement(node.children[i], isSVG),
            childNodes[oldNodeIndex]
          )

          patchElement(element, newChild, {}, node.children[i], isSVG)
        }
      }
    }

    // Remove extra old children.
    for (let i = oldNodeIndex; i < childNodes.length; i++) {
      removeElement(
        element,
        childNodes[i],
        oldNode.children ? oldNode.children[i] || {} : {}
      )
    }
  }

  return element
}

This is a naive implementation. It is terribly unoptimized and does not use element identifiers such as key or id for correct list reconciliation.

But for primitive cases, it works fine.

I won't list the helper functions createElement, updateElement, and removeElement here. They are straightforward, and you can find them in the full source.

One Important Caveat

When updating value on <input> elements, the comparison should be made against the real DOM element's current value property, not only against the old vnode value.

Otherwise, a re-render may overwrite what the user is actively typing and break the cursor position or text selection.

This is one of those small details that makes UI rendering much less trivial than it first appears.

Putting It All Together

Now let's assemble the pieces.

The entire renderer API fits in 5 lines:

export function app(selector, view, initProps) {
  // 1. Mount point.
  const rootElement = document.querySelector(selector || "body")

  // 2. Build initial virtual DOM.
  let node = view(initProps)

  // 3. Mount. The old node is null.
  patch(rootElement, null, node)

  // 4. Return a re-render function.
  return props => patch(rootElement, node, (node = view(props)))
}

That is the whole rendering loop.

A "Hello World" example on top of this renderer looks like this:

import { h, app } from "../src/index"

function view(state) {
  return (
    <div>
      <h2>{`Hello ${state}`}</h2>
      <input value={state} oninput={e => render(e.target.value)} />
    </div>
  )
}

const render = app("body", view, "world")

This tiny library supports component composition and runtime adding/removing of components, so it is enough to demonstrate the core rendering idea.

It is not a complete framework in the practical sense. It is a tiny React-like renderer that demonstrates the core pipeline:

JSX → function calls → virtual nodes → diff → DOM patching

You can see a more complex ToDo example.

What's Missing

Of course, this library lacks a lot:

Lifecycle events — they would not be hard to add, since we control node creation, update, and removal ourselves.
Per-component state updates — something like this.setState would require storing DOM references or instance state per virtual node, which complicates the logic.
Performance — the patchElement code is terribly unoptimized and will not scale to large element trees.
Key-based reconciliation — there is no proper tracking of elements by identifier.
Hooks, refs, context, scheduling, hydration, and Fiber — all the things that make real React a production-grade library.

Conclusion

This library was built for educational purposes only.

Do not use it in production.

But it demonstrates that the core ideas behind React and similar virtual DOM frameworks are surprisingly accessible.

JSX transforms to function calls. Function calls build a tree of plain objects. A diffing function walks that tree and applies the required updates to the real DOM.

The rest — state management, lifecycle, scheduling, Fiber architecture, ecosystem, tooling, and years of edge cases — is engineering on top of this foundation.

P.S. This article was inspired by the excellent Hyperapp library. Some code was adapted from it.

Happy coding!