I'm an architect. I built an AI agent that draws - and outputs IFC models

Axidesk — Sat, 19 Sep 2026 22:41:29 +0000

Every coding agent I tried could write files. None of them could use the computer.

I'm an architect. I don't write code for a living - I design buildings. But I got tired of repeating the same drawings, the same spreadsheets, the same model exports by hand. So I built one: Axio Coder, an open-source (AGPL-3.0) agent that works inside your project, next to you.

Here is the part that turned out to be hard.

The thing almost nobody does: operating native programs

Any agent can read and write files in a repository. That is the easy 80%.

The hard 20% is a program with no API. A CAD window. Photoshop. MS Paint. These are the programs architects and engineers actually live in, and they expose no endpoint.

Most "computer use" approaches screenshot the screen and guess pixel coordinates. On Windows I took a different route: the UI Automation tree. Every native control announces itself - name, type, value - to the operating system. That is how screen readers work. So instead of guessing where a button is, the agent asks the program: where is the "Fill with colour" button? and gets a real answer from the app itself.

That is how it drew a rocket in Paint, stroke by stroke: it picked the tool, dragged from here to there, and then looked at the screen again before the next stroke.

Eyes change everything

An agent that writes code does not need to see. An agent that draws does.

So this one can look: at the screen, at a PDF page, at an image, at a 3D model rendered from any angle - and it can compare what it produced against the reference, side by side.

That closed the loop for me: generate, look, correct, repeat. The agent builds the geometry, renders it, looks at it, and says honestly what is still wrong. Which is how a person works.

It outputs engineering models, not just files

This is the part I have not seen elsewhere.

Give it a reference - a photo, a drawing, a spec with dimensions - and it builds the parametric model, then writes it out in the formats the industry actually uses: IFC, DXF, STEP. Real geometry, measured back from the file it just wrote and checked against the formula that was supposed to produce it.

The generated folder in the repository holds three IFC revisions of the same piece. Each one was looked at and corrected. That is the loop, in public.

The point is not "AI makes 3D". The point is that the output leaves the chat window and lands in ArchiCAD, inside a BIM workflow - where the work continues.

The boring parts that matter

Memory. Notes and past sessions are indexed and retrieved semantically. It does not start from zero every morning.
The diff. Every edit is shown before and after, changed lines highlighted - one click away, from the log or from the session history.

The terminal. A live shell plus a card per process, so you can watch what it runs.
It watches itself. When an agent has tools it will eventually loop. If this one repeats the same call with the same arguments, it stops and changes approach instead of insisting.

What it is not good at

I would rather say it now than have you find out later.

It is a single-user desktop app (Electron + Python), built for me first. It has rough edges. Heavy CAD geometry fails sometimes and the agent has to notice and back off. And it will not replace your CAD software - it does the tedious 80% so you can spend your time on the 20% that needs judgement.

Where this is going

The piece I am working on now is the agent operating other programs in sequence on its own: open the program, do the work, export, look at the result, move on. After that, modelling from a reference image end to end, with me no longer in the middle.

If you are an architect, an engineer, or someone who automates their own work, I would genuinely like to know where this breaks for you.

Repository: github.com/Axidesk/Axio-Coder - AGPL-3.0