DEV Community

Xu Bian
Xu Bian

Posted on • Originally published at marlinbian-site.pages.dev

SketchUp Agent Harness

#ai #design #sketchup #opensource

SketchUp Agent Harness is an early-stage open-source project for connecting design agents such as Claude Code and Codex CLI to SketchUp. The goal is to let designers express intent in natural language and turn that intent into verifiable, executable, repairable project state.

Project repository:

github.com/marlinBian/sketchup-agent-harness

It is not a plugin for only one AI tool. Claude and Codex are entry points. The more stable core is the MCP server, SketchUp Ruby bridge, structured design model, component metadata, runtime skills, and protocol documentation.

What Problem This Project Tries to Solve

Many AI design tools stop at "generate an image from a prompt" or "operate a few software steps for me." That can be useful for inspiration, but it is not enough for real design work.

Designers need a workflow that can keep going:

  • design intent can be recorded
  • floor plans, images, scans, and other source material can be preserved
  • model state can be checked and repaired
  • software execution can be traced back to structured facts
  • visual feedback can become explicit changes instead of being buried in chat history

The core judgment behind SketchUp Agent Harness is that AI design tools need a structured source of truth. In this project, the key file is design_model.json. SketchUp scenes, screenshots, and renders matter, but they are better understood as execution results or review artifacts. They should not be the only long-term truth of the project.

What It Can Do Today

Based on the current project documentation, the 1.0 version is still early, but it already has several foundational capabilities:

  • initialize an isolated design project directory
  • install the SketchUp Ruby bridge
  • install Claude and Codex runtime skills into the project
  • generate simple spaces and bathroom layouts through natural language
  • import DWG, DXF, PDF, image, scan, or photo inputs into a first editable model
  • store structured design truth in design_model.json
  • preserve source evidence, screenshots, versions, and project rules

The intended workflow is not that designers clone the source repository for normal design work. A more typical path is:

  1. Install the harness package.
  2. Install or update the SketchUp Ruby bridge.
  3. Create a clean design project directory.
  4. Run Claude or Codex in that design project directory.
  5. Describe the design goal in natural language.

Why the Source of Truth Matters

If AI only generates an image, designers have limited ability to keep editing, checking, replaying, or repairing it.

If AI only operates a SketchUp scene directly, many design decisions become hidden inside software state. A later agent may not know what was design intent, what was a temporary action, what came from the original drawing, and what was only inferred.

A structured model makes these things explicit:

  • spaces
  • dimensions
  • components
  • rules
  • assumptions
  • source evidence
  • execution plans
  • review and repair clues

That is what lets the agent support diff, validation, repair, and replay. It also lets designers focus more on intent, judgment, constraints, and tradeoffs instead of repeatedly becoming drafting operators in the AI era.

Architecture

One way to understand SketchUp Agent Harness is as a chain from natural language to SketchUp execution:

designer intent
-> agent CLI
-> runtime skills
-> MCP server
-> structured design model
-> SketchUp Ruby bridge
-> SketchUp scene
-> visual review
-> structured repair
Enter fullscreen mode Exit fullscreen mode

The important part is not any single agent model. The important part is the verifiable protocol layer and the structured project state in the middle.

That is why the project includes:

  • MCP server
  • SketchUp Ruby bridge
  • runtime skills
  • protocol specs
  • spatial behavior specs
  • project workspace docs
  • import pipeline docs
  • visual review loop docs

Current Limits

This project should not be described as a mature commercial design platform.

A more accurate description is: an early-stage open-source harness for exploring how natural-language agents can control SketchUp reliably and organize the design process into a checkable structured workflow.

Floor plan import is especially important to frame correctly. The first generated model should be treated as an editable working model, not a survey-grade result. Real component libraries, more complex collision checks, more stable drawing recognition, and fuller design rules still need continued iteration.

What I Plan to Write Next

This project is also the starting point for a set of essays and demos.

The first thread is source of truth:

Why AI design tools should not only pursue "it looks generated," and why they need editable, repairable, replayable structured truth.

The second thread is the designer workflow:

Designers should spend more energy on ideas, judgment, constraints, and tradeoffs instead of manually executing every low-level modeling step.

The third thread is implementation:

How Codex or Claude can use runtime skills, an MCP server, a SketchUp bridge, and design_model.json to turn natural language into verifiable SketchUp project state.

I also plan to produce demo videos based on this project, using a public-safe example to show the full path from natural-language goal to SketchUp model, review, and repair.

Originally published on my personal site:
https://marlinbian-site.pages.dev/projects/sketchup-agent-harness/

More links: GitHub · YouTube · LinkedIn · Bluesky · Mastodon · Discord

Top comments (0)