How to Build 3D & AR Apps with AI — Cursor, Windsurf, Claude Code

#ai #cursor #android #webdev

AI coding assistants are great at generating UI code. But ask them to build a 3D scene or an AR experience, and they usually hallucinate APIs that don't exist.

That changes with MCP servers (Model Context Protocol). MCP gives your AI assistant access to real SDK documentation and validated code patterns.

Here's how to set it up for 3D/AR development.

The Problem

Ask any AI to "display a 3D model in Jetpack Compose" and you'll get:

References to deprecated Sceneform (dead since 2021)
Three.js code when you asked for Android
Made-up Compose APIs that don't compile
Unity boilerplate when you wanted a lightweight solution

The Solution: MCP Servers

MCP servers give AI assistants access to specialized tools. sceneview-mcp provides:

Complete API reference for SceneView (Android, iOS, Web)
Code generation with validation
Model search (Sketchfab integration)
Project analysis

Setup

Claude Code / Claude Desktop

claude mcp add sceneview -- npx sceneview-mcp

Cursor

Add to .cursor/mcp.json:

{
  "mcpServers": {
    "sceneview": {
      "command": "npx",
      "args": ["-y", "sceneview-mcp"]
    }
  }
}

Windsurf

Add to Windsurf MCP settings:

{
  "mcpServers": {
    "sceneview": {
      "command": "npx",
      "args": ["-y", "sceneview-mcp"]
    }
  }
}

What You Can Build

Once configured, ask your AI:

"Build me a 3D model viewer for Android"
"Create an AR furniture placement app"
"Add a 3D product configurator with color switching"
"Build an anatomy viewer for medical education"

The MCP server feeds correct API docs to the AI, so the generated code actually compiles.

Specialized MCP Servers

For domain-specific development, use these specialized servers:

Server	Domain	Install
`sceneview-mcp`	General 3D & AR	`npx sceneview-mcp`
`automotive-3d-mcp`	Car configurators	`npx automotive-3d-mcp`
`healthcare-3d-mcp`	Medical visualization	`npx healthcare-3d-mcp`
`gaming-3d-mcp`	Game prototyping	`npx gaming-3d-mcp`
`interior-design-3d-mcp`	Room planning	`npx interior-design-3d-mcp`

Quick Example

After setup, here's what happens when you ask Claude Code to "build an AR app":

// AI generates this — and it compiles on first try
@Composable
fun ARScreen() {
    val engine = rememberEngine()
    val modelLoader = rememberModelLoader(engine)
    var anchor by remember { mutableStateOf<Anchor?>(null) }

    ARSceneView(
        modifier = Modifier.fillMaxSize(),
        engine = engine,
        modelLoader = modelLoader,
        planeRenderer = true,
        onSessionUpdated = { _, frame ->
            if (anchor == null) {
                anchor = frame.getUpdatedPlanes()
                    .firstOrNull { it.type == Plane.Type.HORIZONTAL_UPWARD_FACING }
                    ?.let { frame.createAnchorOrNull(it.centerPose) }
            }
        }
    ) {
        anchor?.let { a ->
            rememberModelInstance(modelLoader, "models/chair.glb")?.let { m ->
                AnchorNode(anchor = a) {
                    ModelNode(modelInstance = m, scaleToUnits = 0.5f)
                }
            }
        }
    }
}