DEV Community: Muhammed Enes Duran

Why AI-Generated GIS Code Can Run Successfully and Still Be Wrong

Muhammed Enes Duran — Thu, 23 Jul 2026 15:01:21 +0000

Geospatial code has an unusual failure mode: it can execute without errors, produce a clean-looking result, and still be methodologically wrong.

This problem is becoming more important as AI coding agents generate complete GIS workflows from short natural-language instructions.

The issue is not always that an AI agent does not know GeoPandas, rasterio, PostGIS, Earth Engine, or ArcPy. In many cases, it knows the correct API and can produce executable code.

The problem is that API correctness is not the same as geospatial correctness.

A silent CRS failure

Consider a simple buffer operation:

`buffered = gdf.buffer(1000)`

The code looks reasonable. It runs successfully and produces new geometries.

But what does 1000 mean?

When the dataset uses a geographic coordinate reference system such as EPSG:4326, its coordinate units are degrees rather than metres. The operation may still return valid geometries, but they do not represent a 1,000-metre buffer.

A safer workflow makes the spatial assumptions explicit:

if gdf.crs is None:
    raise ValueError("The input dataset has no CRS.")

if gdf.crs.is_geographic:
    projected_crs = gdf.estimate_utm_crs()

    if projected_crs is None:
        raise ValueError("A suitable projected CRS could not be determined.")

    gdf = gdf.to_crs(projected_crs)

buffered = gdf.buffer(1000)

Even this pattern is not universally correct. A locally estimated UTM projection may be unsuitable for datasets that span multiple zones, countries, or continents.

The important point is not one specific code snippet. A defensible workflow must ask:

Is the CRS defined?
Are the coordinate units appropriate?
Is the projection suitable for the geographic extent?
How will the output be verified?

Without those checks, plausible output can hide an invalid spatial operation.

A machine-learning result that looks better than it is

Spatial machine learning has a similar problem.

A common model evaluation starts with a random train/test split:

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42,
)

For independent observations, this can be a reasonable starting point.

Spatial observations, however, are often correlated with nearby observations. A random split can place neighbouring samples in both the training and test sets.

The model then appears to generalize, while it may only be exploiting local spatial similarity.

This can produce an impressive accuracy score that collapses when the model is applied to a genuinely new region.

A more defensible evaluation may require:

spatial blocks or geographic groups
region-level holdouts
distance-based separation
temporal separation for future prediction
maps of residuals and prediction errors
comparison between random and spatial validation

Spatially blocked validation often produces a lower score.

That lower score may be the more honest result.

Rendering is not verification

Maps introduce another form of silent failure.

A map can render correctly while using:

an unsuitable projection
misleading classification boundaries
inconsistent scales across dates
inaccessible colour choices
unreported missing values
geometries silently removed during processing

A change-detection map can look convincing even when the source rasters are misregistered, collected in different seasons, or processed at incompatible levels.

Successful rendering proves that the software produced an image.

It does not prove that the analysis is valid.

What an AI agent needs beyond API knowledge

A general-purpose coding agent may know which function to call. It does not necessarily know when a workflow should stop.

For geospatial tasks, an agent needs explicit operational rules:

Never calculate area or distance in a geographic CRS.
Treat spatial leakage as a default risk in predictive modelling.
Validate geometries and account for input and output row counts.
Preserve georeferencing through raster and deep-learning pipelines.
Report uncertainty when the analytical method supports it.
Verify outputs numerically and visually.
Refuse invalid spatial or temporal comparisons instead of producing a misleading result.

These are not optional improvements added after the analysis.

They are part of the analytical method itself.

Building GeoAI Skills

I built GeoAI Skills to encode these safeguards as reusable Agent Skills.

The current public preview contains 18 skills covering:

geospatial data engineering
PostGIS and spatial SQL
remote sensing
Google Earth Engine
geospatial deep learning
spatial statistics
geostatistics and interpolation
terrain and hydrology
suitability analysis
point clouds and LiDAR
network accessibility
movement trajectories
change detection
cartography and geovisualization
guarded ArcGIS Pro automation
machine-learning experiment standards
software engineering and DevOps practices
multi-stage GeoAI workflow orchestration The objective is not to make an AI agent sound like a GIS expert.

The objective is to make assumptions visible, require verification, and fail loudly when a workflow is not defensible.

A land-suitability request, for example, may involve data preparation, remote sensing, terrain analysis, multi-criteria decision analysis, and cartographic delivery.

GeoAI Skills uses a central orchestrator to route each stage to the relevant specialist skill while maintaining shared safeguards such as CRS checks, leakage prevention, uncertainty reporting, and verification.

Guarded ArcGIS Pro automation

The suite also includes an arcgis-pro-automation skill.

It works with my open-source arcgis-mcp-bridge project to support controlled local ArcPy execution.

*The integration covers workflows involving:
*

ArcGIS Pro projects
file and enterprise geodatabases
vector and raster geoprocessing
spatial and statistical analysis
network analysis
maps and layouts

Because ArcPy operations can modify real projects and datasets, the execution model includes path restrictions and confirmation gates for mutating or destructive actions.

The goal is not unrestricted automation.

It is controlled automation with explicit boundaries.

Testing whether the correct skills activate

A skill is not useful if the agent does not activate it for the right request—or activates it for unrelated requests.

The repository currently contains 131 typed evaluation scenarios across the 18 skills.

These scenarios include:

positive activation cases
negative cases
ambiguous requests
collisions between related skills
artifact-oriented evaluations

For a frozen 17-skill, 120-case routing suite using Claude Code 2.1.214 and Claude Sonnet 5, the published results were:

100% routing precision
92.86% routing recall
92.5% full-route accuracy

These results have an important limitation.

They measure whether the expected skills were selected for that exact runtime, model, and evaluation suite. They do not prove that every generated analysis is correct, and they are not universal compatibility or answer-quality claims.

The eighteenth skill, arcgis-pro-automation, was added after the frozen benchmark and is not included in those headline figures.

Routing evidence, behavioural correctness, and real-world analytical quality should be evaluated separately.

Trying the public preview

The complete suite can be explored and installed interactively with:

npx skills add muend/geoai-skills

A single specialist skill can also be installed independently:

npx skills add muend/geoai-skills --skill postgis-spatial-sql

GitHub repository:

https://github.com/muend/geoai-skills

Skills collection:

https://www.skills.sh/muend/geoai-skills

The project is still an early public preview.

I am particularly interested in reproducible examples of:

GIS code that executed successfully but produced an invalid result
spatial leakage in real machine-learning projects
ArcGIS Pro or ArcPy failure modes that should be guarded explicitly
ambiguous requests that could activate the wrong specialist skill
important verification steps that the current suite does not yet require

What is the most common silent geospatial failure you have encountered?

Building a Secure MCP Bridge for ArcGIS Pro and ArcPy

Muhammed Enes Duran — Wed, 01 Jul 2026 15:44:07 +0000

ArcGIS Pro has a powerful Python runtime through ArcPy, but it is not a lightweight dependency. It is native, licensed, Windows-bound, and tightly coupled to the ArcGIS Pro environment.

That makes it awkward to connect directly to AI tooling.

I built arcgis-mcp-bridge as an independent open-source experiment to solve this problem through the Model Context Protocol (MCP).

GitHub: https://github.com/muend/arcgis-mcp-bridge

PyPI: https://pypi.org/project/arcgis-mcp-bridge/

MCP Registry: io.github.muend/arcgis-mcp-bridge

The problem

ArcPy is extremely useful for GIS automation:

geoprocessing
feature class and geodatabase operations
raster workflows
map and layer management
coordinate reference operations
spatial statistics
network analysis
export and layout workflows

But ArcPy is not a normal pure-Python library. It depends on the licensed ArcGIS Pro Python environment and native Esri runtime components.

For AI/MCP workflows, importing ArcPy directly into the host process is not a clean architecture. If the native runtime crashes, blocks, logs to stdout, or touches the wrong local path, the AI host process is affected too.

So the main design goal was:

Keep the AI/MCP host process lightweight, controlled, and separate from the licensed ArcPy execution runtime.

What arcgis-mcp-bridge does

arcgis-mcp-bridge is a local-first MCP server that exposes ArcGIS Pro’s ArcPy engine over stdio JSON-RPC.

It allows MCP-compatible clients to call a controlled set of ArcPy/geoprocessing tools without importing ArcPy directly into the MCP server process.

The project currently includes:

100 declarative geoprocessing tools
10 GIS verticals
a two-process architecture
ArcPy worker-process isolation
PathGuard filesystem boundaries
confirmation gates for destructive operations
mocked ArcPy tests for CI without an ArcGIS license
optional Sketch-to-GIS / OpenCV pipeline

It is not an official Esri project and not an official Anthropic project. It is an independent open-source bridge for ArcGIS Pro, ArcPy automation, and MCP-based GIS workflows.

Architecture

The project is split into two layers.

Layer A: MCP server

Layer A is the MCP protocol host.

It handles:

stdio JSON-RPC transport
MCP tool registration
Pydantic validation
path validation before worker execution
dispatching jobs to the worker process
returning structured MCP tool results

Layer A does not import ArcPy.

That is intentional. The MCP server should remain lightweight and should not depend on ArcGIS Pro’s native runtime being importable in the same process.

Layer B: ArcPy worker

Layer B is a separate Python process running inside the licensed ArcGIS Pro Python environment.

It handles:

importing ArcPy
executing geoprocessing tools
collecting ArcPy messages
returning one structured JSON result frame

This separation matters because ArcPy can be slow to import, can depend on license state, and can interact with native runtime components. Keeping it in a worker process protects the MCP server process from crashes and native runtime instability.

Why two processes?

The two-process model gives the project a clean failure boundary.

If the ArcPy worker fails, times out, or crashes, the MCP server can return a structured error instead of dying with it.

The basic flow is:

MCP client
   ↓ stdio JSON-RPC
Layer A: MCP server
   ↓ NDJSON subprocess bridge
Layer B: ArcPy worker
   ↓
ArcGIS Pro / ArcPy runtime

This is slower than keeping ArcPy warm in the same process, but it is much safer as a first production boundary.

A future version could add a warm worker pool behind the same execution interface, but the isolation model should remain explicit.

Filesystem safety: PathGuard

GIS workflows often touch sensitive local data:

file geodatabases
enterprise connection files
infrastructure datasets
cadastral data
client project folders
personal or proprietary spatial datasets

Because of that, the bridge uses a PathGuard boundary.

Each filesystem-touching tool declares whether a path is used for:

read
write
read list

The server validates paths before spawning a worker, and the worker validates again before execution.

The goal is not to make arbitrary AI file access convenient. The goal is to make the allowed local execution boundary narrow, explicit, and auditable.

Destructive operation gates

Some ArcPy operations mutate data.

Examples include append, delete, repair geometry, calculate field, projection definition, and other state-changing workflows.

For those operations, the project requires explicit confirmation in the tool input.

The rule is simple:

Destructive tools must require confirm=true.

This gives both the MCP host and the user a clear point where mutation is intentional.

Testing without ArcGIS Pro in CI

A major practical issue with ArcPy projects is CI.

Hosted CI runners usually do not have ArcGIS Pro installed, and they do not have a licensed ArcPy runtime available.

To keep the project testable, the automated test suite mocks ArcPy and focuses on the parts that can be verified without Esri runtime access:

Pydantic contracts
PathGuard behavior
registry invariants
destructive operation gates
worker error mapping
configuration validation
import boundaries

The current suite contains 81 unit tests and runs without ArcGIS Pro.

Real geoprocessing execution still requires a licensed ArcGIS Pro environment on Windows.

Current tool surface

The project currently exposes 100 tools across 10 GIS verticals:

map and layer management
data management
geometry analysis
coordinate reference and projection
raster operations
vision analytics
export and layout
editing and topology
network analysis
spatial statistics

The optional vision component includes a Sketch-to-GIS pipeline using OpenCV. The idea is to extract hand-drawn boundaries from an image and commit them into a geodatabase workflow.

Installation

The package is available on PyPI:

pip install arcgis-mcp-bridge

The project also includes a setup command for preparing an ArcGIS Pro worker environment:

arcgis-mcp-setup

For vision-related functionality:

pip install "arcgis-mcp-bridge[vision]"

Real ArcPy execution requires a licensed ArcGIS Pro Python environment. The server uses ARCPY_PYTHON_PATH to locate the worker interpreter.

Example MCP use cases

Potential workflows include:

list feature classes inside a file geodatabase
buffer parcels and write outputs to a scratch geodatabase
dissolve features by attribute
run raster slope/aspect analysis
export layouts
inspect geometry errors
run spatial statistics tools
prepare repeatable GIS automation tasks from natural language requests

The point is not to let an AI agent freely mutate GIS projects. The point is to expose a controlled, typed, auditable tool surface around common ArcPy workflows.

Links

GitHub: https://github.com/muend/arcgis-mcp-bridge

PyPI: https://pypi.org/project/arcgis-mcp-bridge/

MCP Registry: io.github.muend/arcgis-mcp-bridge

Esri Community discussion:

https://community.esri.com/t5/python-questions/open-source-mcp-bridge-for-arcgis-pro-arcpy/m-p/1711533#M75310

Feedback wanted

I am especially interested in feedback from GIS developers, ArcPy users, and people building MCP servers.

The main questions are:

Does this architecture make sense for a local licensed runtime like ArcGIS Pro?
Which ArcPy tools or workflows should be prioritized next?
What safety boundaries would be required before using this with real project data?
Which ArcGIS Pro versions should be tested first?
Would a warm worker pool be worth the added complexity, or is spawn-per-call isolation the better default?

The project is open source, and feedback is welcome.