DEV Community: Jan Tschada

Surviving Ubuntu 26.04 LTS: Fixing Qt 6 and ArcGIS Maps SDK for Qt on Wayland

Jan Tschada — Sat, 19 Sep 2026 06:00:00 +0000

The Upgrade Trap

If you are a Qt developer working with geospatial data, upgrading to Ubuntu 26.04 LTS (Resolute Raccoon) might have felt less like an upgrade and more like a system break.

We recently migrated our geospatial intelligence stack to the new LTS. Our goal was simple: run a Qt Quick application using the ArcGIS Maps SDK for Qt, powered by fine-tuned, edge-based geospatial agents for spatial grounding on satellite imagery.

Instead, we hit a brick wall. Here is the exact error we got when trying to run the standard Qt Online Installer:

./qt-online-installer-linux-x64-4.11.0.run: error while loading shared libraries: libxcb-icccm.so.4

What followed was a deep dive into the collision between Ubuntu's aggressive Wayland transition, Qt 6's new runtime dependencies, and missing X11 libraries. Here is how we fixed it, and how you can too.

The Root Cause: Wayland, XCB, and a Hidden Dependency

Ubuntu 26.04 defaults to Wayland. However, the ArcGIS Maps SDK for Qt relies heavily on OpenGL and historically has incomplete native Wayland support.

" width="800" height="567">

On some platforms things went from "rendering issues" to an immediate, hard crash.

To get our map rendering correctly, the standard workaround is to force the X11/XCB backend via XWayland using an environment variable:

export QT_QPA_PLATFORM=xcb

The issue is twofold:

Missing XCB Runtime Libraries: Ubuntu 26.04 no longer installs X11/XCB libraries by default. The Qt installer and the ArcGIS SDK need them.
Qt 6 Hard Dependency: Since Qt 6.5+, the xcb platform plugin has a hard runtime dependency on libxcb-cursor0. If this library is missing, the plugin fails to load, and Qt aborts the process immediately.

The Fix: A Step-by-Step Survival Guide

If you are developing Qt applications on Ubuntu 26.04, follow these steps to stabilize your environment.

Step 1: Install the Complete XCB Runtime Set

Don't just install libxcb-cursor0. Install the full suite of libraries that Qt and the ArcGIS SDK expect. This prevents you from playing "whack-a-mole" with missing dependencies.

sudo apt update
sudo apt install libxcb-cursor0 libxcb-xinerama0 libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-render-util0 libxcb-shape0 libxcb-composite0

Step 2: Correctly Configure the Qt Platform Variable

If you are launching your app from a terminal or a script, ensure you use a comma, not a semicolon. For the ArcGIS SDK, forcing XCB is the most stable route:

# Force XCB backend for stable ArcGIS rendering
QT_QPA_PLATFORM=xcb ./YourGeospatialApp

If you want to try Wayland first and fall back to XCB, use a comma:

QT_QPA_PLATFORM=wayland,xcb ./YourGeospatialApp

Step 3: Check for Global Configuration Overrides

Some Ubuntu flavors set QT_QPA_PLATFORM globally. Check if your system is forcing a broken configuration:

grep -r "QT_QPA_PLATFORM" /etc/profile /etc/environment /etc/profile.d/

If you find a global setting with a semicolon (e.g., wayland;xcb), that is a bug you need to work around. Override it in your local shell profile (like ~/.bashrc or ~/.zshrc) using the correct comma syntax.

Step 4: Diagnose "Lying" Error Messages

If your app still crashes, Qt might be reporting a missing libxcb-cursor0 error when the actual missing library is something else entirely. Use ldd to find the real culprit:

ldd ./YourGeospatialApp | grep "not found"

The Payoff: Running Geospatial Intelligence on the Edge

After applying these fixes, our environment stabilized. We were able to run our HUD-styled Qt Quick prototype, successfully rendering satellite imagery of Frankfurt Airport via the ArcGIS Maps SDK.

Our prototype: A HUD-styled Qt Quick interface performing spatial grounding using edge-based geospatial agents on satellite imagery.

The combination of Qt 6, ArcGIS, and Ubuntu 26.04 is incredibly powerful for geospatial intelligence (GEOINT) once the environment is configured correctly. It allows us to run fine-tuned agents directly on the edge, processing high-resolution satellite imagery without relying on constant cloud connectivity.

Conclusion

Ubuntu 26.04 LTS is a solid release, but its aggressive move to Wayland has left the X11-dependent Qt ecosystem in a state of flux. By manually installing the XCB runtime libraries and correctly configuring the QT_QPA_PLATFORM variable, you can build a stable foundation for your geospatial applications.

Have you run into similar Wayland/X11 issues with Qt 6? Let me know in the comments.

From Traffic Accidents to AI Agents: Building Spatial Decision Intelligence with Python

Jan Tschada — Wed, 09 Sep 2026 19:16:54 +0000

How geospatial analytics, simulation, and AI come together to help us understand not just predict our cities

The Problem: AI That Doesn't Understand Space

We're in the middle of an AI agent renaissance. Models can reason, plan, invoke tools, and coordinate complex workflows. Yet when these conversations move from impressive demos to real-world operations, a different challenge emerges.

The question is no longer whether an AI agent can perform a task. The more interesting question is whether it can understand the environment in which a decision must be made.

Cities don't operate as collections of independent systems. Mobility, infrastructure, weather, energy, public transportation, construction projects, and human behavior constantly interact. Understanding these interactions requires more than access to data; it requires context. And in many cases, that context is fundamentally geographic.

This became clear while exploring an Urban Digital Twin scenario for Frankfurt am Main. What started as a straightforward investigation into how geospatial analytics, simulation, and AI could work together revealed something much broader: before an AI agent can support meaningful decisions, it must first understand the spatial relationships that shape how a city functions.

Let me walk you through the approach and the Python code that made it possible, drawn directly from the Spatial Data Science Examples repository.

Step 1: Setting Up Your GIS Environment

First, connect to your GIS portal using ArcGIS API for Python. The notebook uses environment variables to keep credentials secure:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

from arcgis.gis import GIS
from arcgis.features import Feature, FeatureSet
from arcgis.geometry import Envelope, Point
from datetime import datetime
import os
import pandas as pd
from dotenv import load_dotenv

# Load environment variables
def load_vars():
    load_dotenv(dotenv_path="../../../../.env", override=True)
    return (
        os.getenv("ARCGIS_API_KEY"),
        os.getenv("TRAFFIC_ACCIDENTS_FEATURES"),
        os.getenv("TRAFFIC_DATA_FILE"),
        os.getenv("TRAFFIC_FEATURES"),
        os.getenv("NETWORK_DATASET"),
        os.getenv("ELECTRIC_CAR_FEATURES")
    )

api_key, traffic_accidents_path, traffic_data_path, traffic_features_path, network_dataset_path, electric_car_path = load_vars()

# Connect to your GIS
gis = GIS(api_key=api_key)

Note: Make sure you have a .env file in your project root with your ARCGIS_API_KEY and other data paths.

Step 2: Visualizing the Area of Interest

Before diving into analysis, it's helpful to visualize your study area. The repository includes helper functions to create interactive maps:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

from urban_traffic.utils import create_traffic_map, create_map

# Create a traffic-focused map view
traffic_map_view = create_traffic_map(gis)
traffic_map_view

# Or create a map centered on a specific location
map_view = create_map(gis, location="Frankenallee 355, Frankfurt am Main, Germany")
map_view

Step 3: Loading and Preparing Traffic Data

The real work begins with loading traffic data. Here's how the notebook fetches and prepares traffic features:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

from data_engineering.utils import (
    fetch_charging_stations,
    fetch_hotcold_features,
    fetch_hottest_features_by_extent,
    fetch_traffic_accidents,
    get_hotcold_layer,
    get_live_traffic_item
)
from urban_traffic.utils import (
    fetch_traffic_data,
    filter_commute_cars,
    generate_car_renderer,
    generate_routes_renderer,
    prepare_traffic,
    prepare_traffic_accidents,
    read_traffic_accidents_features_by_extent,
    read_traffic_features
)

# Fetch traffic features
traffic_features = read_traffic_features(gis, traffic_features_path)

# Filter for commute cars (cars used for commuting)
commute_cars = filter_commute_cars(traffic_features.copy())
commute_cars.head()

The commute_cars DataFrame contains columns like trip, person, hour, minute, second, bike, car, and pedestrian allowing you to filter and analyze different types of road users.

Step 4: Visualizing Traffic Patterns

You can visualize the filtered traffic data directly on a map:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

# Create a map and plot commute car positions
commute_map_view = create_map(gis)
commute_cars.spatial.plot(
    commute_map_view,
    renderer=generate_car_renderer()
)
commute_map_view.zoom_to_layer(commute_cars)
commute_map_view

Step 5: Identifying Hot and Cold Spots

To understand spatial patterns, the notebook uses hot/cold spot analysis:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

# Fetch hot and cold spots from the commute car data
hotcold_spots_fset, drawing_info = fetch_hotcold_features(gis, commute_cars)

# Add them to the map with transparency
commute_map_view.content.add(
    hotcold_spots_fset,
    drawing_info=drawing_info,
    options={"opacity": 0.7}
)

What we found: Major transportation corridors exhibited elevated densities; exactly as expected. But other locations repeatedly appeared despite not being particularly prominent within the network. Only after examining additional context surrounding land use, infrastructure layouts, and transit interactions did plausible explanations emerge.

Step 6: Advanced Spatial Analysis: Closest Facility Routing

The notebook demonstrates a powerful use case: finding the closest charging station for electric vehicles. This same logic applies to any "closest facility" problem finding the nearest hospital, police car, or fire station:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

import arcpy
from arcpy.management import CopyFeatures
from arcpy.na import AddLocations, GetNAClassNames, Solve

arcpy.env.overwriteOutput = True

def create_analysis_layer(charging_stations: pd.DataFrame, car_positions: pd.DataFrame):
    """Create a closest facility analysis layer using a local routing network."""
    analysis_layer_name = "ClosestChargingStation"
    layer_result = arcpy.na.MakeClosestFacilityAnalysisLayer(
        network_dataset_path,
        analysis_layer_name
    )

    # Get the analysis class names ("Facilities", "Incidents", ...)
    analysis_layer = layer_result.getOutput(0)
    analysis_class_names = GetNAClassNames(analysis_layer)

    # Charging stations as facilities
    input_facilities = "memory/facilities"
    charging_stations.spatial.to_featureclass(input_facilities)
    AddLocations(analysis_layer, analysis_class_names["Facilities"], input_facilities)

    # Car positions as incidents
    input_incidents = "memory/incidents"
    car_positions.spatial.to_featureclass(input_incidents)
    AddLocations(analysis_layer, analysis_class_names["Incidents"], input_incidents)

    return analysis_layer

def solve_routing_problem(analysis_layer, nogo_areas: pd.DataFrame = None) -> pd.DataFrame:
    """Solve the routing problem and return the results as a DataFrame."""
    analysis_class_names = GetNAClassNames(analysis_layer)

    if nogo_areas is not None:
        # Add no-go areas as polygon barriers
        input_barriers = "memory/barriers"
        nogo_areas.spatial.to_featureclass(input_barriers)
        AddLocations(analysis_layer, analysis_class_names["PolygonBarriers"], input_barriers)

    # Solve the routing problem
    solve_result = Solve(analysis_layer)

    # Extract the solved routes
    output_routes = "memory/routes"
    CopyFeatures(analysis_class_names["CFRoutes"], output_routes)
    return pd.DataFrame.spatial.from_featureclass(output_routes)

This approach can be adapted to:

Find the closest hospital to an accident
Dispatch the two closest police cars to a crime scene
Find the three closest fire stations within a five-minute drive time

Step 7: The Critical Insight Multi-Indicator Overlay

When you overlay simulated congestion patterns with historical accident hotspots, a crucial pattern emerges: the areas of greatest operational interest are not necessarily the locations with the highest traffic volumes or the highest accident densities.

More significant are locations where multiple indicators begin reinforcing one another. A corridor with increasing traffic pressure may not represent a major concern on its own. An area with elevated accident rates may not require immediate intervention. When both conditions coexist, however, the situation changes. The combination creates a level of operational risk that neither dataset reveals independently.

Here's how you might combine multiple spatial datasets:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

# Assuming you have accident hotspots and congestion layers
# (These would come from your analysis or simulation)

# Spatial join to find overlap between hotspots and congestion
overlap = gpd.overlay(hotspots, congestion, how="intersection")

# Calculate a composite risk score
overlap['risk_score'] = (
    overlap['accident_density'] * 0.6 + 
    overlap['congestion_level'] * 0.4
)

# Identify high-risk areas
high_risk = overlap[overlap['risk_score'] > overlap['risk_score'].quantile(0.90)]

The "Why" Problem: Explainability in GeoAI

Throughout workshops and discussions, one question appeared repeatedly:

"Why?"

Why is this location being highlighted?

Why was this corridor identified as critical?

Why does this recommendation deserve attention?

The quality of the recommendation was important, but the explanation behind it was often more valuable. Trust depends less on automation than on understanding. Decision-makers need to see the evidence, understand the assumptions, and follow the reasoning process that led to a particular conclusion.

For this reason, the future of GeoAI may have less to do with autonomy than with comprehension.

The most successful systems will not necessarily be those that make decisions on behalf of people. More likely, they will be the systems that help people understand complex situations more quickly, evaluate possible consequences more effectively, and make better informed decisions under uncertainty.

The Takeaway

In a city such as Frankfurt, where every operational challenge ultimately unfolds somewhere, affects someone, and influences something nearby, location becomes far more than another attribute in a dataset.

It becomes the foundation for understanding consequence.

The code snippets above come directly from the Spatial Data Science Examples repository a working implementation of the Urban Digital Twin concept. They illustrate a workflow that moves from raw data to spatial understanding to decision intelligence. And that's exactly the kind of spatial reasoning our AI agents will need if they're going to be truly useful in the real world.

Resources

What's your experience with building spatial reasoning into AI systems? Drop a comment below I'd love to hear how you're approaching this challenge.

Building a Local AI Agent for OSM: 21 Days of Iteration

Jan Tschada — Mon, 22 Jun 2026 21:05:50 +0000

Photo by Andrew Dawes on Unsplash

How I turned a vague idea into a working RAG pipeline for dynamic OSM filter generation

I spent 21 days building a local AI agent that translates natural language requests into osmfilter JSON. This is the technical deep‑dive, the architecture, the failures, the fixes, and the lessons learned. If you're working with local LLMs, embeddings, or OSM, there's something here for you.

The Spark

It started with a simple observation: translating "Find only restricted areas" into an OSM filter is a task that should be automatable, but not with a static keyword map. The nuance is too high. seamark:type=restricted_area is correct for maritime zones, but access=exclusion_zone might be better for land. The LLM needs context.

I also wanted everything to run locally. No API calls, no rate limits, no privacy concerns. Just a consumer GPU and a local model.

What follows is the unvarnished story of building that system over three weeks, working mostly in the evenings.

Week 1: Foundation (Days 1–7)

Day 1–2: The LLM Wrapper

I started with the simplest possible thing: a class that wraps llama-cpp-python and can call the model with a prompt. The goal was to get a JSON response—no fluff.

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

class LocalLLMFunctionCaller:

    def __init__(self, model_path: str, n_ctx: int = 2048, temperature: float = 0.0):
        self.llm = Llama(model_path=model_path, n_ctx=n_ctx, temperature=temperature, verbose=False)

    def call_llm(self, prompt: str, max_tokens: int = 200) -> str:
        response = self.llm(prompt, max_tokens=max_tokens, stop=["\n\n"], echo=False)
        return response["choices"][0]["text"].strip()

Simple. But the stop=["\n\n"] would come back to haunt me later.

I also built the executor.py module, which extracts JSON from arbitrary LLM output using a simple brace‑matching algorithm. No regex—just counting depth.

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

def extract_json(text: str) -> str:
    start = text.find("{")
    if start == -1:
        raise ValueError("No JSON object found")
    depth = 0
    for i, ch in enumerate(text[start:], start=start):
        if ch == "{":
            depth += 1
        elif ch == "}":
            depth -= 1
            if depth == 0:
                return text[start:i+1]
    raise ValueError("Unbalanced JSON braces")

This is one of those pieces of code that just works, and I haven't touched it since.

Day 3–4: The Prompt Templates

I wrote prompts.py with several builders:

build_prompt(): basic function‑calling
build_mcp_prompt(): MCP‑style tool calls
build_osmfilter_prompt(): zero‑shot OSM filter generation
build_osmfilter_prompt_with_examples(): few‑shot with provided examples

The last one became the core. It takes a user query and a string of pre‑formatted examples, and outputs a JSON filter structure:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

def build_osmfilter_prompt_with_examples(user_query: str, examples: str) -> str:
    return f"""
You are an assistant that generates OSM filter expressions...
Examples:
{examples}
User: "{user_query}"
Assistant:
"""

This is where the real work would happen in later weeks.

Day 5–7: The Embedding Model

I needed a way to find relevant examples. Enter bge-small-en-v1.5, a 384‑dimension embedding model that fits in 33 MB. I built LocalLLMEmbedder to handle embeddings and store them in SQLite.

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

class LocalLLMEmbedder:

    def __init__(self, model_path: str):
        self.embed_model = Llama(model_path=model_path, n_gpu_layers=-1, embedding=True, verbose=False)

    def create_embedding(self, text: str) -> np.ndarray:
        result = self.embed_model.create_embedding(text)
        return np.array(result["data"][0]["embedding"], dtype=np.float32)

I then ingested the taginfo-wiki.db dataset, embedding each OSM tag description as a JSON object:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

text_obj = {
    "tgroup": tgroup,
    "key": key,
    "value": value or "",
    "description": desc,
    "implies": implies or "",
    "combination": combo or "",
    "linked": linked or "",
    "status": status or "",
    "approval": approval or ""
}
text = json.dumps(text_obj)
embedding_blob = self.create_embedding_blob(text)

At this point, I had a database with embeddings for every documented OSM tag, and a way to search them.

Week 2: RAG & CLI (Days 8–14)

Day 8–9: Building the Example Database

Embedding OSM tags is one thing. Embedding filter examples is another. I created a table called filter_examples with columns for the natural‑language query, the JSON AST, the extracted tags, and the embedding of the natural‑language query.

I also wrote a parser to extract examples from a plain text file:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

def read_osmfilter_samples(self, file_path: str) -> list:
    # Reads a file with User: / Assistant: blocks
    # Parses the JSON and returns a list of examples

The parser is a state machine that detects User: lines, Assistant: lines, and JSON braces. It's not the most elegant parser, but it works for the 200 or so examples I have.

Day 10–11: Search Functions

I implemented search_filter_examples(), which embeds the user query and computes cosine similarity against all stored examples:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

def search_filter_examples(self, query: str, db_path: str = "taginfo-wiki.db", min_score=0.65, k=10):
    q_emb = self.create_embedding(query)
    q_vec = np.frombuffer(q_emb, dtype=np.float32)
    # Query all stored embeddings, compute cosine similarity, filter by min_score
    scored.sort(reverse=True)
    return scored[:k]

The cosine similarity function is straightforward:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

import numpy as np

def cosine(self, a, b):
    a = np.frombuffer(a, dtype=np.float32)
    b = np.frombuffer(b, dtype=np.float32)
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

Day 12–14: The CLI Entry Point

I built func_cli.py to tie everything together. It accepts a --request and a --model path, retrieves examples, builds the prompt, calls the LLM, and prints the result.

uv run osm-functions --request "Find only restricted areas" --model /path/to/model.gguf

This was the moment of truth. And the first runs—failed.

Week 3: Validation & Polish (Days 15–21)

Day 15–16: The Stop Token Disaster

I kept getting empty responses. The LLM would generate nothing. I added --verbose to see the raw prompt and response, and realized the model was generating a blank line before the JSON. That blank line triggered stop=["\n\n"], cutting off the output before it started.

The fix was simple: remove the stop token entirely.

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

# Before
response = self.llm(prompt, max_tokens=max_tokens, stop=["\n\n"], echo=False)

# After
response = self.llm(prompt, max_tokens=max_tokens, echo=False)

I also added a fallback: if the response is empty, return "{}" and let the caller handle it.

Day 17–18: Candidate Validation

I added build_osmtags_validate_prompt() to filter candidate OSM tags by relevance. The LLM receives a list of candidates (key, value, description) and outputs a JSON array of relevant IDs.

This was crucial for domains like maritime vs. land: seamark:type=restricted_area shouldn't show up for a land‑based query.

Day 19–20: Synthesis Instructions

I realised the LLM was sometimes copying examples blindly, even when a combination of tags was more appropriate. I rewrote the prompt to emphasise synthesis:

"If the examples show different tags that could apply to the user request, combine them into a single filter. Do not copy blindly—adapt."

I also added a confidence threshold: if the top‑k similarity score is below 0.4, the LLM is instructed to ask for clarification.

Day 21: The Agent Loop

The final piece was a simple loop that:

Calls the filter generator
Executes the filter (via execute_osmfilter())
If the feature count is zero, broadens the request and tries again
Logs every decision

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

attempt = 0
while attempt < max_attempts:
    filter_json = generate_filter(request)
    result = execute_osmfilter(filter_json)
    if result.count > 0:
        break

    request = broaden_request(request)
    log(f"Empty result, broadening to: {request}")
    attempt += 1

This loop transforms a static generator into an adaptive explorer.

What I Learned (The Hard Way)

Stop tokens are not your friend. Unless you're 100% certain the model will never output the stop token inside valid content, avoid them. Use max_tokens and parse the output instead.

Embedding quality matters. Embedding the full OSM tag description as a JSON string works, but natural‑language sentences like "key=highway, value=primary, description: A major road" gave better results.

The example database is everything. My initial set was heavily biased toward maritime tags. After adding land‑based examples (parking, access, boundaries), the LLM's choices became more context‑aware.

Verbose output is your best debugging tool. Without --verbose, I would have wasted days chasing the stop token bug.

The agent loop is simple but powerful. Less than 50 lines of code, yet it turns a one‑shot generator into a dynamic, adaptive system. Every attempt is logged—request, retrieved examples, generated filter, feature count. That audit trail is invaluable.

The Current State

The system works, for a definition of "works" that includes "still needs more examples, better prompt engineering, and a few edge cases fixed."

The CLI outputs valid JSON filters for most simple queries. The agent loop can handle empty results by broadening the request. And the whole thing runs on a single GPU with less than 8 GB VRAM.

What still needs work:

Synthesis: Combining multiple tags (e.g., maxspeed + surface) still isn't perfect.
Negation: The LLM struggles with "not highways" style queries.
Example diversity: I need more examples, especially complex ones with multiple tags.
Confidence handling: The 0.4 threshold is a guess; I need to fine‑tune it with actual data.

Code Snippets You Might Actually Use

The RAG search function

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

def search_filter_examples(self, query: str, db_path: str = "taginfo-wiki.db", min_score=0.65, k=10):
    q_emb = self.create_embedding(query)
    q_vec = np.frombuffer(q_emb, dtype=np.float32)
    conn = sqlite3.connect(db_path)
    cursor = conn.cursor()
    rows = cursor.execute("""
        SELECT natural_language, json_ast, tags, embedding
        FROM filter_examples
        WHERE embedding IS NOT NULL
    """).fetchall()
    scored = []
    for natural_language, json_ast_str, tags_str, emb in rows:
        emb_vec = np.frombuffer(emb, dtype=np.float32)
        score = self.cosine(q_vec, emb_vec)
        if score >= min_score:
            scored.append((score, natural_language, json_ast_str, tags_str))

    scored.sort(reverse=True)
    return scored[:k]

The JSON extraction function

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

def extract_json(text: str) -> str:
    start = text.find("{")
    if start == -1:
        raise ValueError("No JSON object found")
    depth = 0
    for i, ch in enumerate(text[start:], start=start):
        if ch == "{":
            depth += 1
        elif ch == "}":
            depth -= 1
            if depth == 0:
                return text[start:i+1]
    raise ValueError("Unbalanced JSON braces")

The CLI entry point (simplified)

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--model", required=True)
    parser.add_argument("--request", required=True)
    parser.add_argument("--verbose", action="store_true")
    parser.add_argument("--top-k", type=int, default=5)
    args = parser.parse_args()

    caller = LocalLLMFunctionCaller(args.model, n_ctx=8192, temperature=0.0)
    embedder = LocalLLMEmbedder("data/bge-small-en-v1.5-q4_k_m.gguf")

    results = embedder.search_filter_examples(args.request, k=args.top_k)
    examples = format_examples(results)
    response = caller.call_llm_with_osmfilter_examples(args.request, examples)

    if args.verbose:
        print(response["prompt"])
        print("\n" + response["response"])
    print(response["result"])

Where I'm Taking This Next

Spatial operators: "within 5 km of a hospital"
Wikidata integration: enrich OSM features with facts
Better synthesis: combine tags more intelligently
Open‑source: once the edge cases are fixed (currently a technical spike)

Questions for You

How do you handle tag ambiguity in your geospatial agentic workflows?
What's your threshold for asking an LLM to clarify vs. taking a best guess?
Have you tried building a RAG pipeline for geospatial data?

I'd genuinely love to hear your experiences.

Links:

Let the AI Ask for Data: Dynamic OSM Extraction for Agents

Building an OSM to RDF Pipeline for AI Agents: A Practical Guide

Jan Tschada — Sat, 09 May 2026 14:09:59 +0000

Photo by Patrick Tomasso on Unsplash

You want your AI agent to understand places; ports, roads, buildings, construction sites. OpenStreetMap is the obvious source. But feeding raw OSM data to an agent is like giving someone a raw database dump and asking for insights. It's possible, but painful.

We spent the last few weeks building a pipeline that turns OSM into clean RDF knowledge graphs. It's not a polished product; it's a working prototype that taught us a lot. This post shares the important steps, the mistakes we made, and the code that actually works.

What you'll get from this post:

Why bash scripts fail silently (and how to fix it)
How to generate RDF from OSM without losing your mind
A working Python pipeline you can adapt
Real examples from maritime and construction extraction

Let's dive in.

The problem with OSM + bash

Our first version was simple: a bash script that calls osmconvert to clip a region, then osmfilter to extract categories like transportation, buildings, and POIs.

Here's what the filter generation looked like (broken version):

# This is what we thought would work
filter_string="(building=house,apartments) or (building=yes)"
osmfilter input.osm --keep="$filter_string" -o=output.osm

Result: An empty file. No error message. Exit code 0.

Turns out osmfilter has a complex syntax including parentheses hell. Or the word "and". It expects a simple key=value or key=value syntax. But the (something) or (something) pattern is what every beginner writes.

We fixed it by manually expanding comma-separated values and removing parentheses:

# Correct syntax (but still fragile)
filter_string="building=house or building=apartments or building=yes"

But maintaining this in bash was a nightmare. Every new filter pattern risked breaking the regex-based parsing. Error handling was non-existent. We couldn't write unit tests. And debugging meant adding echo statements and hoping.

So we made a decision: rewrite the bash part in Python.

Step 1: A proper FilterBuilder (the bug fix that matters)

The heart of the pipeline is filter_builder.py. It takes a list of filter strings from your YAML config and builds a valid osmfilter --keep string.

Here's what it looks like:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

def build_filter_string(self, filters: List[str]) -> str:
    expanded = []
    for filter_str in filters:
        if '=' in filter_str:
            key, values = filter_str.split('=', 1)
            if ',' in values:
                # Expand "building=house,apartments" into multiple
                for value in values.split(','):
                    expanded.append(f"{key}={value.strip()}")
            else:
                expanded.append(f"{key}={values}")
        else:
            expanded.append(filter_str)

    # Join with " or " – no parentheses, no "and"
    return " or ".join(expanded)

Example: ["building=house,apartments", "amenity=school"] becomes "building=house or building=apartments or amenity=school".

We also added validation to catch invalid syntax before calling osmfilter:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

def validate_filter_syntax(self, filter_string: str) -> bool:
        """
        Validate that the filter string uses proper osmfilter syntax.

        Args:
            filter_string: The filter string to validate

        Returns:
            True if syntax appears valid, False otherwise
        """
        if not filter_string:
            return False

        # Check for invalid patterns that caused the original bug
        invalid_patterns = [
            "&&",  # No logical AND operators
            "||",  # No logical OR operators
        ]

        for pattern in invalid_patterns:
            if pattern in filter_string:
                self.logger.warning(f"Invalid filter syntax detected: '{pattern}' in '{filter_string}'")
                return False

        # Basic structure validation
        parts = filter_string.split(" or ")
        for part in parts:
            part = part.strip()
            if not part:
                continue
            if '=' not in part and part != "*":
                self.logger.warning(f"Invalid filter part: '{part}' (missing '=')")
                return False

        return True

This alone eliminated 90% of our "empty file" headaches.

Lesson: Never trust external tools to fail loudly. Always validate inputs before calling them.

Step 2: Retry logic and timeout handling

Bash scripts don't retry. If osmconvert fails because of a temporary network issue (yes, reading a local file can still fail), the whole pipeline dies.

We built a simple retry wrapper:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

def _run_command_with_retry(self, cmd, description, max_retries=3):
    for attempt in range(1, max_retries + 1):
        try:
            result = subprocess.run(cmd, capture_output=True, text=True, timeout=300)
            if result.returncode == 0:
                return True
            else:
                logging.warning(f"Attempt {attempt} failed: {result.stderr}")
                time.sleep(2 ** attempt)  # exponential backoff
        except subprocess.TimeoutExpired:
            logging.warning(f"Timeout on attempt {attempt}")
    return False

Now when osmfilter chokes on a huge file, we retry with increasing delays. It's not elegant, but it works.

Lesson: External commands are black boxes. Wrap them with retries and timeouts, especially when processing large OSM extracts.

Step 3: Generating RDF (the AI-friendly format)

OSM XML is great for editing maps. But AI agents prefer graphs. Specifically, RDF triples.

We added an --rdf flag to the CLI. When enabled, the pipeline does two extra steps after each category extraction:

Convert the OSM file to PBF using osmconvert
Convert the PBF to TTL using osm2rdf

Here's the code (simplified):

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

def _generate_rdf(self, osm_file, category, output_prefix):
    pbf_file = self.working_dir / f"{output_prefix}_{category}.pbf"
    ttl_file = self.output_dir / f"{output_prefix}_{category}.ttl"

    # Step 1: OSM -> PBF
    subprocess.run(["osmconvert", str(osm_file), f"-o={pbf_file}"], check=True)

    # Step 2: PBF -> TTL
    subprocess.run(["osm2rdf", str(pbf_file), "-o", str(ttl_file)], check=True)

    return ttl_file

Now you get both .osm and .ttl files. The TTL file contains triples like:

:node_123456 a :Node ;
    :hasTag "harbor=yes" ;
    :hasName "Bandar Abbas" ;
    :hasWikidata "Q207137" .

An AI agent can load this into memory or a RAG graph database and run SPARQL queries:

SELECT ?port ?name WHERE {
    ?port :hasTag "harbor=yes" .
    ?port :hasName ?name .
    ?port :hasWikidata ?wikidata .
}

Lesson: Don't make agents parse OSM XML. Convert to RDF once, then let them query.

Step 4: Tag control – include only what matters

Raw OSM objects often have dozens of tags. Most are irrelevant for high-level intelligence. A port might have name, name:en, operator, wikidata, seamark:light:colour, seamark:light:period, source, created_by, etc.

We added include_tags to the YAML configuration:

maritime:
  subcategories:
    seamark:
      filters:
        - "landuse=harbour"
        - "harbor=yes"
      include_tags:
        - "name"
        - "name:en"
        - "operator"
        - "wikidata"

In filter_builder.py, we translate this into the --keep-tags argument:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

if keep_tags:
    # osmfilter requires 'all' at the beginning
    cmd.extend(["--keep-tags", "all " + " ".join(keep_tags)])

Now the output RDF only contains those tags. Smaller files, cleaner graphs, faster agent queries.

Lesson: Configuration is better than hardcoding. Let users specify which tags matter for their domain.

Step 5: Directory handling and idempotency

A good pipeline should be safe to run multiple times. We added:

--force flag to overwrite existing files
--dry-run to show what would be executed
Automatic creation of output and working directories

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

if output_file.exists() and not self.force_overwrite:
    logging.info(f"Output exists, skipping: {output_file}")
    return output_file

This saved us countless headaches during testing.

Lesson: Always make your tools idempotent. It costs almost nothing and saves hours.

Real-world examples that worked

We tested on three real scenarios:

1. Maritime features (Strait of Hormuz)

osm-extract --bbox "56.037612,26.951262,56.098037,26.977960" \
            --config maritime_features.yaml \
            --rdf --verbose

Output: A small OSM file and a TTL file containing every port, harbour, and naval base in that area. The RDF was clean enough to answer: "Which ports have a Wikidata ID but no English name?"

2. Construction sites

We created a separate config that only extracts landuse=construction:

construction:
  subcategories:
    landuse:
      filters: ["landuse=construction"]
      include_tags: ["name", "operator", "construction"]

Running this on Tehran gave us a list of active construction zones. An agent monitoring change over time could detect new infrastructure before it's officially announced.

3. Global extraction

Yes, you can extract the whole planet:

osm-extract --bbox="-180,-90,180,90" --config features.yaml --rdf

This takes a while (and a lot of RAM), but it works. We don't recommend it for regular use, but it's good for training geospatial foundation models.

What we learned (the hard way)

1. Empty files are still valuable.

We initially skipped RDF generation for files smaller than 1MB. That was a mistake. A 200-byte file with a single harbor=yes node tells the agent "there is a harbor here, but we know nothing else." That's still intelligence. We removed the size check.

2. Logging saves hours.

We added verbose and debug flags. When something fails, we can see exactly which command was run, what the exit code was, and what stderr said. No more guessing.

3. Unit tests are not optional.

Once we fixed the parentheses bug, we wrote tests for it. That bug has never returned. Every new filter pattern is now tested.

4. OSM's human quirks: different tags, same concept

Compressed files are trivial. The real challenge is that OSM is a human‑generated, consensus‑driven dataset. Two different mappers will tag the same real‑world feature in completely different ways.

The complete pipeline in one diagram (text version)

User input (region or bbox)
    ↓
Geocoder → bounding box
    ↓
osmconvert → region.o5m
    ↓
For each enabled category:
    ├── Get filters from YAML
    ├── Build osmfilter command (with proper syntax)
    ├── Run osmfilter → category.osm
    └── If --rdf:
            ├── osmconvert → category.pbf
            └── osm2rdf → category.ttl
    ↓
Generate manifest.json
    ↓
Done.

What's next (our roadmap)

Auto-generate configs from OSM Wiki – scrape tag documentation and create extraction rules automatically.
Agent-in-the-loop extraction – let an agent request custom filters ("all hospitals with helipads within 10km of a highway").
Spatial reasoning over RDF – load TTL into graph database and enable geospatial queries.

But even without those, the current prototype is useful. It turns OSM from a raw data dump into a structured knowledge source that AI agents can actually consume.

Final thoughts

Building this pipeline took longer than we expected. Most of the time wasn't spent on the core logic – it was spent wrestling with edge cases, silent failures, and undocumented tool behavior - and stupid me situations.

But now it works. It's not perfect, but it's reliable enough for real-world extraction. And because it's open, others can improve it.

Let us know if you would be interested into using it...

How We Built a Geospatial AI That Reads Satellite Images

Jan Tschada — Thu, 30 Apr 2026 04:08:27 +0000

A developer’s guide to analyze_image, analyze_text, and the future of location intelligence.

The motivation

You’re a developer. You know that satellite imagery is everywhere. But most GeoAI solutions are either:

Black‑box desktop tools with 500 buttons
Intelligence‑grade surveillance systems you’d never touch
Overhyped pixel classifiers that can’t tell a port from a parking lot

We wanted something different:

A code‑first, ethical, and actually useful geospatial AI that runs in a Jupyter notebook.

So we built it with:

arcgis.ai.analyze_image – to read satellite photos
arcgis.ai.analyze_text – to write structured reports

A few dozen lines of Python, and ArcGIS Location Platform (beta AI)

No mouse clicks. Just code.

In this post, I’ll show you exactly how it works – including real snippets from our platform.py – so you can steal the ideas (or the whole notebook) for your own projects.

The Core Idea (Super Simple)

User pans a map to any location.
We export that exact view as a georeferenced PNG.
We send the image + a strict, no‑speculation prompt to our beta AI endpoint using analyze_image.
The AI returns a natural‑language description.
We feed that description + location data into analyze_text with a report template.
Out comes a clean Markdown report (Location, Features, Patterns, Risks, Assessment).

That’s it. No complex pipelines. No training custom models. Just clever prompting and two AI functions.

The Code You Actually Care About

Our entire workflow is in platform.py. Let me walk you through the juicy parts.

Export a map image (no screenshot hacks)

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

def export_map(map_view) -> ExportedMapImage:
    bbox = f"{map_view.extent['xmin']},{map_view.extent['ymin']},{map_view.extent['xmax']},{map_view.extent['ymax']}"
    bbox_sr = map_view.extent['spatialReference']['wkid']
    map_layer = MapImageLayer("https://services.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer")
    return ExportedMapImage(
        map_export_result=map_layer.export_map(bbox=bbox, bbox_sr=bbox_sr, image_format="png"),
        bbox=bbox, bbox_sr=bbox_sr
    )

Why this is cool:

The exported image is georeferenced, so we know exactly which coordinates each pixel covers. That means we can later tell the AI: "You’re looking at 25.02° N, 55.04° E – the Port of Jebel Ali."

Download image bytes (streaming, memory‑efficient)

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

def download_bytes(url, timeout=10):
    response = requests.get(url, stream=True, timeout=timeout)
    response.raise_for_status()
    file_bytes = io.BytesIO()
    for chunk in response.iter_content(chunk_size=8192):
        if chunk:
            file_bytes.write(chunk)
    file_bytes.seek(0)
    return file_bytes

Nothing fancy, but it’s solid. We stream the image in 8KB chunks so we don’t blow up memory on huge exports.

Reverse geocode the image center

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

gps_extent = project([exported_image.map_export_result["extent"]], in_sr=bbox_sr, out_sr=4326)[0]
[xmin, ymin, xmax, ymax] = gps_extent.coordinates()
gps_lon = xmin + 0.5 * (xmax - xmin)
gps_lat = ymin + 0.5 * (ymax - ymin)

geocoding_result = reverse_geocode({"x": gps_lon, "y": gps_lat, "spatialReference": {"wkid": 4326}}, lang_code="EN")
address = geocoding_result.get("address")

Now we have a human‑readable address (e.g., "Port of Jebel Ali, Dubai, UAE") to feed into the AI prompts.

The secret sauce: a disciplined prompt for analyze_image

This is where most people mess up. They ask, "Tell me everything about this place." Then the AI hallucinates.

We do the opposite:

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

ai_prompt = f"""
    You are a geospatial imagery analyst.
    Analyze the satellite image using only visible evidence.
    State uncertainty explicitly. Do not speculate.

    Location: {gps_lat}, {gps_lon}
    {address.get('LongLabel') if address else ''}
"""

The rules:

Only what you see
If unsure, say so
No guessing

This dramatically reduces hallucinations. The AI will say "I see what looks like a port, but I cannot confirm the exact type" instead of inventing a naval base.

Call analyze_image

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

image_base64 = f"data:image/png;base64,{base64.b64encode(image_bytes.getvalue()).decode('utf8')}"
analyze_image_result = analyze_image(image=image_base64, prompt=ai_prompt, gis=portal, to_language="EN")

That’s it. One line. The gis instance is your authenticated ArcGIS portal connection.

Generate a structured report with analyze_text

We take the image analysis output and wrap it into a report template.

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0

ai_summarize_prompt = f"""
    You are a senior geospatial analyst.
    Create a professional situational analysis report.
    Output MUST be valid Markdown.
    Sections:
    ## Location
    ## Observed Features
    ## Activity & Patterns
    ## Risk & Threat Assessment
    ## Analyst Assessment
    Write concisely, analytically.
"""

intel_data = {
    "image_analysis": analyze_image_result.results,
    "location": {"latitude": gps_lat, "longitude": gps_lon, "address": address}
}

analyze_text_result = analyze_text(text=str(intel_data), prompt=ai_summarize_prompt, gis=portal, to_language="EN")

Boom. The second AI reorganises the raw description into a clean, sectioned report. No manual parsing required.

Here’s what the AI actually produced for one of our test ports:

Situational Analysis Report - Date 2026-04-20T10:15:32

Location

Geocoordinates: 25.0137 N, 55.0743 E
Identified Site: Mena Jabal Ali, Dubai, United Arab Emirates
Area Overview: Situated in the Jebel Ali district of Dubai, encompassing significant coastal, urban, and artificial landforms.

Observed Features

Prominent artificial island resembling a palm tree (Palm Jebel Ali).

Urban coastal infrastructure comprising dense road networks, residential zones, and potential industrial facilities.

Multiple marinas and docks located along the shoreline, enhancing maritime accessibility.

Visible boundaries distinguishing developed urban tracts from adjacent desert areas.

Pockets of sparse vegetation and undeveloped open land interspersed throughout the region.

Activity & Patterns

Extensive urban development and engineered expansions, typified by the artificial island and substantial coastal modification, suggest large-scale planning and ongoing construction activity.

The presence of marinas and docks points to active maritime transit and probable commercial shipping or recreational boating.

The orderly distribution of residential and industrial areas implies strategic urban zoning.

Noticeable demarcations between green/open land and developed sectors reflect active land management and expansion.

Risk & Threat Assessment

The concentration of critical urban and maritime infrastructure in a narrow coastal corridor increases vulnerability to natural hazards (e.g., sea-level rise, storms) and potential deliberate disruptions.

High-profile artificial landforms may attract heightened attention and pose elevated operational and maintenance risks.

The juxtaposition of industrial, residential, and transport nodes requires robust monitoring to mitigate economic or environmental threats.

Analyst Assessment

Palm Jebel Ali and the surrounding Jebel Ali area represent a nexus of strategic urban expansion, maritime operations, and infrastructural development within Dubai. The imagery confirms sustained investment in coastal engineering and the diversification of land use.

While the overall stability of the site appears high, continued monitoring is recommended to track further urban growth, shifting coastal dynamics, and associated risk profiles, given the region’s prominence and developmental trajectory.

Further detail regarding specific facility functions, population density, or economic assets is not ascertainable from imagery alone.

What’s Coming Next (Roadmap – and we need your help)

Our current version only looks at the image. But an image doesn’t tell you everything.

We plan to integrate:

OpenStreetMap – roads, building footprints, land use, ports
Wikidata – facts about the place (population, founding year, official name)

Then we’ll feed that structured data into analyze_text alongside the image analysis. Imagine the AI saying:

"I see a large building. OSM says it’s a hospital. Wikidata tells me it was built in 1985 and has 200 beds."

That’s real context‑aware geospatial AI.

We’d love your input:

Which OSM tags are most valuable for your use cases?
How would you improve the prompt to reduce hallucinations further?
Would you use this in production? Why / why not?

We’re not building a black‑box image intelligence tool. We’re building an open, ethical, developer‑friendly geospatial AI.

If that resonates with you:

Comment below with your craziest geospatial AI idea
Share this post with one colleague who might find it useful

Let’s make geospatial intelligence accessible to every developer.

Spatial Data Science: A New Way to Solve Problems

Jan Tschada — Fri, 17 Oct 2025 06:26:41 +0000

Welcome to the world of Spatial Data Science in ArcGIS! In this section, we’re going to explore a simple but powerful idea: thinking about where things happen can help us solve some of our most complex challenges.

It All Starts with a Question

At the heart of this approach is something we call "The Geographic Approach." This is just a fancy way of saying we start with the problem itself, not with a specific tool or piece of software. It’s a mindset that encourages us to ask the right questions about location, such as:

📍 Where is it happening? (For example, where are the traffic accidents concentrated?)

⁉️ Why is it happening there? (What is it about those specific intersections that makes them dangerous?)

🗺️ What’s nearby? (Are the accidents close to schools, bars, or major highways?)

❓ How is it changing? (Is the problem getting better or worse over time?)

By starting with these simple questions, we can begin to see the world in a new way.

Spatial Analysis: The Language of Location

When we use maps and data to answer these "where" questions, we are doing spatial analysis. Think of it as learning a new language—the language of location. This language helps us uncover hidden patterns, relationships, and trends that we might completely miss if we just looked at spreadsheets or lists.

Whether we're planning a new park, monitoring the health of a forest, or figuring out the best location for a new store, geography provides the essential context. It connects the dots and tells the full story.

Keep an eye out for our next section, where we’ll share real-world examples, live demos, and step-by-step guides that show how we bring data to life using ArcGIS. You’ll see just how powerful it can be to ask, "Where?"

📍Location
🎬 Technical Session

Mapping the geospatial patterns of broadcasted news

Jan Tschada — Fri, 19 May 2023 16:53:15 +0000

Mapping the geospatial patterns of broadcasted news allows a geospatial analyst to gain insights into common and unusual geospatial patterns. We decided using one of the most comprehensive news collection named "Global Database Events of Tone and Language" (GDELT) as the ground truth.

The geography of worldwide news coverage

Understanding the geospatial patterns of such a massive knowledge graph is often difficult for non geospatial experts. The machine learning algorithms extract articles and features from websites in real-time. The geocoding engine matches extracted locations against over eleven million well-known place names. An article mentioning a place like "New York" leads to an extracted feature location. But, the article does not have to be specific about the named location. For instance, an article regarding "The impact of the COVID pandemic on capital markets." — mentioning "New York is no exception" — leads to a location match. We should expect some false positives, but the sum of all extracted locations should reflect a geospatial pattern and give us a coarse-grained overview.

Accessing the geospatial features using the geoprotests API

The geoprotests API offer ready-to-use geospatial features representing broadcasted news related to protests and demonstrations. You can use these geospatial features to build various mapping and geospatial applications.

Every geospatial result support the GeoJSON and Esri FeatureSet format out of the box. All endpoints support an optional date parameter for filtering the results. For best performance, the serverless cloud-backend calculate the geospatial aggregations of the last 24 hours between midnight and 1 AM. The serverless functions save these geospatial features and yesterday should be the latest available date. Without specifying a date, we calculate the geospatial features of the last 24 hours on-the-fly.

Ramp up your development environment

You need to activate your Rapid API account. Please, check out the RapidAPI Account Creation and Management Guide for more details.

Setup your Python based development environment using your weapon of choice. Using pip creating a virtual environment is one simple approach.

python -m venv geoint

# Linux
source geoint/bin/activate

# Windows
geoint/Scripts/activate

The Python module requests offers easy and elegant access to http functions. You need to install this module using pip.

pip install requests

Accessing the broadcasted news of 31th December 2022

The hotspot endpoint offers access to statistically significant named locations. The count of news mentioning these locations define the degree of significance.

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0
import requests

url = 'https://geoprotests.p.rapidapi.com/hotspots'

querystring = {
  'date': '2022-12-31',
  'format': 'geojson'
}

# Authenticate: https://rapidapi.com/auth
api_key = '<SIGN-UP-FOR-KEY>'
headers = {
  'x-rapidapi-host': 'geoprotests.p.rapidapi.com',
  'x-rapidapi-key': api_key
}

geojson_response = requests.request('GET', url, headers=headers, params=querystring)
geojson_response.raise_for_status()
features = geojson_response.json()

The features dictionary represents the named locations in GeoJSON format.

{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "id": 41415,
      "geometry": {
        "type": "Point",
        "coordinates": [
          37.6156,
          55.7522
        ]
      },
      "properties": {
        "OBJECTID": 41415,
        "name": "Moscow, Moskva, Russia",
        "timestamp": "2022-12-31T00:00:00",
        "count": 77
      }
    }
  ]
}

Accessing the aggregated broadcasted news

The spatial aggregations are hexagonal geospatial features having a specific property representing the count of the news mentioning a location being within a hexagonal grid cell. Spatial analyst's use mapping capabilities visualizing hot and cold spots of these spatial aggregations.

Let us use the powerful mapping capabilities of ArcGIS and visualize the aggregated broadcasted news as a feature set. You need to specify the output format to esri.

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0
url = 'https://geoprotests.p.rapidapi.com/aggregate'

querystring = {
  'date': '2022-12-31',
  'format': 'esri'
}

headers = {
  'x-rapidapi-host': 'geoprotests.p.rapidapi.com',
  'x-rapidapi-key': api_key
}

esri_response = requests.request('GET', url, headers=headers, params=querystring)
esri_response.raise_for_status()
aggregated_features = esri_response.json()

The aggregated features represent the corresponding hexagonal grid cells as a Python dictionary. You need to setup the arcgis Python Module into your development environment. The contained map widget offers powerful mapping capabilities for your Jupyter Notebook environment. Follow the Install and Setup Guide.

# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0
from arcgis.gis import GIS
from arcgis.features import FeatureSet

# Create a FeatureSet from the features
aggregated_featureset = FeatureSet.from_dict(aggregated_features)

# Anonousmly connect to ArcGIS Online
gis = GIS()

# Create a simple map view
map_view = gis.map('Europe')

# Add the FeatureSet as a layer
map_view.add_layer(aggregated_featureset)

You should tweak the rendering of the features using a dedicated renderer instance. The renderer is an optional parameter of the add_layer method. But, you could also use a different approach for mapping the aggregated features. The spatial-enabled dataframe offers plotting capabilities in combination with the map view. So that you can easily define rendering supporting natural class breaks.

# Create a simple map view
map_view = gis.map('Europe')

# Add the FeatureSet as a layer
aggregated_featureset.sdf.spatial.plot(map_view, 
                                      renderer_type='c',
                                      method='esriClassifyNaturalBreaks',
                                      class_count=5, 
                                      col='count', 
                                      cmap='YlOrRd',
                                      alpha=0.35)

The produced map shows the hotspot in red being Moscow, Russia with a mention count of 77, as we expected.

Any feedback is welcome, keep on mapping!

Feel free to try it out: geoprotests API.