DEV Community

Cover image for I asked a mapping API for lakes named Huron. It sent me Lake Baikal.
Sanish Kumar
Sanish Kumar

Posted on

I asked a mapping API for lakes named Huron. It sent me Lake Baikal.

I was testing a geospatial library against a public demo server and sent it a filter:

GET /collections/lakes/items?filter=name LIKE '%Huron%'&filter-lang=cql2-text
Enter fullscreen mode Exit fullscreen mode

It replied 200 OK. It reported 25 matches. Here is what came back:

curl against the pygeoapi demo returning every lake in the collection for a Huron filter

Lake Baikal. Lake Winnipeg. Great Slave Lake. Lake Victoria. Lake Tanganyika.

Not one of them is Lake Huron. The server had ignored my filter and handed back the entire collection. No error. No warning. No header saying "I don't do that."

This is the public pygeoapi demo, and it is not broken. It was being honest in a way I wasn't listening to.

The spec has a footgun

OGC API - Features puts filtering in a separate part of the standard. A service publishes what it supports at /conformance, and pygeoapi's list includes these two:

http://www.opengis.net/spec/cql2/1.0/conf/cql2-text
http://www.opengis.net/spec/cql2/1.0/conf/basic-cql2
Enter fullscreen mode Exit fullscreen mode

and does not include this one:

http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/filter
Enter fullscreen mode Exit fullscreen mode

The first two mean "I understand the CQL2 text encoding." The third means "my items endpoint will actually apply a filter you send it."

Those are very different promises and they look nearly identical in a list of forty-four URIs. I read the first two and assumed the third.

An unknown query parameter is not an error in HTTP. So filter= gets dropped on the floor, you get a 200, and you get everything. If you aren't counting rows, it is indistinguishable from success.

Point the same request at ldproxy's demo, which does implement Part 3, and it behaves exactly as you'd expect. Same query, same encoding, same status code, completely different meaning:

Side-by-side conformance comparison: pygeoapi missing conf/filter returns 25 of 25 features, ldproxy returns 1 of 1

Why I care more than I used to

For a human clicking through a web map, this is a bug you would probably catch. Twenty-five results when you expected one is the kind of thing that makes you squint.

For an agent, it's poison.

Give a model access to a spatial API and it will write queries with total confidence. It has no way to distinguish "the filter matched 25 things" from "the filter was discarded and you're looking at the whole table." It will summarise Lake Baikal as a result about Huron and move on to the next tool call. Nothing downstream catches it, because nothing downstream knows what the right answer looked like.

The dangerous failure mode was never an API returning an error. It's an API returning something plausible.

What I changed

I've been building VoiceGIS, which compiles plain-language requests into typed, checked operations against spatial data. It has a notion of capabilities — what a given layer can actually do — and until recently I populated those by hand.

Now they're derived from the service itself:

import { catalogFromOgcService } from 'voicegis/adapters';

const { catalog, conformance, warnings } =
  await catalogFromOgcService('https://demo.pygeoapi.io/master');

conformance;
// { cql2Text: true, basicCql2: true, filter: false, canFilter: false }
Enter fullscreen mode Exit fullscreen mode

canFilter: false, so the derived catalog gets layer.visibility, query.clear, selection.clear and data.export — and no query operations whatsoever. Ask it to filter and it refuses before a single request leaves the process:

{
  "status": "needs_input",
  "executed": false,
  "issues": [{ "code": "catalog_capability_missing" }]
}
Enter fullscreen mode Exit fullscreen mode

along with a warning that spells it out in English:

This service does not advertise OGC API - Features Part 3 filtering with CQL2-Text. Attribute queries are NOT enabled: a service in this state has been observed to accept a filter, answer 200, and return every feature as though it had been applied.

Run the same function against ldproxy and you get twenty-one layers with full query capabilities, field types pulled from /queryables, and the geometry property detected per collection — ldproxy calls it geom, pygeoapi calls it geometry, and hardcoding either one is its own quiet little bug waiting to happen.

The shape underneath

The pattern I keep returning to is: make the system refuse rather than approximate.

A request moves through four stages, and the design work is entirely in what each stage is permitted to assume.

Pipeline diagram: request, compiler, validate and policy, adapter, receipt — with the catalog derived from conformance feeding the validation step

Nothing is a string. "area is greater than 2 hectares" becomes:

{ type: 'comparison', field: 'area_ha', operator: 'gt', value: 2, unit: 'hectare' }
Enter fullscreen mode Exit fullscreen mode

which is something you can validate, log, diff, and re-check server-side. There is no moment at which a generated SQL string exists to be wrong.

The rule that took me longest to accept: a half-understood request executes none of its parts. go to Delhi and show hydrants used to fly to Delhi and quietly skip the layer it didn't recognise. Now it does nothing and explains why. It feels worse in the moment. It is correct — a partial side effect nobody asked for is much harder to notice than a refusal.

Writing the tests for that rule turned up three more of the same species in my own code:

  • show hospitals and schools toggled hospitals and silently dropped schools.
  • zoning is 5 star retail on a text field parsed as the number 5 carrying the unit "star retail", matched zero rows, and reported success.
  • magnitude > 5 bananas treated bananas as a unit and behaved exactly like > 5.

All three returned ready. All three were wrong. I had written every one of them myself while being quite pleased about type safety.

The part I'd defend in a code review

Deriving capabilities from /conformance means some services end up with a nearly useless catalog. pygeoapi comes out read-only. Sooner or later someone will open an issue asking me to just send the filter and hope for the best.

I won't, and the Lake Baikal result is the reason. A tool that answers 80% of questions correctly and 20% wrongly-but-plausibly is worse than a tool that answers 80% and refuses the rest, because from the outside you cannot tell those two categories apart. The refusal is the feature.

What it doesn't do

The compiler is deterministic — a grammar, not a model. That makes it fast: about 0.02 ms to compile, and constant regardless of dataset size, because it only ever touches the catalog, never the features. It's also never creatively wrong.

It is also narrow. High precision, low recall. It says needs_input more often than I would like.

The fix is not to loosen it. It's to put a model in front that proposes a plan, and keep the validator behind it refusing anything ungrounded. That's the next thing I'm building, and the validator is the half that already exists.

The OGC adapter has been tested end to end against exactly one conformant service, which is why it's marked experimental. One data point is not a compatibility matrix.

Try it

Live demo, real USGS earthquake data, no signup: https://voicemap-three.vercel.app/

Type show earthquakes where magnitude is greater than 5. Then switch off the export permission and ask it to export something. Watch it stop at Authorize and tell you why.

npm install voicegis
Enter fullscreen mode Exit fullscreen mode

Zero runtime dependencies, MIT. Source and issues: https://github.com/SanishKumar/VoiceGIS

If you maintain an OGC API - Features service, it's worth checking what your /conformance claims against what your items endpoint actually does with a filter parameter. I'd genuinely like to know how common this is — I've only surveyed two.

Top comments (0)