DEV Community

Roman Kotenko
Roman Kotenko

Posted on

How I Normalized 30+ Different 511 Traffic APIs Into One REST Endpoint

#go #api #webdev #architecture

Every US state and Canadian province runs its own 511 traveler information system. They all serve the same kind of data — traffic incidents, cameras, road conditions, message signs — but every single one does it differently.

I spent the past year building Road511, a unified API that normalizes data from 57 jurisdictions into one consistent REST endpoint. Here's what I learned about wrangling 30+ incompatible APIs into a single schema.

The Problem

Let's say you want traffic camera feeds for a route from New York to Chicago. You'll need to integrate:

  • New York — ibi511 REST API with custom JSON
  • Pennsylvania — ASP.NET map layer markers
  • Ohio — OHGO Public API with api-key param auth
  • Indiana — CARS platform REST microservices
  • Illinois — Travel Midwest POST endpoints with bbox body

Five states, five completely different APIs. Different authentication, different response formats, different field names for the same data. And that's just cameras — add events, road conditions, and message signs, and you're looking at 20+ integrations for one cross-state route.

Now multiply that by the entire continent.

The Format Zoo

Here's a sample of what "traffic event" looks like across different platforms:

Transnomis (GA, ID, WI, and 15 more states) — flat JSON with FlexString fields that can be either strings or numbers:

{
  "ID": "12345",
  "Head": "Crash on I-85 NB",
  "EventType": "incidents",
  "Severity": "Major",
  "Latitude": "33.7590",
  "Longitude": "-84.3880"
}
Enter fullscreen mode Exit fullscreen mode

WZDx (24 US states) — GeoJSON with deeply nested properties:

{
  "type": "Feature",
  "geometry": { "type": "LineString", "coordinates": [...] },
  "properties": {
    "core_details": {
      "event_type": "work-zone",
      "road_names": ["I-95"],
      "direction": "northbound",
      "description": "Lane closure for bridge repair"
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

ArcGIS FeatureServer (KY, OH, TX, and 15 more) — Esri's paginated query format:

{
  "features": [{
    "attributes": {
      "OBJECTID": 456,
      "INCIDENTTYPE": 2,
      "DESCRIPTION": "Multi-vehicle crash",
      "LATITUDE": 38.0406,
      "LONGITUDE": -84.5037
    },
    "geometry": { "x": -84.5037, "y": 38.0406 }
  }],
  "exceededTransferLimit": false
}
Enter fullscreen mode Exit fullscreen mode

Wyoming — Protobuf binary files, Base64-encoded and XOR'd with a static key. Seriously.

CARS GraphQL (NE, CO) — a GraphQL BFF layered over REST microservices.

Quebec — WFS GeoJSON from one endpoint, ASP.NET Element.ashx markers from another.

New England Compass (ME, NH, VT) — C2C XML portal with ATMS-format feeds.

That's 8+ distinct protocols. Same data, wildly different delivery.

The Architecture

The solution is a plugin registry with a single function signature:

type FetchFunc func(ctx context.Context, sr SourceResource) (*FetchResult, error)

type FetchResult struct {
    Events        []TrafficEvent
    Features      []Feature
    ResponseBytes int
}
Enter fullscreen mode Exit fullscreen mode

Every adapter — whether it's parsing Transnomis JSON, decoding Wyoming protobuf, or querying ArcGIS pagination — implements this one interface. It takes a source resource config (URL, credentials, jurisdiction code) and returns normalized events and features.

Registration happens at init time:

func init() {
    Register("ky_arcgis", "events", fetchKYArcGISEvents)
    Register("ky_arcgis", "cameras", fetchKYArcGISCameras)
    Register("ky_arcgis", "rest_areas", fetchKYArcGISRestAreas)
    Register("ky_arcgis", "ferries", fetchKYArcGISFerries)
    Register("ky_arcgis", "signs", fetchKYArcGISSigns)
}
Enter fullscreen mode Exit fullscreen mode

523 Register() calls across 76 adapter files. The scheduler doesn't know or care what format each source uses — it just calls Fetch() and gets back normalized data.

The Normalized Model

Everything converges into two tables:

traffic_events — time-bounded incidents with lifecycle tracking:

type TrafficEvent struct {
    ID                 string
    Source             string      // "on", "ga", "tx"
    Jurisdiction       string      // "ON", "GA", "TX"
    Type               EventType   // incident, construction, closure, weather
    Severity           Severity    // minor, moderate, major, critical
    Status             EventStatus // active, archived
    Title              string
    Description        string
    AffectedRoads      []string
    Direction          string
    LanesAffected      string
    Latitude, Longitude float64
    StartTime          time.Time
    EndTime            *time.Time
    EstimatedEndTime   *time.Time
    RoadClass          string      // interstate, us_highway, state_highway, local
    Metadata           json.RawMessage // source-specific fields preserved
}
Enter fullscreen mode Exit fullscreen mode

features — generic table for everything else. Cameras, signs, weather stations, rest areas, bridge clearances, truck routes, EV charging — all use the same table with a feature_type discriminator and type-specific fields in a JSONB properties column. Adding a new data type requires zero schema migrations.

The Hard Parts

1. Lifecycle Tracking

A traffic incident isn't a static record. It escalates (minor to major), lanes change, ETAs shift, and eventually it clears. Each source reports the current state differently — some send updates, some just stop including the event.

The solution: diff the current state against the previous fetch. Track every change in an event_history table — severity changes, description updates, lane changes, archival. This enables analytics like clearance time percentiles and corridor reliability scoring.

2. Coordinate Chaos

Most sources send WGS84 (lat/lng). Some send Web Mercator (EPSG:3857). Some send state plane coordinates. Wyoming sends coordinates embedded in XOR'd protobuf. ArcGIS returns coordinates in its own spatial reference that may or may not be 4326.

PostGIS handles the heavy lifting, but each adapter has to know what it's receiving and transform accordingly.

3. Rate Limiting at Scale

57 jurisdictions, each with multiple resource types (events, cameras, signs, weather), each polling every 1-5 minutes. That's hundreds of outbound requests per minute to external 511 systems.

The scheduler uses per-server semaphores with configurable concurrency limits and request gaps. Circuit breakers back off on repeated failures. Adaptive backoff increases poll intervals when a source is slow or returning errors.

4. Schema-Free Feature Types

When I started, I had separate tables for cameras, signs, rest areas. Every new data type meant a migration. The pivot to a generic features table with JSONB properties was the best architectural decision in the project. Adding EV charging stations (100k+ from NREL) or bridge clearances (621k from FHWA) required zero schema changes.

The API

After all that normalization work, the API is straightforward:

Get active incidents in Ontario:

curl "https://api.road511.com/api/v1/events?jurisdiction=ON&type=incident&status=active" \
  -H "X-API-Key: your_key"
Enter fullscreen mode Exit fullscreen mode 
{
  "data": [
    {
      "id": "on_ev_12345",
      "jurisdiction": "ON",
      "type": "incident",
      "severity": "major",
      "title": "Multi-vehicle collision on Highway 401 WB",
      "affected_roads": ["Highway 401"],
      "direction": "Westbound",
      "lanes_affected": "2 of 4 lanes closed",
      "latitude": 43.6532,
      "longitude": -79.3832,
      "start_time": "2026-03-29T08:15:00Z",
      "estimated_end_time": "2026-03-29T12:00:00Z"
    }
  ],
  "total": 85,
  "limit": 100,
  "offset": 0,
  "has_more": false
}
Enter fullscreen mode Exit fullscreen mode

Get traffic cameras in Georgia as GeoJSON:

curl "https://api.road511.com/api/v1/features/geojson?type=cameras&jurisdiction=GA" \
  -H "X-API-Key: your_key"
Enter fullscreen mode Exit fullscreen mode

Drop that response directly into Leaflet, Mapbox, or any GeoJSON-compatible tool.

Query truck restrictions along a corridor:

curl "https://api.road511.com/api/v1/truck/corridor?from_lat=41.88&from_lng=-87.63&to_lat=40.71&to_lng=-74.01&buffer_km=5&height=4.2&weight=36" \
  -H "X-API-Key: your_key"
Enter fullscreen mode Exit fullscreen mode

Returns every bridge clearance, weight restriction, and truck route segment within 5km of the Chicago-to-NYC corridor that your truck can't clear.

What's in the Data

This isn't just traffic events. The normalized dataset includes:

  • 10,000+ live cameras with direct image/stream URLs
  • Dynamic message signs with current displayed text
  • 621,000 bridges from the FHWA National Bridge Inventory (height, weight, condition)
  • 479,000 truck route segments from the FHWA National Network
  • 100,000+ EV charging stations from NREL
  • Weather stations with real-time RWIS readings
  • Rest areas and truck parking with amenity data
  • Seasonal weight restrictions (spring load limits from multiple states)
  • Road conditions (surface state, chain requirements)
  • Work zones, ferries, weigh stations, service vehicles, and more

Tech Stack

  • Go — chi router, pgx v5, slog
  • PostgreSQL + PostGIS — spatial queries, GeoJSON generation, corridor intersection
  • Redis — response caching, feature detail caching (nil-safe, optional)
  • Vue 3 + Leaflet — live traffic map

Try It

The API is live with a free tier (no credit card):

If you're building anything with traffic data — fleet routing, navigation, insurance risk, smart city dashboards — I'd love to hear what data you need. The hardest part isn't the code, it's knowing which 511 systems have which data in which format. After 57 jurisdictions, I've got a pretty good map.

Top comments (0)