From Google Maps to MapBox

#webdev #mapbox #maps

Hi. Over the past two years, I have been working with maps and I want to share some knowledge writing maps from the ground up for a large real estate project.

The Problem 😏

This was our map as of two years ago. All of the data would load at page load and Google Maps would render basic pins and polygons.

The first problem with this implementation was that we couldn't render thousands of markers at the same time. The size of the data to load on the client was over 10MB.

The second problem was the lack of UI performance - rendering speed, inertia when dragging, and smoothness while zooming.

And our dream was to render 3D buildings 🏢

Attempt #1 – Stick to Google Maps 🤔

To solve the first problem we considered clusterization. It looked less like a solution, and more like "hey, we are not able to render all the data you want to see, so here is your circle with a number, and try to zoom in more." Admission of defeat. Not an option 😎

So we moved towards KML. Despite the fact that KML is an XML format, debugging can be time-consuming: you send a KML payload to the Google server and get nothing in the response. No errors, just blank tiles. Having assembled a working prototype, we got the following:

We thought "cool, now we’re only rendering markers in the viewport as many as we want!" But with this came two new problems:

Flickering while zooming. Any new zoom level is a new request to the Google tile server. Tiles are raster images so they are reloaded, not scaled.
Markers were blurry on high-resolution displays. And there was no way to fix this problem.

Attempt #2 – MapBox 😳

While searching for solutions we found MapBox. Unlike Google Maps, MapBox works with vector tiles and provides a very flexible API with mapbox-gl-js.

Instead of downloading all of the GeoJSON data to the client, we could now prepare our data to be distributed in the vector .pbf tiles format:

Convert GeoJSON file to .mbtiles with tippecanoe
Serve this file with a tile server - MapBox Studio or tileserver-gl, if you prefer a self-hosted solution.

The result exceeded all expectations 😱

Smoothly. Tiles are rendered without any flickering between zoom levels.

Rendering 3D Models 🙀

Not so long ago MapBox added support for Custom Layer, with a low-level API to WebGL context. This makes it possible to render 3D models.

As of now, nothing like this can be done with Google Maps 🤷‍♀️

The Magic of Extrusion 🧙‍♂️

Extrusion is another powerful feature of mapbox-gl-js. It helps us render 3D buildings on the fly. Unlike the 3D model example above, fill-extrusion works without any .glb or .obj files, rendering is based on provided GeoJSON properties. For example, the height property from the OpenStreetMaps database. Since we render tile by tile on the fly, this approach turned out to be very scaleable. The property fill-extrusion comes in handy when we want to render a single apartment or even an entire floor 😯

This example is not interactive, without hover and click events, but they are super easy to add.

Not Enough Minerals 💎

MapBox Studio and MapBox as a service are brilliant. Without any deep knowledge of what’s under the hood of the map or tile server, it helps create fast beautiful maps. But this service costs a pretty penny. So if you explore using it, it’s important to weigh out whether or not it will have a direct impact on your bottom line. In our case, it didn’t. 💸

But it is possible to rid yourself of the payments. Let’s a look at how to do that.

The first step is to move tile distribution to our end. For both map tiles and data tiles (if you do not render GeoJSON data directly).

First, you need to create a Style JSON file. It has at least 3 important fields you need to host by yourself:

Sprite - all icons that are used on the map. Take a look at the Maki Icons.
Glyphs - fonts. All fonts that are used on the map must be converted to .*pbf. For the most common font families, you check out fonts.pbf
Sources - the most challenging part. First, you need to find out OpenStreetMaps .mbtiles. There are two main options that we have found: buy them at OpenMapTiles (the latest data is not free for commercial use) or download OSM .pbf data at Geofabrik (free and updated quite often), and convert it to tiles with openmaptiles.

After hosting all of these style assets (I just have to mention maputnik - worthwhile layers editing tool) your map can be rendered by mapbox-gl-js without any Access Tokens.

And then it’s time to think about how to render the vector data on top of it.

What have we tried?

Export MySQL data to GeoJSON file, convert it to *.mptiles file with tippecanoe and upload it to MapBox Studio.
Once we did this, we had to give up with Studio, we set up tileserver-gl from MapTiler. And it’s great – all we have to do is set up a single config file.
Coming up in a minute 🙊

So we ended up with a cheap, fully self-hosted map that makes no concessions to the ones you have to pay for 💰

Tiles on The Fly 🚀

In the end, our weakest pipeline was a dynamic data update - MySQL => PHP => GeoJSON => tippecanoe => MBTiles => upload tiles to tileserver-gl => reload Docker container 🤯

MARTIN comes to the rescue here ✨

It is a tile server written in Rust and works on top of PostGIS. I don't know who came up with the idea, but it is genius.

You just have to create a database table (we were using PostgreSQL), fill in geo data, and point MARTIN to serve this table as a source. It works out of the box, without any config files. If you need to perform geospatial operations at the moment of tile creation (e.g. do not render geometries that intersect) then function sources is the right place to do it. With a caching proxy (e.g. NGINX-based) in front of MARTIN, you will get a blazingly fast tile server ⚡

The TL;DR 💡

Switching from raster to vector tiles decreases loading time - often vector tiles are lighter with the ability to overzoom.
Vector maps are easy to customize with MapBox Studio or maputnik.
With OpenStreetMaps it is easy to add or update data like streets or POI, unlike Google Maps.
HERE have Satellite tiles with a significant free of charge limit.
It’s not a good idea to use GeoJSON source for heavy data visualization. Serve it with tiles and let your users download only the data, that is visible in the viewport.
Best not to hold a lot of data in property keys of GeoJSON Feature, it will make tiles heavier. If you need any properties after a click, you can load it later with an XHR request.
If you can afford to pay for MapBox service, they certainly deserve every dollar. But if not, a step by step self-hosted tile server is not hard to implement.