During the past month, I have created a specific style for generating cycling maps, using only publicly available data.
Goal
In July and August, I was on holyday in Poland, Slovakia and Hungary. For finding the way in those countries, I had a map (good old paper). I was cycling from camping to camping, so a good map was required. Unfortunately, Poland and Hungary had no good maps made for the purpose of cycling. That means that sometimes I could not find the way, and needed to use a mobile phone (mostly using the app Maps.Me with data of OpenStreetMap).
This problem stayed in my head. I am very enthusiastic about projects like OpenStreetMap, which collects and publishes the data for free (license: ODbL). The data quality is usually great, and the map details are far greater than for example Google Maps, Apple Maps or TomTom. Also, the data is updated very quickly if something changes, all by volunteering contributors.
The goal of this project is as follows: “Generate a printable map of any region in the world, using only publicly available data and tools, usable for cycling.” The process described in the following sections is the end result of this project. It took some weeks to learn about the technology, tooling and data. Then, by trial and error, the resulting map was improved step by step.
Stack
To reach the project goal, some tooling is needed. Fortunately, many people have gone before me that have a lot of knowledge about generating maps. I collected this knowledge from blogs, guides and websites.
For this project, I used the following stack:
- Linux Subsystem for Windows. My main machine is Windows, and most of the tools do not run on Windows. Also, command-line scripting is not that easy in Windows. The command-line, apt, and the Linux ecosystem is essential for getting and running the needed tools.
- Mapnik 2.x. C++ and Python bindings are available, so it is more a library than a program. There is a newer version of Mapnik (3.x), but the Python bindings do not exist or have no documentation (or both).
- Python 2.7. For Mapnik 2.x the bindings are for Python 2.7. This is pre-installed in every mainstream Linux installation.
- GDAL. These libraries are used for generating the contour lines and hillshade for the height data. There is a command-line interface with very good online documentation.
- Postgres (with GIS extensions). A Postgres database has extensions (PostGIS) which allow loading spatial data (with locations), and generating efficient indices for querying spatial data. This is essential for processing the data quickly from the database to the map.
Getting the data
The data comes from two places. The content of the map (roads, features, land use, etc.) comes from OpenStreetMap. That data has been refined (sorted, filtered and grouped) by GeoFabrik. This means that ambiguity of some data is removed, and the data is labeled in a structured way. Also, they publish a nice document describing the structure of the data. For each country (and sometimes part of the country) a separate zip file must be downloaded.
The other required data is height data. This data is used for generating contour lines and ‘hillshading’ (if there is a steep hill, it is colored a darker shade on one side). The height data of the world is available from a number of different sources, but I get it from here. The data is measured by NASA satellites of the Shuttle Radar Topography Mission (SRTM) project.
Finally, I use the OpenStreetMap data a second time. This time, it is downloaded from another source. For the borders of countries, the GeoFabrik data is not usable (their free service doesn’t have this data, but their premium service does). The boundaries are downloaded from wambachers-osm.website instead.
Processing the data
A common way for spatial data (any data with a location attached to it) to be formatted is shapefiles. A shapefile describes vector data, a point, line or polygon, possibly with attributes. For example, a shapefile containing roads will contain lines with the shapes of the roads, and attributes with the name of the road and the speed limit. The lines can be used on the map, and the attributes can be used to style the data differently for each type of road. Each set of data has multiple files, each with a different responsibility. Most GIS applications and scripts can work with shapefiles.
For our use case shapefiles would work, but they are limited in their performance and possibilities. For example, if I want to create a map that spans multiple areas or countries, I would have to change the script to add more data sources. That is why all shapefiles are imported into a Postgres database. Shapefiles with the same kind of data are imported into the same table, and can be queried once to find all the data of that type for the map.
The height data also has to be processed. The height data does not come as a shapefile (it does not have a vector shape), but rather as a raster file. It can be seen as a large grid of ‘pixels’ having some value. In addition to the value, each pixel can have attributes, just like shapefiles. The SRTM3 data has a file per latitude and longitude degree squared.f
The height data is processed in two ways. First of all, the contour lines are generated from the height raster. The GDAL libraries are used for this. The interval of the contours can be specified (I use every 20 meters). The result is a shapefile, containing lines for each contour, with a height attribute which can be used to style the contour on the map.
gdal_contour -i 20 -snodata -32768 -a height N49E019.hgt N49E019.shp
In addition to contour lines, I also generate the hillshading. This is also done with the GDAL libraries, with the hillshade command. Note that the result is not a shape file, but another raster file. The raster file contains the ‘shade’ of the hill, a value between 0 and 1 depending on the orientation and steepness of the slope. The defaults work well: the light source comes from the north west (default for many maps), at an angle of 45 degrees.
gdaldem hillshade N49E019.hgt N49E019.shade
Creating the map style
Once the data is ready, the map can be generated by making style for each part of the data. The description that follows took many steps, and manual checking of the rendered result. This styling is in no way perfect, but at least optimized for cycling.
Each section below can be seen as a layer (actually, they are the names of the Mapnik layers). The first layer is on the bottom, and every following layer is printed ‘over’ it. Sometimes I use a transparent layer to add information to the map, while keeping the information under it visible.
Background: land use
The background of the map will be white, with an overlay of forest (light green).
Contours and hillshade
Over the background, the hillshade will be printed (with a low opacity), such that hills become visible. The contours are added to that, such that hills and mountains will become visible by rings of contour lines. Normal contour lines are every 20 meters, while thicker contour lines are every 100 meters.
Cities and military
Two special types of land use are added after the hillshading. Cities (with residential land use), and military areas are marked especially. Cities need to be visible on the map: they give structure to the road network and names. Military areas are off-limits for civilians.
Borders
Borders of countries need to be visible. They get a transparent thick background such that they are easily visible in the large-scale structure of the map.
Water
Water is important to get a good overview of the area (together with forest areas). Lakes give structure to the map, and streams and rivers also give information on hills, mountains and valleys. Rivers and streams always flow perpendicular to contour lines, and parallel to valleys.
Trains
Train tracks show connections between large cities and towns. They are visible in the landscape because they are long and straight. Stations are often listed on road signs.
Roads (and bridges)
Most of the work has been done on roads. They sound easier than they are, because there are many types and they connect with each other.
The general idea for roads is “more important is on top”. That means small paths first, then dirt roads, then general small roads, then tertiary roads (small roads with a name), then secondary roads, primary roads and trunk roads (between a primary road and motorway, the use differs per country). Finally, motorways. And to make things hard: cycleways need to be visible over every kind of road, but not too invasive.
But that is not always the case:
- For all roads, the borders should always be on the bottom, and the roads themselves should be on top.
- If a tertiary road contains a bridge over a motorway, it has to be displayed over the motorway (even the border!).
- If there are ‘link’ segments, for example going from a secondary road to a trunk road, the crossing has to contain the link sections on the bottom, and the main roads on top.
These requirements make a beautiful list of (some repeated) styles for roads and their borders.
Names of cities, roads, and contours
One of the primary features of a map is the names of the things on the map. Cities are the most important: they give information on where roads lead. All names of cities, villages, hamlets and even suburbs of cities are printed.
But roads also have names (references). They are often listed on road signs. If there is space on the map, the names are printed along the road.
Finally, the contour line heights. They have little priority, but can give context how high the surroundings are. If there is any space left on the map, the heights of the 100-meter contour lines are also printed.
Places of interest (icons)
The last layer of the map are icons of places of interest. Especially in a city they can be good references (think of churches with a high tower). I have selected a list of places of interest that do not clutter the map too much, and give enough information to be useful. The map contains places of worship (churches, mosques, synagogues, etc.), shops, bicycle repair shops, campings and castles.
Each icon has a circle background (partly transparent) to make it more visible in the whole.
Creating the map
In Mapnik the layers above can be defined using a Python script. For each layer a data source is used, and the features of that data source are printed on the map using some style. The features can be filtered such that a style is only applied to some features. For example: the data source for roads contain all roads. It has many styles, for example one for primary roads which are a bridge.
The Python script creates all layers, and Mapnik generates an XML of the description of the map. This XML file is portable and can be used directly by anyone, without running the Python script.
Using the map description (either from the XML file of from the Python script), Mapnik can output many kinds of images. It has many options for JPG and PNG files, but it can also create PDF files. That is useful for vector features with infinite resolution (good for printing).
Scripting it all together
In addition to the Python script for creating the map, I have also made a bash script which downloads the data required for the map in an automated way. For some areas multiple files have to be downloaded and processed, and this is cumbersome to do by hand.
The bash script will download all the height data, process it into hillshade and contour lines and add it to the Postgres database. Then it will download the country borders of the selected countries and add them to the Postgres database. Finally, the map data is downloaded and also added to the database.
This makes running the scripts for a different set of countries easy: only the names of the countries, and the latitude/longitude ranges of the map are needed.
Conclusion
This is the end result of the map.
For me, this is a long time wish which has come true. I just needed a problem for which generating a custom map was the solution. I plan to use a printed version of my own map the next time I go on a cycling holiday, and test it in the real world.
The source code and scripts can be found on in the GitHub repository. Feel free to leave comments and suggestions for improvement!
Generating a custom cycling map with Mapnik and Carto
View the blogpost describing this project at https://dev.to/hiddewie/creating-a-custom-cycling-map-3g2a.
Cartography and features
Getting started
There are three scripts in this repository:
-
download.shDownloads the required data into a Postgres database for the map.
-
generate.pyA Python script which will generate the map for a Mapnik configuration. The Mapnik configuration can be generated by running Carto against
project.mml. -
bounds.pyA Python script which will output a list of bounding boxes that will fit the configured page size and bounding box perfectly.
See the environment variables which can be configured for the scripts below.
Manually
Make sure you have a running Postgres database, with a gis schema with GIS extensions enabled.
Run the command
./download.shto download the data and insert it into the database. Make sure that the environment variables listed below are set.
Run the command
carto project.mml …
Top comments (7)
Thank you!
I do not use an electronic map for two reasons. On the bike, anything electronic can become wet if taken out of a waterproof pocket (which happens often). More importantly, the battery would be empty in a day. During my holiday I do not want to be dependent on electricity, so I don't charge my phone unless I really need to, as infrequently as possible.
Hello, you should have a look at cyclosm.org ;)
Hi!
I did't know that service, thanks. Many things mentioned in the article are also on that map. But some information is missing that I find useful (mostly Places of Interest), see the attached screenshot with the same bounding box as the cover photo of this post.
It only exists since september, there is still lot to improve.
For places of interest I guess they can viewed in higher zoom.
If you wish you can create tickets for your requirements on github, we will try to solve them.
I think the exact issue is outlined in github.com/cyclosm/cyclosm-cartocs...
Viewing map content using a web browser is very different than using a printed paper map.
If I find issues for the web version I will create issues in Github.
Hello Hidde,
Congratulations man, Your map is beautiful !
I would like to use it as a basemap in my app. Are you planning to publish it as a web service ?
Best,
Alexandre
Hi Alexandre,
Thanks! I did not yet consider publishing the map as web service map tiles. At the moment the map is only useful for printing, as there is no zoom, and the data is downloaded from many sources before making the map. Running the script as a web service would be possible, but different choices would have to be made for updating the data and automatically rendering the tiles from that information.
Then the tiles would still only be useable on one zoom level.
For now there will be no web service. I might make one in the future if there is enough interest.