POC Custom Street View for Vienna

TLDR

A POC of a Street View client using open data: https://webkappa.karten.wien

Intro

In 2020 my beautiful hometown - the City of Vienna, Austria ("Stadt Wien") completed mapping the streets with Lidar Scanners and Cameras mounted on cars.

The good thing: The collected data is completely Open Data and so you can create your own "Street View" based on it, completely independent from Google. Using the laser scanner data even more useful things can be created, the City of Vienna will use it e.g. for traffic planning in the future.

I tried to create a "Street View" client based on the provided images.

Data

The images where taken by 6 cameras, all in all ~250 megapixels.

Data can be requested via an online tool - after some hours or days of internal processing, you get a link of a .tar file containing images and meta data:

The file image_meta.txt is a TSV of all images taken for the requested set:

It contains the following columns (unused columns are omitted):

• trajectory_id ID of trajectory (part of folder name)
• sensor_id ID of the camera (essential to get the direction of the camera/part in the cubemap - see later)
• x_m/y_m Latitude and longitude/position of the car (projection EPSG:31256)
• rz_rad Z-Rotation = facing of car in radiants

The photos are stored in way, that a "cubemap" consisting of 6 single photos can be created (negative X,Y,Z and positive X,Y,Z) - the "sensor" = camera used are mapped to the cubemap position as follows:

• sensor id ending in 0: positive Y
• ending in 1: positive Z
• ending in 2: positive X
• ending in 3: negative Z
• ending in 4: negative X
• ending in 5: negative Y

The unfolded cubemap would look like this:

Each photo has a resolution of 7130x7130 and has about 5MB, thus the complete cubemap has up to 60MB. So even about 100 meters of street consists of about 2 GB of raw photo data (one cubemap every ~ 3 meters). That's why for this POC the images are scaled down to 2048x2048 later.

Compiling the data

Enough of theory, let's get physical

A node.js was created to do the following:

• Retrieve list of images from image_meta.txt TSV
• Scale images down to 2048x2048 using graphicsmagick and rename to a unique filename (<hash of position>_{p,n}{x,y,z}.jpg)
• Transform position of cubemap to EPSG:4326
• Create a JSON file for each cubemap to store meta data (e.g. facing)
• Save position, image filenames to postgres/postgis database

The backend

Now that we have the data in place, there's a need of a small backend to provide the frontend with 2 sets of information

• Get all possible positions to mark them on a map
• Get the nearest position based on a given coordinate

As we have all the information for that in our database, a small express based backend was set up fast.

• One API route just queries SELECT array[st_y(position), st_x(position)] as ll FROM cubemaps and returns the positions.
• The second route basically does the some, but orders by distance to the given position using st_distance

The frontend

A full screen map based on leaflet was set up. It initialy retrieves information about all possible positions and displays it as blue circles using L.circle.

A click on the map gets the nearest position from the backend and opens a panorama viewer based on three.js and panolens.

A small map in the bottom right corner shows the current facing of the panorama viewer and also all other possible positions within 500 meters.

Navigation is done by clicking the positions there - you can also navigate by clicking inside the cubemap - but that's not done yet and kind of buggy right now.

Where can I see it?

Here (It's german, sorry): https://webkappa.karten.wien

Still todo

• Cleanup, cleanup, refactor, refactor
• Think about mobile version
• Find memory leaks (switching between cubemaps is done pretty dumb)
• Import more Cubemaps (Unfortunately the download tool seems to be overloaded currently)
• Make navigation within the panorama possible (like Google does it...)

Thanks!

• Creator and maintainers of leaflet
• Creator and maintainers of panolens.js
• Anita Graser for exploring the sample data of 2020 blog article
• The City of Vienna for making this data open!