DEV Community: Léo Leplat

Create a temperature map with Mapbox and OpenWeatherMap

Léo Leplat — Mon, 04 Oct 2021 20:19:52 +0000

A temperature maps shows the current temperature in a color scale, from blue (cold) to red (hot). In this tutorial, I will show how to create a web-based temperature map using JavaScript, Mapbox, and OpenWeatherMap.

To follow this tutorial, you just need some knowledge in web development (HTML, CSS, Javascript).

We will:

Create a map using Mapbox.
Fetch temperature data using OpenWeatherMap.
Use a custom Mapbox layer to display a color scale of the temperature.

The result is displayed here and the source code is accessible here.

1. Setting up the project

The first step is to set up the web page. Since it's only a web application running on a client side, we just need a HTML and a JavaScript file.

Create an empty map.js file. This file will contain the map creation and the retrieval of weather data. Then, create an index.html file with the following code:



<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8">
        <title>Temperature map</title>
        <style>
            body { margin: 0; padding: 0; }
            #map { position: absolute; top: 0; bottom: 0; width: 100%; }
        </style>
    </head>
    <body>
        <div id="map"></div>
        <script src="map.js"></script>
    </body>
</html>

We just add a map div that takes the entire available space. It will contain the map.

2. Creating the map

To display a map in our web page, we need a map provider. Many exist, but we will use Mapbox in this tutorial because it is highly customizable and free up to 50,000 map load per month (which means that 50,000 monthly requests on your web page can be made).

First, create a Mapbox account. Then, go to your profile page and copy your default public token. This is used to identify yourself when requesting map data.

We can now add the map on our web page. Mapbox provides SDK for different platforms (Android, iOS, Unity) so since we want to create a web application we are going to use Mapbox GL JS. This SDK can be installed via a npm module, or by simply including the JavaScript and CSS files in the <head> of our HTML file:



<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8">
        <title>Temperature map</title>
        <link href="https://api.mapbox.com/mapbox-gl-js/v2.3.0/mapbox-gl.css" rel="stylesheet">
        <script src="https://api.mapbox.com/mapbox-gl-js/v2.3.0/mapbox-gl.js"></script>
        <style>
            body { margin: 0; padding: 0; }
            #map { position: absolute; top: 0; bottom: 0; width: 100%; }
        </style>
    </head>
    <body>
        <div id="map"></div>
        <script src="map.js"></script>
    </body>
</html>

In the map.js file, add the following lines:



mapboxgl.accessToken = 'your_mapbox_access_token';      // past your default public token

const map = (window.map = new mapboxgl.Map({
    container: 'map',                           // 'map' is the id of the HTML div
    style: 'mapbox://styles/mapbox/light-v10'
}));

This is the simplest way of creating a map. The style parameter will determine how your map will look like. Mapbox provides default styles, but you can also create your own by using Mapbox Studio.

Other parameters, such as the inital zoom and geographical centerpoint of the map, can be specified. You will find all available parameters on the API documentation.

3. Gathering weather data

The next step is to collect data on the current weather by the mean of a weather API. Several free providers exist. For this tutorial, I chose to use OpenWeatherMap.

OpenWeatherMap is an online service that provides among other things global weather data. To use this API, you will have to create an account and get your API key. The free plan includes 60 calls to the API per minute.

The documentation for getting the current weather data is available here. In this tutorial, we will fetch the current temperature in degree Celsius by geographic coordinates. Therefore, the URLs will be like: https://api.openweathermap.org/data/2.5/weather?units=metric&lat=latitude&lon=longitude&appid=apiKey.

Since the goal of this tutorial is to create a temperature map, we want to fetch the temperature at different locations that cover the entire globe. In geography, latitude goes from -90° to 90°, and longitude from -180° to 180°. However, Mapbox uses the Web Mercator projection, which projects the poles at infinity, so we cannot see the poles. Therefore, let's define some points going from -80° to 80° for the latitude.



const startingLatitude = -80;
const startingLongitude = -180;
const endingLatitude = 80;
const endingLongitude = 180;
const n = 10;                   // n² is the number of points

const points = [];
for (let i=0; i < n; i++) {
    for (let j=0; j < n; j++) {
        points.push({
            lat: startingLatitude + i * (endingLatitude - startingLongitude)/n,
            lng: startingLongitude + j * (endingLatitude - startingLongitude)/n,
            val: 0              // val will store the temperature
        })
    }
}

Each element of the points array is an object with a latitude, a longitude, and a value corresponding to the temperature for that location. We set the default temperature to 0, so now let's get the real value:



const startingLatitude = -80;
const startingLongitude = -180;
const endingLatitude = 80;
const endingLongitude = 180;
const n = 10;

(async() => {
    const points = [];
    for (let i=0; i < n; i++) {
        for (let j=0; j < n; j++) {
            points.push({
                lat: startingLatitude + i * (endingLatitude - startingLatitude)/n,
                lng: startingLongitude + j * (endingLongitude - startingLongitude)/n,
                val: 0
            })
        }
    }
    // Create the URLs
    const baseUrl = "https://api.openweathermap.org/data/2.5/weather?units=metric&lat=";
    const apiKey = 'your_weather_api_key';
    const urls = points.map(point => baseUrl + point.lat + "&lon=" + point.lng + "&appid=" + apiKey);
    // Fetch the weather data
    const weathers = await Promise.all(urls.map(async url => {
        const response = await fetch(url);
        return response.text();
    }));
    // Set the temperature
    points.forEach((point, index) => {
        point.val = JSON.parse(weathers[index]).main.temp;
    })
})();

Three steps are done here:

First, we need to define the URLs which will be used to communicate with the weather API. We use the model URL presented above and we set the latitude and longitude of the previously defined points. Remember to set your API key available here.
Then, we perform the requests with the fetch(url) function. Since it's an asynchronous operation and we need to wait for it to finish, we use the keyword await. The function Promise.all() is used to perform all the requests in parallel, and, with the keyword await, we wait for all requests to finish. await has to be inside an async function, so we surround the whole code with (async() => {})();. If you don't understand this part, here is the equivalent code if we had only one URL to fetch:



  (async() => {
      const response = await fetch(url);
      const weather = await response.text();
  })();

We fetch a response from an URL, and then we store the content of the response in the weather variable. These two operations are asynchronous, so we need to use await.

We have now a weathers array containing some weather data for all our points, so we just need to extract the temperature by using the documentation.

We have now an array of locations with associated temperatures.

4. Creating the color scale

We can now create the color scale by providing the array of points to the map.

Each element of a Mapbox map is part of a layer: there is a layer for roads, tunnels, parks, countries, for every type of label... So the color scale we want to add to the map will be implemented by a layer defined by us.

Mapbox natively provides a heatmap layer. This is close to what we want, but not exactly. Take a look on these pictures:

The Mapbox heatmap layer can represent the density of points in an area (like on the first picture). The color doesn't depend on the value of a point but on the density of points (actually the value can change the color but it is mainly the density that will affect the color).

For our temperature map, we want a heatmap that can define a color to any location by making an average of the values of the surroundings points (like on the second picture).

I created a way of representing this type of heatmap. The project is available on GitHub here. Download the interpolateHeatmapLayer.js file, place it in your working folder and include it before the map.js script:



<body>
    <div id="map"></div>
    <script src="interpolateHeatmapLayer.js"></script>
    <script src="map.js"></script>
</body>

You can now call the interpolateHeatmapLayerinterpolateHeatmapLayer.create() function to create the layer, and add it to the map. Here the whole script:



mapboxgl.accessToken = 'your_mapbox_access_token';

const startingLatitude = -80;
const startingLongitude = -180;
const endingLatitude = 80;
const endingLongitude = 180;
const n = 10;

(async() => {
    const points = [];
    for (let i=0; i < n; i++) {
        for (let j=0; j < n; j++) {
            points.push({
                lat: startingLatitude + i * (endingLatitude - startingLatitude)/n,
                lng: startingLongitude + j * (endingLongitude - startingLongitude)/n,
                val: 0
            })
        }
    }

    const baseUrl = "https://api.openweathermap.org/data/2.5/weather?units=metric&lat=";
    const apiKey = 'your_weather_api_key';
    const urls = points.map(point => baseUrl + point.lat + "&lon=" + point.lng + "&appid=" + apiKey);

    const weathers = await Promise.all(urls.map(async url => {
        const response = await fetch(url);
        return response.text();
    }));

    points.forEach((point, index) => {
        point.val = JSON.parse(weathers[index]).main.temp;
    })


    const map = (window.map = new mapboxgl.Map({
        container: 'map',
        style: 'mapbox://styles/mapbox/light-v10'
    }));

    // When the map is loaded, create and the color scale layer
    map.on('load', () => {
        const layer = interpolateHeatmapLayer.create({
            points: points,             // use our array of points
            layerId: 'temperature'      // define an ID for the layer
        });
        map.addLayer(layer);
    });
})();

The color scale layer can only be added when the map is loaded, so we add an handler to the load event. The layer expects at least:

An array of points, each point being an object containing a latitude lat, a longitude lon, and a value val.
An ID, which is the unique Mapbox layer name.

Other parameters are:

opacity: a number between 0 and 1 describing the transparency of the color.
minValue: define the value corresponding to the blue color. When it's not defined, this is the lowest temperature of points which is represented by the blue color. There is also maxValue for the red color.
framebufferFactor: number between 0 and 1. In short, if the framebuffer factor is around 0, the computation will be faster but less accurate and the opposite if it's around 1. The default value is 0.3. Take a look at the readme file of the GitHub project if you want to know what exactly this parameter is.
p: a factor affecting the computation of the color, from 0 to around 15. A high value makes the color uniform around each point. Once again, take a look at the GitHub project if you want to know more.
roi: list of coordinates (like points) defining the area of interest, aka the area where the temperature map will be present. By default, the entire world is colored.
valueToColor: function defining the range of color of the temperature map. By default, a low temperature is colored blue, a medium green and a high red.

Right now, the temperature is represented above everything else on the map. Instead, it would be better to have the different labels above the temperature map, so we are going to use the second parameter of the map.addLayer() function. The Mapbox documentation states that this parameter is the "ID of an existing layer to insert the new layer before, resulting in the new layer appearing visually beneath the existing layer". The layers are defined in the style parameter of the map, so we need to find the lowest label layer of the light-v10 style (the one used here).

To do this, go to Mapbox Studio, create a new style, and choose the Monochrome template with the light variation. Then, go to the Layers tab on the left panel. All labels appear, the first ones being the upper layers (appearing above the others on the map). The last layer with label written on it is road-label. Therefore, perform the following modifications to your script:



// ...
(async() => {
    // ...
    map.on('load', () => {
        const layer = interpolateHeatmapLayer.create({
            points: points,
            layerId: 'temperature'
        });
        map.addLayer(layer, 'road-label');
    });
})();

The temperature map is now below the labels.

5. Conclusion

You know now how to create a temperature map with Mapbox and OpenWeatherMap. I wrote this article to provide an example of using the interpolate heatmap layer, but it can also be used for visualizing generic geographical data.

Create a Virtual Machine viewer with Spice-GTK

Léo Leplat — Sun, 29 Aug 2021 15:13:21 +0000

One way to work with Linux virtual machines (VM) is to use Libvirt to manage the VM, and Spice to interact with the VM. The Libvirt API and the Libvirt Application Development Guide show how to create an application with Libvirt. Such guide not being present for Spice, I will show in this tutorial how to create a GTK application in C for interacting and controlling a VM.

To follow this tutorial, you will need:

The Fedora distribution (it should work on other distro but you will have to change the dnf commands).
Some basic knowledge in C and GTK.

1. Creating a VM

The first step is to create a VM running a Spice server. I recommend using the virt-manager software. Plenty of tutorials showing how to install virt-manager and create a VM are available on the Web, so I will not detail this part.

By default, a Spice server is used when creating a VM with virt-manager, but you can check it by opening the VM, going to the Details view and on the Display Spice tab. Here, you should choose the Type as Spice server. If the VM is running, note the port used as we will need it later (by default, it should be 5900).

2. Installing dependencies

To develop this application, we need both the development files of GTK3 (GTK4 is not supported by Spice-GTK) and Spice-GTK.

GTK3 can be installed with:

sudo dnf install gtk3-devel

And Spice-GTK:

sudo dnf builddep spice-gtk
wget https://www.spice-space.org/download/gtk/spice-gtk-0.39.tar.xz
tar -xf spice-gtk-0.39.tar.xz
cd spice-gtk-0.39/
meson builddir && cd builddir
meson compile
sudo meson install

3. Creating the application

Let's now create the application. The final source code is available on Github.

We can begin with a basic window and include GTK and Spice-GTK to make sure everything is installed properly:

#include <gtk/gtk.h>
#include <spice-client-gtk.h>

void activate (GtkApplication *app, gpointer user_data) {
    GtkWidget* window = gtk_application_window_new(app);
    gtk_window_set_title(GTK_WINDOW(window), "VM Viewer");
    gtk_window_set_default_size(GTK_WINDOW (window), 800, 600);
    gtk_widget_show_all(window);
}

int main (int argc, char **argv) {
    GtkApplication *app;
    int status;

    app = gtk_application_new("org.gtk.example", G_APPLICATION_FLAGS_NONE);
    g_signal_connect(app, "activate", G_CALLBACK(activate), NULL);
    status = g_application_run(G_APPLICATION(app), argc, argv);
    g_object_unref(app);

    return status;
}

To compile, we need both gtk+-3.0 and spice-client-gtk-3.0:

gcc main.c `pkg-config --cflags gtk+-3.0 spice-client-gtk-3.0` `pkg-config --libs gtk+-3.0 spice-client-gtk-3.0`

We can now start using the Spice library. The API reference is available here.

The first thing to do is to create a Spice session, pass the connection information to it and connecting to the VM:

#include <gtk/gtk.h>
#include <spice-client-gtk.h>

SpiceSession* spiceSession;

void onClose() {
    // Close the Spice session
    spice_session_disconnect(spiceSession);
}

void activate (GtkApplication *app, gpointer user_data) {
    GtkWidget *window = gtk_application_window_new(app);
    gtk_window_set_title(GTK_WINDOW(window), "VM Viewer");
    gtk_window_set_default_size(GTK_WINDOW (window), 800, 600);
    g_signal_connect(G_OBJECT(window), "destroy", onClose, NULL);
    gtk_widget_show_all(window);

    // Create a Spice session
    spiceSession = spice_session_new();

    // Pass the URI of the VM to the session
    GValue uri = G_VALUE_INIT;
    g_value_init(&uri, G_TYPE_STRING);
    g_value_set_static_string(&uri, "spice://localhost?port=5900");
    g_object_set_property(G_OBJECT(spiceSession), "uri", &uri);
    g_value_unset(&uri);

    // Connect to the VM
    spice_session_connect(spiceSession);
}

int main (int argc, char **argv) {
    GtkApplication *app;
    int status;

    app = gtk_application_new("org.gtk.example", G_APPLICATION_FLAGS_NONE);
    g_signal_connect(app, "activate", G_CALLBACK(activate), NULL);
    status = g_application_run(G_APPLICATION(app), argc, argv);
    g_object_unref(app);

    return status;
}

The URI is of the form spice://hostname?port=XXXX. Here, we connect to a local machine but it could also be a distant machine (if the firewall of the distant host permits it). The port can be found by opening the VM with Virt-manager, going to the Details view and on the Display Spice tab. By default, it should be 5900. It is also possible to access the port with Libvirt. The session must be closed before leaving the application, which is done here by the onClose() function connected to the destroy signal of the window.

When a spice session is connecting, "channels" are created. There is one channel for each type of interaction, for example:

The Main channel, which handles communication initialization (channels list), migrations, mouse modes, multimedia time, and agent communication.
The Display channel, which renders the remote display.
The USB Redirection channel, which can redirect USB devices from the host to the VM.
The Cursor channel, which updates the cursor shape and position.
The Inputs channel, which control the server mouse and keyboard.
And more described in the API reference.

The first channel we are going to use is the Main channel, because it can tell us whether the connection to the VM was a success or not. To do this, we have to connect the session to the channel-new signal. This signal will emit each time a channel is created. Once the Main channel is created, we have to connect it to the channel-event signal. This signal will indicate whether the session successfully connected to the VM.

#include <gtk/gtk.h>
#include <spice-client-gtk.h>

GtkWidget *window;
SpiceSession* spiceSession;

void onClose() {
    spice_session_disconnect(spiceSession);
}

// Called when an event happens to the Main channel
void channelEvent(SpiceChannel *channel, SpiceChannelEvent event, gpointer user_data) {
    // If there is a connection error, we end the application
    if (event == SPICE_CHANNEL_ERROR_CONNECT) {
        printf("SPICE_CHANNEL_ERROR_CONNECT\n");
        g_signal_emit_by_name(window, "destroy");
    }
}

// Called when a new channel is created
void newChannel(SpiceSession *session, SpiceChannel *channel, gpointer user_data) {
    // Get the channel type
    gint channelType; 
    g_object_get(channel, "channel-type", &channelType, NULL);
    printf("%d: %s\n", channelType, spice_channel_type_to_string(channelType));

    // Type 1 corresponds to the Main channel
    if (channelType == 1) {
        // Connect the Main channel to the channel-event signal
        g_signal_connect(G_OBJECT(channel), "channel-event", G_CALLBACK(channelEvent), NULL);
    }
}

void activate (GtkApplication *app, gpointer user_data) {
    window = gtk_application_window_new(app);
    gtk_window_set_title(GTK_WINDOW(window), "VM Viewer");
    gtk_window_set_default_size(GTK_WINDOW (window), 800, 600);
    g_signal_connect(G_OBJECT(window), "destroy", onClose, NULL);
    gtk_widget_show_all(window);

    spiceSession = spice_session_new();

    // Connect the session to the channel-new signal
    g_signal_connect(G_OBJECT(spiceSession), "channel-new", G_CALLBACK(newChannel), NULL);

    GValue uri = G_VALUE_INIT;
    g_value_init(&uri, G_TYPE_STRING);
    g_value_set_static_string(&uri, "spice://localhost?port=5900");
    g_object_set_property(G_OBJECT(spiceSession), "uri", &uri);
    g_value_unset(&uri);
    spice_session_connect(spiceSession);
}

int main (int argc, char **argv) {
    GtkApplication *app;
    int status;

    app = gtk_application_new("org.gtk.example", G_APPLICATION_FLAGS_NONE);
    g_signal_connect(app, "activate", G_CALLBACK(activate), NULL);
    status = g_application_run(G_APPLICATION(app), argc, argv);
    g_object_unref(app);

    return status;
}

The newChannel() function is called each time a channel is created. We can get the type of the channel (main, display, USB redirection...) by looking at its channel-type property. This property is an enum (so a number), but we can get its corresponding string with the spice_channel_type_to_string() function. For example, the enum 1 corresponds to the Main channel.

The channelEvent() function is called each time the state of the main channel changes. We use it here to detect any connection error.

We can now display the screen of the VM. We will use a GTK display widget provided by Spice. It requires the Display channel, so it will be created in the newChannel() function:

#include <gtk/gtk.h>
#include <spice-client-gtk.h>

GtkWidget *window;
SpiceSession* spiceSession;

void onClose() {
    spice_session_disconnect(spiceSession);
}

void channelEvent(SpiceChannel *channel, SpiceChannelEvent event, gpointer user_data) {
    if (event == SPICE_CHANNEL_ERROR_CONNECT) {
        printf("SPICE_CHANNEL_ERROR_CONNECT\n");
        g_signal_emit_by_name(window, "destroy");
    }
}

void newChannel(SpiceSession *session, SpiceChannel *channel, gpointer user_data) {
    gint channelType; 
    g_object_get(channel, "channel-type", &channelType, NULL);
    printf("%d: %s\n", channelType, spice_channel_type_to_string(channelType));

    if (channelType == 1) {
        g_signal_connect(G_OBJECT(channel), "channel-event", G_CALLBACK(channelEvent), NULL);
    }
    // Type 2 corresponds to the Display channel
    else if (channelType == 2) {
        // Get the channel ID of the Display channel
        gint channelId; 
        g_object_get(channel, "channel-id", &channelId, NULL);

        // Create the display widget
        SpiceDisplay* spiceDisplay = spice_display_new(session, channelId);

        // Add the display widget to the main window 
        gtk_container_add(GTK_CONTAINER(window), GTK_WIDGET(spiceDisplay));
        gtk_widget_show_all(window);
    }
}

void activate (GtkApplication *app, gpointer user_data) {
    window = gtk_application_window_new(app);
    gtk_window_set_title(GTK_WINDOW(window), "VM Viewer");
    gtk_window_set_default_size(GTK_WINDOW (window), 800, 600);
    g_signal_connect(G_OBJECT(window), "destroy", onClose, NULL);

    spiceSession = spice_session_new();

    g_signal_connect(G_OBJECT(spiceSession), "channel-new", G_CALLBACK(newChannel), NULL);

    GValue uri = G_VALUE_INIT;
    g_value_init(&uri, G_TYPE_STRING);
    g_value_set_static_string(&uri, "spice://localhost?port=5900");
    g_object_set_property(G_OBJECT(spiceSession), "uri", &uri);
    g_value_unset(&uri);
    spice_session_connect(spiceSession);
}

int main (int argc, char **argv) {
    GtkApplication *app;
    int status;

    app = gtk_application_new("org.gtk.example", G_APPLICATION_FLAGS_NONE);
    g_signal_connect(app, "activate", G_CALLBACK(activate), NULL);
    status = g_application_run(G_APPLICATION(app), argc, argv);
    g_object_unref(app);

    return status;
}

To create the display widget, we need the channel-id property of the Display channel.

The Inputs channel will automatically redirects the mouse and keyboard to the VM, and the Cursor channel will automatically updates the cursor shape and position, so we have now a fully working viewer.

Finally, I will show how to redirect USB devices to the VM. Spice provides a widget for this, the Spice USB device selection widget. I will put it inside a new window and display it when the display widget is launched:

#include <gtk/gtk.h>
#include <spice-client-gtk.h>

GtkWidget *window;
SpiceSession* spiceSession;

void onClose() {
    spice_session_disconnect(spiceSession);
}

void channelEvent(SpiceChannel *channel, SpiceChannelEvent event, gpointer user_data) {
    if (event == SPICE_CHANNEL_ERROR_CONNECT) {
        printf("SPICE_CHANNEL_ERROR_CONNECT\n");
        g_signal_emit_by_name(window, "destroy");
    }
}

void newChannel(SpiceSession *session, SpiceChannel *channel, gpointer user_data) {
    gint channelType; 
    g_object_get(channel, "channel-type", &channelType, NULL);
    printf("%d: %s\n", channelType, spice_channel_type_to_string(channelType));

    if (channelType == 1) {
        g_signal_connect(G_OBJECT(channel), "channel-event", G_CALLBACK(channelEvent), NULL);
    } else if (channelType == 2) {
        gint channelId; 
        g_object_get(channel, "channel-id", &channelId, NULL);
        SpiceDisplay* spiceDisplay = spice_display_new(session, channelId);

        gtk_container_add(GTK_CONTAINER(window), GTK_WIDGET(spiceDisplay));
        gtk_widget_show_all(window);

        // Create a window
        GtkWidget* USBWindow = gtk_window_new(GTK_WINDOW_TOPLEVEL);
        gtk_window_set_title(GTK_WINDOW(USBWindow), "USB redirection");

        // Put the USB widget inside the new window and display it
        GtkWidget* USBWidget = spice_usb_device_widget_new(session, "%s %s %s at %d-%d");
        gtk_container_add(GTK_CONTAINER(USBWindow), GTK_WIDGET(USBWidget));
        gtk_widget_show_all(USBWindow);
    }
}

void activate (GtkApplication *app, gpointer user_data) {
    window = gtk_application_window_new(app);
    gtk_window_set_title(GTK_WINDOW(window), "VM Viewer");
    gtk_window_set_default_size(GTK_WINDOW (window), 800, 600);
    g_signal_connect(G_OBJECT(window), "destroy", onClose, NULL);

    spiceSession = spice_session_new();

    g_signal_connect(G_OBJECT(spiceSession), "channel-new", G_CALLBACK(newChannel), NULL);

    GValue uri = G_VALUE_INIT;
    g_value_init(&uri, G_TYPE_STRING);
    g_value_set_static_string(&uri, "spice://localhost?port=5900");
    g_object_set_property(G_OBJECT(spiceSession), "uri", &uri);
    g_value_unset(&uri);
    spice_session_connect(spiceSession);
}

int main (int argc, char **argv) {
    GtkApplication *app;
    int status;

    app = gtk_application_new("org.gtk.example", G_APPLICATION_FLAGS_NONE);
    g_signal_connect(app, "activate", G_CALLBACK(activate), NULL);
    status = g_application_run(G_APPLICATION(app), argc, argv);
    g_object_unref(app);

    return status;
}

The second argument of the spice_usb_device_widget_new() function is a string describing each device inside the Spice USB device selection widget, with the following format:

First %s: manufacturer.
Second %s: product.
Third %s: descriptor (a [vendor_id:product_id] string).
First %d: bus.
Second %d: address.

4. Conclusion

You can now create a simple VM viewer with Spice and GTK. More advanced options are described in the API reference.

To create an application able to fully manage and interact with virtual machines, I recommend using the Libvirt API along with Spice-GTK.

Create a C/C++ app with the ACR122U (Linux)

Léo Leplat — Fri, 27 Aug 2021 15:59:07 +0000

The ACR122U NFC Reader is a PC-linked contactless smart card reader/writer. In this tutorial, we are going to see how to create a simple C/C++ application running on Linux that communicates with the ACR122U. This application will be used to send and retrieve data from the MIFARE Ultralight and MIFARE Classic 1K tags.

To follow this tutorial, you will need:

An ACR122U.
A contactless smart card following the ISO/IEC 14443 Type A specification (like the MIFARE Ultralight and MIFARE Classic 1K tags).
A Linux distribution (this tutorial was tested on Fedora and Ubuntu but it should be working on any other distro).
Some basic knowledge in C.

1. Installing the driver

The first step is to install the driver of the ACR122U. This step is needed for both developing and running the application we are going to create.

Let's get to the official website, on the Downloads section.

Two downloads are available for Linux:

PC/SC Driver Package, which contains packages for different distributions (Debian, Fedora, Ubuntu, Raspbian...). This is the easiest way to install the driver if you are using one of these distributions.
PC/SC Drivers, which contains the sources of the driver. Here, it's a bit more complex because we have to build the driver before installing it, but it can work on any Linux distribution.

I will show how to build and install the PC/SC Drivers. First, download and uncompress the sources:

wget https://www.acs.com.hk/download-driver-unified/12030/ACS-Unified-Driver-Lnx-Mac-118-P.zip
unzip ACS-Unified-Driver-Lnx-Mac-118-P.zip
cd ACS-Unified-Driver-Lnx-Mac-118-P/

A README file indicates what is required to build the sources:

pcsclite 1.8.3 or above
libusb 1.0.9 or above
flex
perl
pkg-config

In addition, you will need to install these two other packages:

pcsclite-devel
libusb-devel

So you should install all of these packages first. On Fedora and Ubuntu, the command would be:

# Fedora
sudo dnf install pcsc-lite libusb flex perl pkg-config pcsc-lite-devel libusb-devel

# Ubuntu
sudo apt-get install pcscd libpcsclite1 libusb-1.0-0 flex perl pkg-config libpcsclite-dev libusb-1.0-0-dev

Then we can uncompress, configure, build, and install the sources:

tar -xf acsccid-1.1.8.tar.bz2
cd acsccid-1.1.8/
./configure
./configure --disable-dependency-tracking # for Ubuntu
make
sudo make install

Two more steps are needed. First, you need to make sure that the pcscd service is running:

sudo systemctl start pcscd
sudo systemctl enable pcscd

Then, you have to run the following command:

echo "blacklist pn533_usb" | sudo tee -a /etc/modprobe.d/blacklist-libnfc.conf

This is used to avoid conflicts between the driver and one of the default kernel modules.

You can now reboot your computer.

If you are using Yocto, I created a recipe available here.

2. Installing the SDK

The next step is to install the SDK, which is required for developing the application but not for running it.

We will not use the SDK proposed on the official website. It is not open-source and only available for the i386 and x86_64 CPU architecture, which means that it is not possible to use it on Raspberry Pi for example (with their ARM CPU architecture).

Instead, we are going to use PCSC lite. You will need the systemd-devel package, as well as the libudev-dev package for Ubuntu:

# Fedora
sudo dnf install systemd-devel

# Ubuntu
sudo apt-get install libsystemd-dev libudev-dev

Then you can download, uncompress, configure, build, and install the sources:

wget https://pcsclite.apdu.fr/files/pcsc-lite-1.9.3.tar.bz2
tar -xf pcsc-lite-1.9.3.tar.bz2
cd pcsc-lite-1.9.3/
./configure
make
sudo make install

3. Check that the ACR122U is detected by your computer (optional)

Before starting to create the application, it's better to make sure your computer recognizes the reader: plug in the reader, install the pcsc-tools package, and run the pscs_scan command.

If your reader is not detected, the following message will appear:

Waiting for the first reader...

A solution is to restart the pcscd service:

sudo systemctl restart pcscd

If it still doesn't work, you can run the following command:

systemctl status pcscd

which may give you the reason of why the reader is not detected.

4. Creating the application

Let's now create the application. I will use the C language with the gcc compiler, but it's also compatible in C++ with g++. To communicate with the reader, we are going to use the PC/SC Lite API (WinSCard). All the code is written is a main.c file which is available on GitHub.

In this tutorial, we will be using the MIFARE Ultralight and MIFARE Classic 1K tags.

First, let's check that we can include the WinSCard library:

#include <winscard.h>

int main() {
    return 0;
}

The header file of the WinSCard library is in /usr/local/include/PCSC. Therefore, the program can be compiled and run with:

gcc main.c -lpcsclite -I/usr/local/include/PCSC
./a.out

Now, we will use the documentation of the WinSCard API to communicate with the reader. The first step is to establish a context (which has to be released at the end of the application):

#include <stdio.h>
#include <stdlib.h>
#include <winscard.h>

SCARDCONTEXT applicationContext;

void establishContext() {
    SCardEstablishContext(SCARD_SCOPE_SYSTEM, NULL, NULL, &applicationContext);
}
void releaseContext() {
    SCardReleaseContext(applicationContext);
}

int main() {
    establishContext();

    releaseContext();
    return 0;
}

We use a global variable applicationContext because we will need it for future functions. This code does not check whether the operations succeeded or not. Therefore, we add:

#include <stdio.h>
#include <stdlib.h>
#include <winscard.h>

SCARDCONTEXT applicationContext;

void establishContext() {
    LONG status = SCardEstablishContext(SCARD_SCOPE_SYSTEM, NULL, NULL, &applicationContext);
    if (status == SCARD_S_SUCCESS) {
        printf("Context established\n");
    } else {
        printf("Establish context error: %s\n", pcsc_stringify_error(status));
        exit(1);
    }
}
void releaseContext() {
    LONG status = SCardReleaseContext(applicationContext);
    if (status == SCARD_S_SUCCESS) {
        printf("Context released\n");
    } else {
        printf("Release context error: %s\n", pcsc_stringify_error(status));
        exit(1);
    }
}

int main() {
    establishContext();

    releaseContext();
    return 0;
}

This will give us some information on the success of each operation, and exit the program as soon as a failure occures.

Then, we have to list the available readers:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <winscard.h>

SCARDCONTEXT applicationContext;
LPSTR reader = NULL;

void establishContext() {}
void releaseContext() {}

void listReaders() {
    DWORD readers = SCARD_AUTOALLOCATE;
    LONG status = SCardListReaders(applicationContext, NULL, (LPSTR)&reader, &readers);

    if (status == SCARD_S_SUCCESS) {
        char *p = reader;
        while (*p) {
            printf("Reader found: %s\n", p);
            p += strlen(p) +1;
        }
    } else {
        printf("List reader error: %s\n", pcsc_stringify_error(status));
        exit(1);
    }
}
void freeListReader() {
    LONG status = SCardFreeMemory(applicationContext, reader);
    if (status == SCARD_S_SUCCESS) {
        printf("Reader list free\n");
    } else {
        printf("Free reader list error: %s\n", pcsc_stringify_error(status));
        exit(1);
    }
}

int main() {
    establishContext();
    listReaders();

    freeListReader();
    releaseContext();
    return 0;
}

We add a global variable reader which will be used to identify the reader for further use. The function SCardListReaders allocates some memory for reader, which has to be freed with the SCardFreeMemory function before the end of the program. If you get an error here even so the reader is plugged in, be sure to check that the ACR122U is detected by your computer.

Note that if the SCardListReaders function fails, the program exits directly without releasing the context. I kept this behavior to make this tutorial as simple as possible, but this is not something you want to do with real applications.

The next step is to establish a connection with a tag. Put a tag on the reader (for example a MIFARE Ultralight), and run the following code:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <winscard.h>

SCARDCONTEXT applicationContext;
LPSTR reader = NULL;
SCARDHANDLE connectionHandler;
DWORD activeProtocol;

void establishContext() {}
void releaseContext() {}
void listReaders() {}
void freeListReader() {}

void connectToCard() {
    activeProtocol = -1;

    LONG status = SCardConnect(applicationContext, reader, SCARD_SHARE_SHARED, SCARD_PROTOCOL_T0 | SCARD_PROTOCOL_T1, &connectionHandler, &activeProtocol);
    if (status == SCARD_S_SUCCESS) {
        printf("Connected to card\n");
    } else {
        printf("Card connection error: %s\n", pcsc_stringify_error(status));
        exit(1);
    }
}
void disconnectFromCard() {
    LONG status = SCardDisconnect(connectionHandler, SCARD_LEAVE_CARD);
    if (status == SCARD_S_SUCCESS) {
        printf("Disconnected from card\n");
    } else {
        printf("Card deconnection error: %s\n", pcsc_stringify_error(status));
        exit(1);
    }
}


int main() {
    establishContext();
    listReaders();
    connectToCard();

    disconnectFromCard();
    freeListReader();
    releaseContext();
    return 0;
}

We have here two more global variables:

connectionHandler: used to communicate with the tag.
activeProtocol: it is the established protocol to this connection. We need to keep track of the protocol because we let WinSCard choose which protocol to use, as we gave SCARD_PROTOCOL_T0 | SCARD_PROTOCOL_T1 as the fourth argument of the SCardConnect function. You can specify which protocol to use by giving only SCARD_PROTOCOL_T0 for example. The protocol will be required when we want to send a custom command to the the tag.

Like before, the connection must be closed before the end of the program.

Now that we have a connection to the tag, we can get some information from it, like its ATR. The ATR can be used to identify the type of card. A complete list is available here.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <winscard.h>

SCARDCONTEXT applicationContext;
LPSTR reader = NULL;
SCARDHANDLE connectionHandler;
DWORD activeProtocol;

void establishContext() {}
void releaseContext() {}
void listReaders() {}
void freeListReader() {}
void connectToCard() {}
void disconnectFromCard() {}

void getCardInformation() {
    BYTE ATR[MAX_ATR_SIZE] = "";
    DWORD ATRLength = sizeof(ATR);
    char readerName[MAX_READERNAME] = "";
    DWORD readerLength = sizeof(readerName);
    DWORD readerState;
    DWORD readerProtocol;

    LONG status = SCardStatus(connectionHandler, readerName, &readerLength, &readerState, &readerProtocol, ATR, &ATRLength);
    if (status == SCARD_S_SUCCESS) {
        printf("\n");
        printf("Name of the reader: %s\n", readerName);
        printf("ATR: ");
        for (int i=0; i<ATRLength; i++) {
            printf("%02X ", ATR[i]);
        }
        printf("\n\n");
    } else {
        printf("Get card information error: %s\n", pcsc_stringify_error(status));
        exit(1);
    }
}

int main() {
    establishContext();
    listReaders();
    connectToCard();

    getCardInformation();

    disconnectFromCard();
    freeListReader();
    releaseContext();
    return 0;
}

Finally, we can send a custom command to the reader. Going back to the ACR122U webpage, you can download the API Driver Manual of ACR122U NFC Contactless Smart Card Reader. This document presents a list of commands we can send to the reader. For example, let's retrieve the firmware version of the reader (on page 24). The associated command is FFh 00h 48h 00h 00h.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
#include <winscard.h>

SCARDCONTEXT applicationContext;
LPSTR reader = NULL;
SCARDHANDLE connectionHandler;
DWORD activeProtocol;

void establishContext() {}
void releaseContext() {}
void listReaders() {}
void freeListReader() {}
void connectToCard() {}
void getCardInformation() {}

void sendCommand(uint8_t command[], unsigned short commandLength) {
    const SCARD_IO_REQUEST *pioSendPci;
    SCARD_IO_REQUEST pioRecvPci;
    uint8_t response[300];
    unsigned long responseLength = sizeof(response);

    switch(activeProtocol) {
        case SCARD_PROTOCOL_T0:
            pioSendPci = SCARD_PCI_T0;
            break;
        case SCARD_PROTOCOL_T1:
            pioSendPci = SCARD_PCI_T1;
            break;
        default:
            printf("Protocol not found\n");
            exit(1);
    }

    LONG status = SCardTransmit(connectionHandler, pioSendPci, command, commandLength, &pioRecvPci, response, &responseLength);
    if (status == SCARD_S_SUCCESS) {
        printf("Command sent: \n");
        for (int i=0; i<commandLength; i++) {
            printf("%02X ", command[i]);
        }
        printf("\nResponse: \n");
        for (int i=0; i<responseLength; i++) {
            printf("%02X ", response[i]);
        }
        printf("\n\n");
    } else {
        printf("Send command error: %s\n", pcsc_stringify_error(status));
        exit(1);
    }
}

int main() {
    establishContext();
    listReaders();
    connectToCard();

    getCardInformation();

    printf("Firmware command:\n");
    uint8_t firmwareCommand[] = { 0xFF, 0x00, 0x48, 0x00, 0x00 };
    unsigned short firmwareCommandLength = sizeof(firmwareCommand);
    sendCommand(firmwareCommand, firmwareCommandLength);

    disconnectFromCard();
    freeListReader();
    releaseContext();

    return 0;
}

The switch statement is needed here because the established protocol to the connection will be determined on runtime. If we had only used SCARD_PROTOCOL_T0 in the SCardConnect function, we could have directly written pioSendPci = SCARD_PCI_T0; instead of the switch statement.

We have now a running application that can be used to send custom commands to the reader and the tag. We will apply this to send and retrieve data from the MIFARE Ultralight and the MIFARE Classic 1k. Let's start with the MIFARE Ultralight.

The MIFARE Ultralight is composed of 16 pages with 4 bytes each. The user memory starts from the fifth page and the data is unprotected (we can access without authentication). You can find more information on the data sheet of the tag.

On page 16 and 17 of the API Driver Manual of ACR122U, we can see the read and write commands. Let's write a function that read 16 bytes from the fifth page (page 04h) and write 4 bytes to the fifth page:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
#include <winscard.h>

SCARDCONTEXT applicationContext;
LPSTR reader = NULL;
SCARDHANDLE connectionHandler;
DWORD activeProtocol;

void establishContext() {}
void releaseContext() {}
void listReaders() {}
void freeListReader() {}
void connectToCard() {}
void disconnectFromCard() {}
void getCardInformation() {}
void sendCommand(uint8_t command[], unsigned short commandLength) {}

void mifareUltralight() {
    printf("### MIFARE Ultralight ###\n");
    uint8_t pageNumber = 0x04;

    // Read 4 blocks (16 bytes) starting from pageNumber
    uint8_t readCommand[] = { 0xFF, 0xB0, 0x00, pageNumber, 0x10 };
    unsigned short readCommandLength = sizeof(readCommand);
    sendCommand(readCommand, readCommandLength);

    // Write 1 block (4 bytes) to pageNumber
    uint8_t data[] = { 0x00, 0x01, 0x02, 0x03 };
    uint8_t writeCommand[9] = { 0xFF, 0xD6, 0x00, pageNumber, 0x04 };
    unsigned short writeCommandLength = sizeof(writeCommand);
    for (int i=0; i<4; i++) {
        writeCommand[i+5] = data[i];
    }
    sendCommand(writeCommand, writeCommandLength);
}

int main() {
    establishContext();
    listReaders();
    connectToCard();

    getCardInformation();

    printf("Firmware command:\n");
    uint8_t firmwareCommand[] = { 0xFF, 0x00, 0x48, 0x00, 0x00 };
    unsigned short firmwareCommandLength = sizeof(firmwareCommand);
    sendCommand(firmwareCommand, firmwareCommandLength);

    mifareUltralight();

    disconnectFromCard();
    freeListReader();
    releaseContext();
    return 0;
}

Finally, let's do the same with the MIFARE Classic 1k. This tag consists of 16 sectors of 4 blocks, each block containing 16 bytes. The data is protected by a 6-bytes key, the default key being FFh FFh FFh FFh FFh FFh. You can find more information on the data sheet of the tag.

With the MIFARE Classic 1k, we need do authenticate before accessing a sector, which is indicated on page 12 and 13 of the API Driver Manual of ACR122U.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
#include <winscard.h>

SCARDCONTEXT applicationContext;
LPSTR reader = NULL;
SCARDHANDLE connectionHandler;
DWORD activeProtocol;

void establishContext() {}
void releaseContext() {}
void listReaders() {}
void freeListReader() {}
void connectToCard() {}
void disconnectFromCard() {}
void getCardInformation() {}
void sendCommand(uint8_t command[], unsigned short commandLength) {}
void mifareUltralight() {}

void mifareClassic() {
    printf("### MIFARE Classic ###\n");
    uint8_t blockNumber = 0x04;

    // Load Authentication Keys
    uint8_t key[6] = { 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF };
    uint8_t authenticationKeysCommand[11] = { 0xFF, 0x82, 0x00, 0x00, 0x06 };
    unsigned short authenticationKeysCommandLength = sizeof(authenticationKeysCommand);
    for (int i=0; i<6; i++) {
        authenticationKeysCommand[i+5] = key[i];
    }
    sendCommand(authenticationKeysCommand, authenticationKeysCommandLength);

    // Authenticate
    uint8_t authenticateCommand[] = { 0xFF, 0x86, 0x00, 0x00, 0x05, 0x01, 0x00, blockNumber, 0x60, 0x00 };
    unsigned short authenticateCommandLength = sizeof(authenticateCommand);
    sendCommand(authenticateCommand, authenticateCommandLength);

    // Read 1 block (16 bytes) at blockNumber
    uint8_t readCommand[] = { 0xFF, 0xB0, 0x00, blockNumber, 0x10 };
    unsigned short readCommandLength = sizeof(readCommand);
    sendCommand(readCommand, readCommandLength);

    // Write 1 block (16 bytes) at blockNumber
    uint8_t data[] = { 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F };
    uint8_t writeCommand[21] = { 0xFF, 0xD6, 0x00, blockNumber, 0x10 };
    unsigned short writeCommandLength = sizeof(writeCommand);
    for (int i=0; i<16; i++) {
        writeCommand[i+5] = data[i];
    }
    sendCommand(writeCommand, writeCommandLength);
}

int main() {
    establishContext();
    listReaders();
    connectToCard();

    getCardInformation();

    printf("Firmware command:\n");
    uint8_t firmwareCommand[] = { 0xFF, 0x00, 0x48, 0x00, 0x00 };
    unsigned short firmwareCommandLength = sizeof(firmwareCommand);
    sendCommand(firmwareCommand, firmwareCommandLength);

    //mifareUltralight();
    mifareClassic();

    disconnectFromCard();
    freeListReader();
    releaseContext();
    return 0;
}

5. Conclusion

You know now how to send and retrieve data from the MIFARE Ultralight and Classic 1k with the ACR122U. This should be enough for most of the applications. Otherwise, you can take a look at:

The PC/SC Lite API (WinSCard) documentation.
The API Driver Manual of ACR122U NFC Contactless Smart Card Reader.
The data sheets of your own tags.