DEV Community: Gergana Young

Authorisation with Casbin and Koa Part 3

Gergana Young — Tue, 25 May 2021 09:03:20 +0000

In the first part of this series of posts we set up some configuration and a few Casbin policies that we based on a set of defined rules. In the second part we talked about adding these policies to our Koa application as middleware. In this third and last post I'd like to talk a little bit about how you can add exceptions to these rules.

Today we are going to talk about introducing another set of middleware to our Koa application that will enable us to skip the authorisation middleware that we added with Casbin when certain conditions are met.

Why would we want to do this? Every application ends up with some exceptions to the rules. In this case, you may have an endpoint that does not require any authentication or authorisation. So if we look at our rules that we created in the first post:

All endpoints containing /admin can only be accessed by admin users.
All endpoints that do not contain /admin can be accessed by normal users.
Anonymous users can only access the endpoint to create a user, after which they will become a user.

Say, for example, we now want to add a new endpoint called /promotions. All users should be able to access this endpoint, even if they have not registered or if an anonymous account has not been created for them. It would be really difficult to add a rule like this to the Casbin policies, so instead, we create a new middleware that would skip checking the policies for this endpoint.

The exceptions

We will first create an object that will store all of the endpoints that we would like to add exceptions for, it can look something like this:

const noAuthEndpoints = {
  promotions: {
    path: `^\/promotions$`,
    method: 'GET',
  },
};

As you can see I am also specifying the method that is allowed. This will ensure that we only allow very specific endpoints here. We do not want to accidentally allow something we are not supposed to.

The middleware

Now we can create our middleware to check if the incoming request matches any of the exceptions, it would look something like this:

export function shouldSkipAuth(requests, contextRequest) {
  for (const requestKey of Object.keys(requests)) {
    const request = requests[requestKey];
    const regex = new RegExp(request.path);

    if (regex.test(contextRequest.path) && request.method === contextRequest.method) {
      return true;
    }
  }
  return false;
}

export function authExceptions(requests, middleware) {
  return async (ctx, next) => {
    if (shouldSkipAuth(requests, ctx.request)) {
      return await next();
    }

    return await middleware(ctx, next);
  };
}

We are testing every item in the object we created to see if any of them match our request. If yes, then we just run the next action. If no, we run the middleware that is passed into the function.

Using the middleware

If you remember from last weeks article, in order to use the middleware in our Koa application we had something that looked like this:

app.use(authorisation(enforcer));

We will now take this line and change it a little to use our exceptions middleware, the final result will look like this:

app.use(authExceptions(noAuthEndpoints, authorisation(enforcer)));

Running the Koa application should now allow you to access the new endpoint without having a user or role, and all of the previous rules should still apply.

A note on testing

A very important next step here is adding tests that will ensure that these rules behave exactly as they are supposed to. You should try have a test for every positive and negative case for each new rule that you add. This should be both the policy rules we created in Casbin as well as the exception rules we created in this post.

Summary

Casbin is a really nice tool and works well with Koa middleware. In these series of posts the examples we created are relatively simple cases, but they are a good starter for getting used to creating authorisation policies. A lot more can be done here to make your application really secure.

image by Dom J on pexels.com

Authorisation with Casbin and Koa Part 2

Gergana Young — Mon, 17 May 2021 07:39:44 +0000

In the first part of this series of posts we spoke about how to setup the Casbin policies and configuration. Now that we have that working and we have tested it in the online editor we are ready to add it as middleware to our Koa application.

Note: I am using Koa as an example here since this is what I am used to. If you prefer using Express or another framework for node, you will need to change the below examples a little bit in order for it to work.

Koa

Koa is a web framework for nodejs, it was designed by the same team that created Express. It was built in a way to make it smaller and more robust. However, we are not here to discuss which web framework to use, we just want to talk about authorisation.

There are already a few different libraries that contain Koa and Casbin middleware, but it is also pretty easy to build your own. This is what we are going to do now.

Node-Casbin

The first thing we need to do is install the Node-Casbin library into our project, we will do this using npm with the following command:

$ npm install casbin --save

The middleware

Now that we have Casbin installed we can use the Casbin enforcer in a middleware function to check the request against our policies and return a 403 if the request is not allowed. We do this as follows:

export function authorisation(enforcer) {
  return async (ctx, next) => {
    const role = '<extract role from authentication middleware>';
    const path = ctx.request.path;
    const method = ctx.request.method;

    const authorised = await enforcer.enforce(role, path, method);
    if (!authorised) {
      ctx.status = 403;
      return;
    }

    await next();
  };
}

const role = '<extract role from authentication middleware>'; - this line will need to be replaced with whatever function can extract the current user's role. Assigning roles to users is out of scope for this article, otherwise we might be here for another few weeks.
const authorised = await enforcer.enforce(role, path, method); - this is the line that does all of the work, it uses the Casbin enforcer to check the role (subject), path (object) and method (action) agains the policies we defined in the previous post.

Using the middleware

In the setup of our Koa application we will need to create a Casbin enforcer and use it in the app middleware.

The first step for this is to import the newEnforcer function from node-casbin:

import { newEnforcer } from 'casbin';

const { newEnforcer } = require('casbin');

Next we need to create an enforcer with the configuration file (model.conf) and policy file (policy.csv) we created in the previous post:

const enforcer = await newEnforcer('path/to/model.conf', 'path/to/policy.csv');

And finally we use our middleware (where the app below id your Koa application):

app.use(authorisation(enforcer));

Summary

You should now be able to run your Koa application and test out the authorisation policies we set up in the previous post. In the last post in this series we will talk about an easy way to skip this authorisation Koa middleware for certain paths. Thanks again for reading.

Authorisation with Casbin and Koa Part 1

Gergana Young — Mon, 10 May 2021 08:03:39 +0000

This series of posts is an introduction on how you can add Casbin to your Koa application. However, before we get started I would like to add a disclaimer, I am not a security expert and you should definitely have someone who knows about security on the web to look over your policies before implementing them in production.

Casbin

Casbin is an open-source authorisation library that supports pretty much all of the access control models and a whole range of programming languages. There is a really useful online editor where you can select your access control model and create access policies to test out what will and will not work for your project.

Note: Casbin is an authorisation library, not an authentication library. I.e. There is no username and password verification or storage. You would need to use a different library to do that.

RBAC

In this post we will design a (sort of) role based access control policy in Casbin. In the next post of this series we will implement these policies as Koa middleware in our application. There are two things that we need in order to get our policies working a configuration file and a policy file.

The configuration file is where we will define how Casbin should act on our subject, object and action. In the policy file we will define what the subject, object and action for each user type will be. In order to do this, let's talk about what we actually want to authorise first.

We will have three types of users, normal users, anonymous users, and admin users. We will define their roles as user, anonymous and admin. We will need to define the permissions for these users based on what endpoints and actions they can perform. Let's make up some rules:

All endpoints containing /admin can only be accessed by admin users.
All endpoints that do not contain /admin can be accessed by normal users.
Anonymous users can only access the endpoint to create a user, after which they will become a user.

Config definition

Our config definition will look as follows:

[request_definition]
r = sub, obj, act

[policy_definition]
p = sub, obj, act, eft

[policy_effect]
e = some(where (p.eft == allow)) && !some(where (p.eft == deny))

[matchers]
m = r.sub == p.sub && regexMatch(r.obj, p.obj) && regexMatch(r.act, p.act)

What do these things mean?

request_definition: This is the definition of what we will receive in each request, i.e. the subject, object and action.
policy_definition: This is how we will define each policy in our policy file, each policy will also have a subject, object and action as well as an effect where we will define whether this is an allow or a deny policy.
policy_efect: This is where we define our function that determines the effect of each policy. Our function is checking the policies which are allowed and not denied.
matchers: In order to determine whether a policy matches the request we use the matcher function definition. In our case we are using a direct comparison of the subject and a regexMatch on both the object and the action.

Policy definition

Let us write our policies and then the last two will make a little bit more sense. As we said in the policy definition we will be defining our policies ad subject, object, action, effect. We just need to add a p in front so that our config can recognise it as a policy. Ours will look as follows:

p, admin, ^/[a-z\-]*/admin, (GET)|(PUT)|(POST)|(DELETE)|(OPTIONS), allow
p, anonymous, ^/user/?$, (POST), allow
p, user, ^/[a-z]+, (GET)|(PUT)|(POST)|(DELETE)|(OPTIONS), allow
p, user, ^/[a-z\-]*/admin, (GET)|(PUT)|(POST)|(DELETE)|(OPTIONS), deny

These four policies should be enough to cater for the rules we defined earlier. Try this out in the online editor to see it working.

Summary

That is all we need for setting up our policies using Casbin. If you would like to read more about how to create the configurations and policies take a look at the Casbin getting started documentation. There are also a lot of different implementations available for different languages in the Casbin GitHub repository. In the next post we will talk about how to add these policies into our Koa middleware.

Puppeteer in an AWS Lambda Function Part 2

Gergana Young — Mon, 16 Nov 2020 07:01:36 +0000

In my previous post I talked a little bit about how you can get started with running a puppeteer script in an AWS Lambda function. Using the puppeteer-core and chrome-aws-lambda libraries, you can get your code down to a size that will fit into the 50MB limit.

This is a relatively good solution, that will work for most cases. However, there are some drawbacks:

If your code becomes a bit more complex you might end up exceeding the limit.
You might have multiple functions that require puppeteer.
Every time you make a change to some of your code, you have to re-upload everything to AWS. This can take some time as these libraries are still more than 40MB in size.

Lucky for us, there is one solution to all 3 of these problems, we use layers. In this post, I will take you through extracting the two libraries required to run puppeteer, into an AWS Lambda layer and using this layer inside your function.

Creating your layer

The puppeteer-core library is dependent on the chrome-aws-lambda library as that is what installs the chromium bin file that is needed. This is why we move both of them into one layer. I found that even though our Lambda function can interact with the libraries inside a layer as if they were in its own node_modules folder, the libraries themselves do not interact with each other in the same way. Which means that leaving puppeteer-core in our Lambda function and only moving chrome-aws-lambda will not work. (This was discovered by trial and error 😅).

The easiest way to do this is to create a new folder for your layer and add a package.json with the two libraries as dependencies. Since these are node libraries, AWS requires them to be inside a folder called nodejs, so our folder structure would be layer/nodejs/package.json and the file would look something like this:

{
  "name": "puppeteer-layer",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "create-layer": "pushd ../../ && rm -rf layer.zip && popd && npm i && cd ../ && zip -r ../layer.zip *"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "chrome-aws-lambda": "^5.3.1",
    "puppeteer-core": "^5.3.1"
  }
}

We have one script in our package.json file, it will create the zip file that we will upload to our layer in AWS. We now run npm run create-layer and upload our file using the AWS management console. Go to the console and log in, select Lambda, from the side menu select Layers, click on Create layer fill in all of the details and upload the zip file.

Another note on running locally: Once you've moved these dependencies out of your Lambda function's package.json you might struggle to run this locally. Putting them into the dev dependencies is the easiest way to get around this problem.

Using your layer

The nice thing here is that none of your code has to change, just remember to remove those dependencies (or make them dev dependencies). Upload your new Lambda function without the dependency on puppeteer-core and chrome-aws-lambda and then we can tell it about the layer. In the console, select your function, and select Layers, you will see an empty list of layers. Click on Add a layer, select the Custom layers option and your newly created layer should appear in that list. Once added, everything should work as it did before.

Note: One last thing to keep in mind is that a layer is also limited to 50MB size, which means that you will not be able to move all of your dependencies to a single layer if they exceed this size. However, you can create and use more than one layer in your Lambda function.

Conclusion

Now that we have a layer we can easily create multiple functions that use these libraries and not worry about the size of our own code. We can also update our code much easier, since we would have a much smaller zip file to work with.

Puppeteer in an AWS Lambda Function Part 1

Gergana Young — Mon, 09 Nov 2020 07:51:44 +0000

Puppeteer is a really useful tool, it provides us with a way to access the Chrome (or Chromium) DevTools protocol inside our Node.js code. It can be used for many different things, such as testing, generating pre-rendered content, automating form submissions, etc. The one problem with puppeteer is that it requires some resources to run, which makes sense, since it starts up a browser instance and runs through all the actions you have told it to as if a person was doing it.

Until recently, I had only ever run puppeteer locally, which is pretty easy to set up and debug. However, when you want to automate some of these processes, it becomes a bit more challenging. I did some research, and found out that my best option, for AWS at least, was to run my process inside a Lambda function. In this way all of the resources would be isolated and when the function is stopped everything will be cleaned up and deleted.

There was only one problem left to solve, how do we make sure that we do not exceed the 50MB limit of a Lambda function. This was quite a big one, considering the puppeteer library itself is around 300MB due to the fact that it installs a browser with it. In this post, I'd like to take you through what libraries you will need in order to be able to use puppeteer inside a Lambda function.

Using `puppeteer-core`

This core package allows us to install the core tools of puppeteer without installing a browser. It's great for this because it is only around 2MB.

Although, when using puppeteer-core we still don't have a browser for our Lambda function. Not to worry, there's a solution for this problem too, the chrome-aws-lambda package, which can also be installed using npm was made for this.

Once you have both these packages installed, setting up puppeteer inside your Lambda becomes relatively easy. Your code will look something like this:

const chromium = require('chrome-aws-lambda');

export const handler = async (event, context, callback) => {
  let result = 'result';
  let browser;

  try {
    browser = await chromium.puppeteer.launch({
      args: chromium.args,
      defaultViewport: chromium.defaultViewport,
      executablePath: await chromium.executablePath,
      headless: chromium.headless,
      ignoreHTTPSErrors: true,
    });
    const page = await browser.newPage();

    // all your puppeteer things
  } catch (error) {
    return callback(error);
  } finally {
    if (browser !== null) {
      await browser.close();
    }
  }

  return callback(null, result);
};

The next step is to zip the whole thing up and upload it to an AWS Lambda function. Even with these reduced size packages, the zipped file is still around 40MB. This means that you will not be able to see the code when opening the function in the AWS web console, so you will have to do all your debugging locally.

A note on running locally: If you need to run this Lambda locally to test it, you will still need to use the main puppeteer package. You could either install it globally, or install it as a dev dependency in your project (recommended).

A note on the aws-sdk: Keep in mind that Lambda functions have access to all of the aws-sdk features, so you don't have to install that as another package. This would take up too much space in your allocated 50MB. If you need to test locally you can add this to your dev dependencies as well.

Conclusion

That's the first part of getting puppeteer into a Lambda function. We managed to accomplish most of our goal, by using other tools created for this exact purpose. In my next post I will cover what can be done if you still exceed the 50MB limit.

Online card games for quarantine

Gergana Young — Sun, 26 Apr 2020 08:56:25 +0000

A little while ago friends of ours taught us how to play a card game called Canaster. We played it a couple of times, but since the whole self isolation and quarantine thing started, we haven't been able to play. A couple of weeks ago, I started looking online to see if I could find a version we could play even though the four of us could not be together. In my research I found that this game is really popular, but there was one problem. It seems that everyone has slightly different rules for it, and no one seemed to have the same rules that we were taught.

So, one Sunday, my husband and I decided it was time to build our own Canaster, the way we wanted to play it with our friends. We got the main idea after playing some games from Jackbox Games. We decided to build the entire thing using JavaScript and as few helper libraries and frameworks as possible. In this post, I'd like to tell you a little bit about our technology choices, why we made them, and the things we still need to work on.

The frontend

The frontend consists of 3 pages:

The main page which displays the game in its current state and if there is no game selected it displays the navigation buttons. This is the page that the host computer would run and will be shared with the other players over whatever video calling software you are using.
The rules page shows the rules as we learned them, in case someone stumbles across the game and wants to try it.
The hand or player page, where every player can join an existing game and view their cards as well as perform different actions, like drawing or discarding cards. This would usually be accessed from the player's phone or another device which they are not using for the video call.

We also built a few vanilla JavaScript web components. These do things like show the cards, show each player, and pop-up error messages. This part was really fun as I had never built pure web components without using a library or framework before.

It can sometimes take a player a long time to decide what to do on their turn. This means that, depending on your device's settings, the screen starts turning off or locking. We decided to try implementing a keep awake function. The first attempt involved the Wake Lock API, but due to it still being in an origin trial in Chrome, we decided that this would not work for us.

We then tried NoSleep.js, we ran one complete game with this. While playing we noticed that my computer, which we were using as the host, had it's fan going crazy. After some digging we found this issue. We have now decided that, at least for the time being, we are going to remove this library and just rely on the players to keep their devices awake. When we find a better solution, I will definitely write another post about it.

The server

We built the server using Node and Express, this we chose because we have of experience in it, so it was the easiest option. The hardest part about the server was implementing all of the rules. We started out by hard coding a lot of them, but then I realized that this won't work for long. There were a lot of places that needed the same rule, and hard coding meant that if we wanted to change it, we would need to change it multiple times. We created a separate rules object that we could export and import into any page that needed it.

There is still one section of this code that needs to be refactored and cleaned up because it contains pretty much all of the rules of the game. Other than that, and a few more hard coded rules, we are pretty much done with the server.

One of the interesting things we decided to use was the Node --experimental-modules flag. This is an experimental feature which allows us to use import and export for .js files in Node, rather than having to use the require syntax. We are using Node v12, so the feature is not enabled by default, this means there are two things we have to do in order to be able to use modules:

Add the type field to your package.json file: "type": "module"
Add the --experimental-modules when you run the code: node --experimental-modules index.js

If you are interested in reading more about this experimental feature, take a look these Node v14 docs.

Communication

Since we were using Node with Express we could easily create endpoints. These endpoints are accessed by the front-end using the Fetch API.

Although, how does a player know it is their turn and how does the main game screen know whose turn it is and what cards are being played. We decided it was time to introduce web sockets with socket.io. In order to keep all of the different games separate, as well as separate the messages sent to the main screen and the player we decided to use rooms for our sockets. These rooms allow us to limit the information being sent by giving the socket you are connecting to a name, you will then only receive the data sent to that specific name.

Data

Storing the game state is still a bit of a problem in the current version of the game. Everything is currently stored in memory, this comes with a few problems:

If the server restarts, we loose the games that were being played.
If it doesn't restart, there is currently no way to delete the games, so the server will eventually run out of memory.

The reason we chose this, is because having a datastore is relatively expensive in the cloud hosting world. However, we are planning on trying to find a way to fix this in future.

As for the second problem, being able to delete a game can be added as an extra button on the screen, but we can't rely on people clicking it. So the plan is to add an Azure function that will run once every night and delete all games that are older than 24 hours. More on how to build a function like this will come in a future post.

Hosting

We decided to use Azure as a hosting platform. If you'd like to learn more about how to automatically deploy your Node applications to Azure using pipelines, take a look at this post I wrote about it. It talks about using Restify and lit-html, but can easily be adapted to any other Node backend.

Security

Well, please don't judge, there is none, unless you count that it runs on HTTPS. There are two of things that we need to fix:

The endpoints are accessible from the browser, i.e. you can get the entire game state and see other people's cards if you know the correct endpoint to go to.
If another person with the same name as you enters the same game as you they will be able to see your hand.

We have ideas on how to fix both of these problems, so fixes for them are coming soon.

Conclusion

Even though we spent a few days on a weekend building this game, it is actually turning out to be exactly what we wanted. You can check out the code on GitHub if you are interested. There are still some problems, it needs some refactoring and a bit less hard coding. It has been fun trying to get all of the different parts working together and we are going to continue improving it. I hope this inspires some people to build a game they miss playing with their friends. You never know what you'll learn in the process.

Smart Lightsaber - Bluetooth peripherals with JavaScript

Gergana Young — Thu, 12 Mar 2020 08:10:29 +0000

I talk about Bluetooth a lot, you may have noticed this from my previous posts and if you have listened to my talks. Bluetooth has become my favourite wireless technology, even though working with it can be very frustrating at times. It does not provide the most reliable wireless connection, it is far from perfect, and the distances it works over are always less than advertised. However, once you understand the basics, it is really fun to work with.

When it came to choosing a wireless technology for the lightsaber we are building, my first thought was definitely Bluetooth. In this post I'd like to tell you a little bit about how a Bluetooth peripheral, such as our lightsaber, is built, using a library called bleno.

Before we get started, there are two other posts that I have written regarding Bluetooth, you should check them out as they would help with understanding parts of this post:

As described in the posts above, there are two types of Bluetooth devices:

The central device (or GATT Client) which is in charge of scanning for devices, connecting and reading/writing data to the other device. This is usually a computer or a phone.
The peripheral device (or GATT Server) is the device being connected to. It can perform certain functions and it exposes those functions over Bluetooth so that the central device can see and make use of them.

A Bluetooth connection can only ever be from a central to a peripheral device. My previous posts define how you can create a central device using the Web Bluetooth API and what that actually means. In this post I would like to take you through the other side of a Bluetooth connection. We will build the GATT Profile for our lightsaber using bleno. It will have one service and two characteristics, one for changing the colour of the light and one for reading the button status.

Installing bleno

Our lightsaber runs on a Raspberry Pi Zero which has Raspbian Buster Light installed on it. In order to get bleno working, there are a few things we need to do.

Firstly, we need to have Node installed, I use nvm to manage my Node versions. At the time of writing this post the bleno library has some problems with Node version 10, so I am using version 8.

Next we need to install some libraries that will help us control Bluetooth on the Pi Zero. We do this by running the following command:

sudo apt-get install bluetooth bluez libbluetooth-dev libudev-dev

Now we can install bleno, by using npm:

npm install bleno

Building the characteristics

Let's start with creating the functions of the device, our two characteristics. Each characteristic has a list of properties that define what it can actually do. There are two properties that we care about in this example:

Write - a characteristic that receives data from a central device and performs some function with it. In our case, this will change the colour of the light.
Notify - a characteristic that reads data from the peripheral device and sends it onto the central device at a set interval. In our case, this will read the value of the button to determine if it is pressed or not and send that onto the central device.

Each characteristic also needs a universally unique identifier (UUID). I used a generator to create random UUIDs for these examples.

Light characteristic

This characteristic will receive an unsigned integer array with three numeric values between 0 and 255, one for red, green and blue. This is what the code looks like:

const bleno = require('bleno');
const { LIGHT_CHAR_UUID } = require('./characteristics');

module.exports = class LightCharacteristic extends bleno.Characteristic {
  constructor() {
    super({ uuid: LIGHT_CHAR_UUID, properties: ['write'] });

    this.red = 0;
    this.green = 0;
    this.blue = 0;
  }

  onWriteRequest(data, offset, withoutResponse, callback) {
    try {
      if (data.length !== 3) {
        callback(this.RESULT_INVALID_ATTRIBUTE_LENGTH);
        return;
      }

      this.red = data.readUInt8(0);
      this.green = data.readUInt8(1);
      this.blue = data.readUInt8(2);
      //change colour of light based on values

      callback(this.RESULT_SUCCESS);
    } catch (err) {
      console.error(err);
      callback(this.RESULT_UNLIKELY_ERROR);
    }
  }
};

There are a few parts of this code that I would like to point out:

const bleno = require('bleno'); - firstly, we have to import the bleno library.
class LightCharacteristic extends bleno.Characteristic - we are extending the Characteristic class from bleno in order to create our characteristic.
super({ uuid: LIGHT_CHAR_UUID, properties: ['write'] }); - in the constructor we are creating our new characteristic by setting its UUID and its properties.
onWriteRequest(data, offset, withoutResponse, callback) - we then override the onWriteRequest method so that when the central device sends data to this characteristic we can control what happens.
if (data.length !== 3) - we have some error checking to ensure that the central device is sending the correct data.
this.red = data.readUInt8(0); - we read our 3 values and change the colour of the lightsaber.
callback(this.RESULT_INVALID_ATTRIBUTE_LENGTH);, callback(this.RESULT_SUCCESS); and callback(this.RESULT_UNLIKELY_ERROR); - based on what happens inside our method, we have to use the callback function to notify the central device that the commands have finished executing.

That's our light characteristic completed.

Button Characteristic

The button characteristic will read the value of the button, 1 or 0, every 500 milliseconds and if the value has changed it will send that data to the central device inside a Buffer. Let's take a look at the code:

const bleno = require('bleno');
const { BUTTON_CHAR_UUID } = require('./characteristics');

module.exports = class ButtonCharacteristic extends bleno.Characteristic {
  constructor() {
    super({ uuid: BUTTON_CHAR_UUID, properties: ['notify'] });

    this.buttonValue = '0';
  }

  onSubscribe(maxValueSize, updateValueCallback) {
    this.updateValueCallback = updateValueCallback;
  }

  onUnsubscribe() {
    this.updateValueCallback = null;
  }

  sendNotification(value) {
    if (!this.updateValueCallback) return;

    if (value !== this.buttonValue) {
      this.buttonValue = value;
      const notification = new Buffer(2);
      notification.writeInt16LE(this.buttonValue);

      this.updateValueCallback(notification);
    }
  }

  start() {
    this.buttonInterval = setInterval(() => {
      const data = readButton(); //read value of button
      this.sendNotification(data.toString());
    }, 500);
  }

  stop() {
    clearInterval(this.buttonInterval);
    this.buttonInterval = null;
  }
};

The start of this characteristic is very similar to the previous one, the only difference being the property which is set to 'notify'. However, the methods we override in this class are different:

onSubscribe(maxValueSize, updateValueCallback) - this method is called when a central device connects to a peripheral device and starts listening for this particular characteristic value.
onUnsubscribe() - we only want the peripheral device to continue sending the data if there is a central device listening. We use the unsubscribe method to clear the callback so that data does not continue being sent after the central device stops listening.
sendNotification(value) - this is where most of the work happens, we check if there is a central device listening, we check if the button value has changed, we convert the value to a Buffer and send it onto the central device.
start() and stop() - these two functions are only called internally on the peripheral device. When we fist set up the device we will start the notify characteristic, when we power it down, we will stop it.

That's all we need for the button characteristic. Now both of our characteristics are completed.

Adding in our service

Every peripheral device has one or more services, they are there to combine similar functions (characteristics) together. In our case, since we only have two characteristics, we can probably make do with just one service. We will call it our primary service and give it a UUID. We use functionality from bleno to do this:

const bleno = require('bleno');

const LightCharacteristic = require('./characteristics/lightCharacteristic');
const ButtonCharacteristic = require('./characteristics/buttonCharacteristic');

const lightWrite = new LightCharacteristic();
const buttonRead = new ButtonCharacteristic();
buttonRead.start();

function createService() {
  const service = new bleno.PrimaryService({
    uuid: PRIMARY_SERVICE_UUID,
    characteristics: [buttonRead, lightWrite]
  });

  bleno.setServices([service], (err) => {
    console.log(err || 'configuration done');
  });
}

GATT Profile

Lastly, now that we have a service that contains our two functions, we need to wrap that and broadcast it over Bluetooth so that central devices can scan for it. We do this by listening to two events, the first is the stateChanged event:

bleno.on('stateChange', (state) => {
  if (state === 'poweredOn') {
    bleno.startAdvertising('Lightsaber', [PRIMARY_SERVICE_UUID], (err) => {
      if (err) console.error(err);
    });
  } else {
    buttonRead.stop();
    bleno.stopAdvertising();
  }
});

There is only one state we care about, the poweredOn state. When the device turns on and this script starts running this event will fire with the state set to poweredOn. When this happens we need to startAdvertising the Bluetooth device. We give the device a name and the UUID of our primary service, this makes it discoverable by central devices. If the state is set to anything else, we stop the notify characteristic and stopAdvertising.

The other event we need to listen to is the advertisingStart event. This is triggered by the call to startAdvertising that we just talked about. The code looks as follows:

bleno.on('advertisingStart', (err) => {
  if (err) {
    console.error(err);
    return;
  }

  createService();
});

All we need to do when this event fires is check for errors and call the createService method we created earlier.

Summary

We created our characteristics, wrapped them in a service and added all the code needed to make the GATT profile of our peripheral device visible to central devices. That's all we need for our lightsaber to become Bluetooth enabled.

I only really mention the Bluetooth controlling parts of the code in this post. If you would like to read more about how to implement the button and LED strip of the lightsaber, you can take a look at my post on Lights and sounds with the Raspberry Pi Zero.

If you would like to take a look at the complete lightsaber code, have a look at my GitHub repo.

I now have a fully functioning Bluetooth lightsaber, but am continuously working on improving it. So keep an eye on my blog and follow me on Twitter if you'd like to find out what I'm planning next.

Lights and sounds with the Raspberry Pi Zero

Gergana Young — Wed, 18 Dec 2019 13:43:30 +0000

IoT projects can be tons of fun, but figuring out all the hardware is not always easy. As I mentioned in my previous post, my team and I are building a lightsaber. A lightsaber requires at least two things, lights and sounds. In this post I will tell you how to set up an RGB LED strip (WS2812b), a sound hat (WM8960) and a button to control them both, on the Pi Zero. I will talk about the mistakes I made, and what you can do to avoid them, as well as the libraries I ended up using.

My Raspberry Pi Zero is running Raspbian Buster Lite, and I followed this tutorial to set it up.

Sound

Setting up the sound hat is actually not that difficult. You just need to install the drivers:

git clone https://github.com/waveshare/WM8960-Audio-HAT
cd WM8960-Audio-HAT
sudo ./install.sh
sudo reboot

After you reboot you should be able to test that the sound works with the following command speaker-test -c2 -twav -l7.

As you may have noticed from my previous posts, I am a big JavaScript fan, so that is why the first thing I tried was to play the sound I needed with a JavaScript library called play-sound. It is super easy to use:

const player = require('play-sound')(opts = {});

player.play('my-sound.wav', (err) => {
  if (err) console.log(err);
});

I did not have any problems with it (until I connected the button, but more on that later).

Lights

There are quite a lot of tutorials online about how to set up a WS2812b LED strip on your Raspberry Pi, some of them mention that it should be the exact same for the Pi Zero as well (it is). However, I had a few problems.

Firstly, I could not use the PWM (Pulse Width Modulation) pin, because that is the same pin that is being used by the sound. I eventually found a library that explained that if you need to use both sound and the LED strip, you can use the SPI (Serial Peripheral Interface) instead. I decided to use the Python version, since there was no JavaScript. I set it all up and nothing worked.

I thought that my next problem, was a power problem. A lot of the tutorials mention that if you have more than 30 LEDs (I had 45) you need external power. I measured with my multi-meter, and it showed that everything was fine, it wasn't a power problem. I was pulling my hair out, trying different libraries, trying just a single LED, nothing worked.

I looked a bit more closely, at one last tutorial, and realised something. Their strip had an arrow on it, for the direction that you are supposed to connect the wires. And then I looked closely at my LED strip:

There is an arrow, and my wires were soldered onto the wrong end of it.

So... If you solder your wire to the correct end of the LED strip, it's actually really easy to setup the Python library. You install it using pip sudo pip install rpi_ws281x and you can use it like this:

#!/usr/bin/env python3

import time
from rpi_ws281x import PixelStrip, Color

# LED strip configuration:
LED_COUNT = 45
LED_PIN = 10
LED_FREQ_HZ = 800000
LED_DMA = 10
LED_BRIGHTNESS = 255
LED_INVERT = False
LED_CHANNEL = 0

strip = PixelStrip(LED_COUNT, LED_PIN, LED_FREQ_HZ, LED_DMA, LED_INVERT, LED_BRIGHTNESS, LED_CHANNEL)
strip.begin()

for i in range(strip.numPixels()):
    strip.setPixelColor(i, 255, 0, 0)
    strip.show()
    time.sleep(0.0200)

This script turns on the LEDs (in red) one by one with a short delay in between. You can see that we are setting the LED_PIN to 10, that is the only SPI pin on the Pi. You have to also enable the SPI interface:

Run sudo raspi-config
Go to Interfacing Options
Select SPI
Select Yes
Reboot the Pi

You also need to change the maximum byte transfer and frequency of the SPI bus:

In /boot/cmdline.txt append spidev.bufsiz=32768
In /boot/config.txt add core_freq=250
Reboot the Pi

After this, if all is connected correctly, your LED strip will work (you do have to run it with sudo).

Button

The button should have been the easiest thing to connect. There are many libraries for GPIO (General Purpose Input Output) out there, some that I have used previously, so I thought I'd try them again, for example:

I tried JavaScript first, hence the first two, but I eventually settled for the last one. I was having this weird problem, and it took me a while to figure out that it was not the library's fault. Every time I ran the code for the button, the sound would stop working, on reboot, it would be fixed again. It took me hours to figure this out, but it has to do with the fact that the PWM pin on the Raspberry Pi Zero, is kind of a normal GPIO pin and there is software that makes it into a PWM pin. There are other pins like this on the Pi, and if you happen to connect your button to one of those, it breaks the sound. I'm not sure if this is documented anywhere, but the way I figured it out was via trial and error.

You install the library with pip using the command pip install RPi.GPIO and this is the code you can use to detect if the button (connected to GPIO pin 26) is pressed or not:

import RPi.GPIO as GPIO

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)

GPIO.setup(26, GPIO.IN, GPIO.PUD_UP)

if GPIO.input(26):
    print("pressed")
else:
    print("released")

In this code we are setting up the board and the pin that we are using. We are specifying that the pin is an input pin and then checking whether the button is pressed or not.

Summary

Those are the three main components that I needed working on the Raspberry Pi so that we could have a functioning lightsaber. There were quite a few problems along the way, but it's all working together now. I'm hoping that this tutorial will help you if you are struggling with any of the same problems. Keep an eye out here as well as on Twitter for future updates on how the lightsaber building is going (we're very close to V1.0).

Lightsaber prototyping with the Nordic Thingy:52

Gergana Young — Wed, 18 Dec 2019 13:42:34 +0000

We're building a lightsaber!!! I don't want to give too much away, but a few weeks ago we started with the 3D print… I decided that while I wait for all of the components to arrive, I would get started on a prototype.

Obviously, our lightsaber would have to be wireless, so I thought we could rely on Bluetooth. Since I have a Nordic Thingy:52 that I haven't played with yet, I thought it would be a great place to start for prototyping our lightsaber. It has the four things we need:

Lights - because it's in the name
Sound - to make it more cooler
Button - so it turns off when you drop it (also known as the dead Jedi switch)
Accelerometer - so we can detect movement

Connecting to the Thingy and using all of its Bluetooth attributes we can build a pretty good prototype. In order to identify all of these, I used this repo created by Nordic so that you can easily prototype with the Thingy:52. I decided it would be best to connect directly to the Bluetooth services and characteristics. In this way, when all of the components arrive, and we create our own custom Bluetooth lightsaber peripheral, we can just change the UUIDs and be up and running in no time!

If you'd like to know a bit more about how Bluetooth and Web Bluetooth work, you should check out the other two posts I wrote about these technologies:

In order to keep this post short, here are the links to the HTML and CSS we'll be using. We will concentrate more on writing and understanding the JavaScript. We want to accomplish the following:

Connect to the lightsaber and check it's battery status.
When the button is pressed - turn on the led and play the turning on sound
When the button is released - turn off the led and play the turning off sound.
Be able to change the colour of the led when the lightsaber is on.
Play different sounds when the lightsaber is being moved around using the accelerometer data.

Let's get started!

Connecting and battery

The first thing we need to do is initiate a scan for the device and connect to it. We also need to include all of the services that we might need in the optional services. Let's define a few variables so that we can interact with our HTML and so that we have all of the attribute UUIDs that we will need.

//buttons
const connectButton = document.getElementById('connectButton');
const disconnectButton = document.getElementById('disconnectButton');
const colourButton = document.getElementById('colourButton');

//divs shown and hidden based on lightsaber status
const connect = document.getElementById('connect');
const control = document.getElementById('control');
const off = document.getElementById('off');

//spans displaying information loaded from lightsaber
const batteryStatus = document.getElementById('batteryStatus');
const orientationX = document.getElementById('orientationX');
const orientationY = document.getElementById('orientationY');
const orientationZ = document.getElementById('orientationZ');

//services
const batteryServiceUuid = 'battery_service';
const motionServiceUuid = 'ef680400-9b35-4933-9b10-52ffa9740042';
const userInterfaceServiceUuid = 'ef680300-9b35-4933-9b10-52ffa9740042';
const soundServiceUuid = 'ef680500-9b35-4933-9b10-52ffa9740042';

//characteristics
const batteryCharUuid = 'battery_level';
const orientationCharUuid = 'ef680404-9b35-4933-9b10-52ffa9740042';
const ledCharUuid = 'ef680301-9b35-4933-9b10-52ffa9740042';
const btnCharUuid = 'ef680302-9b35-4933-9b10-52ffa9740042';
const soundConfigCharUuid = 'ef680501-9b35-4933-9b10-52ffa9740042';
const speakerCharUuid = 'ef680502-9b35-4933-9b10-52ffa9740042';

let device, batteryCharacteristic, orientationCharacteristic;
let ledCharacteristic, btnCharacteristic, soundConfigCharacteristic, speakerCharacteristic;

Now we can connect:

connectButton.onclick = async () => {
  device = await navigator.bluetooth.requestDevice({
    filters: [{ namePrefix: 'atc' }],
    optionalServices: [batteryServiceUuid, motionServiceUuid, userInterfaceServiceUuid, soundServiceUuid]
  });
  const server = await device.gatt.connect();

  batteryCharacteristic = await getCharacteristic(server, batteryServiceUuid, batteryCharUuid);
  orientationCharacteristic = await getCharacteristic(server, motionServiceUuid, orientationCharUuid);
  ledCharacteristic = await getCharacteristic(server, userInterfaceServiceUuid, ledCharUuid);
  btnCharacteristic = await getCharacteristic(server, userInterfaceServiceUuid, btnCharUuid);
  soundConfigCharacteristic = await getCharacteristic(server, soundServiceUuid, soundConfigCharUuid);
  speakerCharacteristic = await getCharacteristic(server, soundServiceUuid, speakerCharUuid);

  //what to do if the device is disconnected (code in next block)
  device.ongattserverdisconnected = disconnect;

  //display changes
  connected.style.display = 'block';
  connectButton.style.display = 'none';
  disconnectButton.style.display = 'initial';
};

const getCharacteristic = async (server, serviceUuid, characteristicUuid) => {
  const service = await server.getPrimaryService(serviceUuid);
  const char = await service.getCharacteristic(characteristicUuid);

  return char;
};

We also need to cater for disconnecting:

disconnectButton.onclick = async () => {
  await device.gatt.disconnect();
  disconnect();
};

const disconnect = () => {
  device = null;
  batteryCharacteristic = null;

  connected.style.display = 'none';
  connectButton.style.display = 'initial';
  disconnectButton.style.display = 'none';
};

Now let's read the initial percentage of the battery and also listen to the characteristic changes so that we know when the battery level changes:

const setUpDevice = async () => {
  //get initial battery value
  const batteryValue = await batteryCharacteristic.readValue();
  batteryStatus.innerText = batteryValue.getInt8(0);
};

const listen = () => {
  batteryCharacteristic.addEventListener('characteristicvaluechanged', (evt) => {
    const value = evt.target.value.getInt8(0);
    batteryStatus.innerText = value;
  });
  batteryCharacteristic.startNotifications();
};

Don't forget to call these two methods from the connect method:

connectButton.onclick = async () => {
  //...

  await setUpDevice();
  listen();
};

Test this out by running it using something like http-server.

Button and LED

We need to be able to turn the LED on and off based on whether the button is pressed or not. We do this by listening to the status of the button inside the listen() function we created earlier:

let ledColour = new Uint8Array([1, 0, 0, 255]);
let lightsaberOn = false;
const toggleLed = async (toggle) => {
  if (toggle) {
    await ledCharacteristic.writeValue(ledColour);
    lightsaberOn = true;
    control.style.display = 'block';
    off.style.display = 'none';
  } else {
    await ledCharacteristic.writeValue(new Uint8Array([0]));
    lightsaberOn = false;
    control.style.display = 'none';
    off.style.display = 'block';
  }
};

const listen = () => {
  //...

  btnCharacteristic.addEventListener('characteristicvaluechanged', (evt) => {
    const value = evt.target.value.getInt8(0);
    toggleLed(value);
  });
  btnCharacteristic.startNotifications();
};

We should also turn everything off when we start up by calling this function inside our setUpDevice() function:

const setUpDevice = async () => {
  //...

  //turn off when starting up
  await toggleLed(false);
};

Lets also allow our Jedi to change the colour of their lightsaber if they want to. We already have a colour picker and button in the HTML and we can use them like this:

const hexToRgb = (hex) => {
  const r = parseInt(hex.substring(1, 3), 16); //start at 1 to avoid #
  const g = parseInt(hex.substring(3, 5), 16);
  const b = parseInt(hex.substring(5, 7), 16);

  return [r, g, b];
};

colourButton.onclick = async () => {
  ledColour = new Uint8Array([1, ...hexToRgb(colourPicker.value)]);
  ledCharacteristic.writeValue(ledColour);
};

Sound

Since we are going to have a set of pre-recorded lightsaber sounds, I thought it would be easiest to use the sample sounds on the Thingy:52. To do that we need to change the code we wrote above to start up the speaker:

const setUpDevice = async () => {
  //...

  //turn on speaker
  const dataArray = new Uint8Array(2);
  dataArray[0] = 3 & 0xff;
  dataArray[1] = 1 & 0xff;
  await soundConfigCharacteristic.writeValue(dataArray);
};

We are going to use the first sample as the turning on sound and the second as the turning off sound:

const getSampleSound = (sound) => {
  return new Uint8Array([sound & 0xff]);
};

const toggleLed = async (toggle) => {
  if (toggle) {
    //...
    await speakerCharacteristic.writeValue(getSampleSound(0));

  } else {
    //...
    await speakerCharacteristic.writeValue(getSampleSound(1));

  }
};

Accelerometer

Lastly, we are going to use the motion of the lightsaber to trigger a few other sounds. We will first print out the X, Y and Z co-ordinates on the screen so we can see them changing. Add the following to the listen() function:

const listen = () => {
  //...

  orientationCharacteristic.addEventListener('characteristicvaluechanged', (evt) => {
    let data = evt.target.value;
    let w = data.getInt32(0, true) / (1 << 30);
    let x = data.getInt32(4, true) / (1 << 30);
    let y = data.getInt32(8, true) / (1 << 30);
    let z = data.getInt32(12, true) / (1 << 30);
    const magnitude = Math.sqrt(Math.pow(w, 2) + Math.pow(x, 2) + Math.pow(y, 2) + Math.pow(z, 2));

    if (magnitude !== 0) {
      x /= magnitude;
      y /= magnitude;
      z /= magnitude;
    }

    playSound(x, y, z);

    orientationX.innerText = `X: ${x.toFixed(2)}`;
    orientationY.innerText = `Y: ${y.toFixed(2)}`;
    orientationZ.innerText = `Z: ${z.toFixed(2)}`;
  });
  orientationCharacteristic.startNotifications();
};

The calculation above is directly from the Nordic repo, I did some reading on what quaternion rotation is, but I will not try to explain it.

Now we can choose a different sample per movement and play them as the lightsaber moves around:

let position = { x: 0, y: 0, z: 0 };
const playSound = async (x, y, z) => {
  if (lightsaberOn) {
    if (Math.abs(position.x - x) > 0.2) {
      await speakerCharacteristic.writeValue(getSampleSound(2));
    } else if (Math.abs(position.y - y) > 0.2) {
      await speakerCharacteristic.writeValue(getSampleSound(3));
    } else if (Math.abs(position.z - z) > 0.2) {
      await speakerCharacteristic.writeValue(getSampleSound(4));
    }
  }

  position.x = x;
  position.y = y;
  position.z = z;
};

We've also added a check in there to make sure that our lightsaber doesn't continue to make sounds when it turns off.

Summary

Wow, that was quite a lot of code. If you got lost anywhere along the way you can check out the complete code on GitHub. We now have a lightsaber prototype. It has all of the things we require from our lightsaber, lights, sounds, dead Jedi switch. It just doesn't really look like a lightsaber, but that's because it's just a prototype, right?

I will definitely be posting more about this, and so will the rest of my team. So keep an eye out on this blog and my team's Twitter account for more updates on the complete fully functioning lightsaber.

Restify, lit-html and Azure Pipelines

Gergana Young — Wed, 18 Dec 2019 13:40:45 +0000

Azure DevOps (formerly TFS) is a set of services created by Microsoft to make CI/CD easier for projects. We have been using it for the last seven months as source control. Recently, we had to deploy one of our old applications to an Azure Web App and we decided it was time to try some of the other features of Azure DevOps, the Pipelines. The pipeline we created ensures that every push to the master branch of our project goes straight to production (no pressure).

I went through a few different tutorials on how to build and deploy using the pipelines, but I kept missing something and things were just not working. This is why I decided to write this post with a step by step guide on what I did, in case someone else runs into the same problems I did.

We will be deploying a very simple web app which uses LitElement and restify. We will start by putting the code in a private repo on Azure DevOps. We will then create a Web App in Azure. The last two things we will need are the two pipelines, one that will build our code and one that will deploy it to Azure on a successful build. Let's get started.

The app

The app we will be using can be found in this GitHub repo. Using Azure Pipelines it is possible to deploy directly from GitHub, but in this example I would like to show you how we can do it from a repo on Azure DevOps. Get the code from the repo above and let's start by creating a project. Go to Azure DevOps and create an account if you do not have one already. Create a new project and click on repos, follow the instructions to create a repo and commit the code above into that repo. Once our code is pushed, we are ready to start with the deployment.

The web app

We need to create an app service plan and a web app on Azure. Open the Azure portal and click on create new resource, search for Web App and click create.

Fill in all of the required details, remembering to choose the publish method as code and the operating system as Linux.

Once the web app is created, we have to add the port on which it will run. We do this by going to the Configuration settings and under "Application Settings" add a new setting with name "PORT" and value "8080". Now we can start the deployment.

The build pipeline

The first thing we need to do is create a build pipeline that will build our code (and test it when we add tests). We need to create a file in the root of our directory named azure-pipelines.yml. This file will define all of the steps we need to build our code. It will look like this:

trigger:
  - master

pool:
  vmImage: 'Ubuntu-16.04'

steps:
  - task: NodeTool@0
    inputs:
      versionSpec: '10.x'
    displayName: 'Install Node.js'

  - script: |
      npm install
      npm run-script build
    workingDirectory: app
    displayName: 'Install the app dependencies and build the bundle'

  - script: |
      npm install
    workingDirectory: api
    displayName: 'Install the server dependencies'

  - task: CopyFiles@2
    inputs:
      SourceFolder: '$(System.DefaultWorkingDirectory)/api'
      Contents: |
        **/*
        *
        !node_modules/
      TargetFolder: '$(Build.ArtifactStagingDirectory)/api'

  - task: CopyFiles@2
    inputs:
      SourceFolder: '$(System.DefaultWorkingDirectory)/app/build/default'
      Contents: |
        **/*
        *
        !node_modules/
      TargetFolder: '$(Build.ArtifactStagingDirectory)/api/public'

  - task: DeleteFiles@1
    inputs:
      SourceFolder: '$(Build.ArtifactStagingDirectory)/api/public'
      Contents: src/services/config.js

  - task: CmdLine@2
    inputs:
      Script: 'mv config.prod.js config.js'
      WorkingDirectory: '$(Build.ArtifactStagingDirectory)/api/public/src/services'

  - task: PublishBuildArtifacts@1
    inputs:
      pathtoPublish: '$(Build.ArtifactStagingDirectory)/api'
      artifactName: api

There are eight steps in total, they do the following:

Step 1: Install Node on the build agent.

- task: NodeTool@0
  inputs:
    versionSpec: '10.x'
  displayName: 'Install Node.js'

Step 2: Install the dependencies for the web app and build it using the polymer cli.

- script: |
    npm install
    npm run-script build
  workingDirectory: app
  displayName: 'Install the app dependencies and build the bundle'

Step 3: Install the dependencies for the API.

- script: |
    npm install
  workingDirectory: api
  displayName: 'Install the server dependencies'

Step 4: Copy all of the api files to the build directory.

- task: CopyFiles@2
  inputs:
    SourceFolder: '$(System.DefaultWorkingDirectory)/api'
    Contents: |
      **/*
      *
      !node_modules/
    TargetFolder: '$(Build.ArtifactStagingDirectory)/api'

Step 5: Copy all of the app files to the public folder in the api build directory.

- task: CopyFiles@2
  inputs:
    SourceFolder: '$(System.DefaultWorkingDirectory)/app/build/default'
    Contents: |
      **/*
      *
      !node_modules/
    TargetFolder: '$(Build.ArtifactStagingDirectory)/api/public'

Step 6: Delete the development configuration file.

- task: DeleteFiles@1
  inputs:
    SourceFolder: '$(Build.ArtifactStagingDirectory)/api/public'
    Contents: src/services/config.js

Step 7: Rename the production configuration file.

- task: CmdLine@2
  inputs:
    Script: 'mv config.prod.js config.js'
    WorkingDirectory: '$(Build.ArtifactStagingDirectory)/api/public/src/services'

Step 8: Publish all of the files from the build directory.

- task: PublishBuildArtifacts@1
  inputs:
    pathtoPublish: '$(Build.ArtifactStagingDirectory)/api'
    artifactName: api

If we push this file to Azure DevOps and go into Pipelines > Builds, we will see our build pipeline.

The release pipeline

Now that our code is building successfully, it is time to deploy it to Azure. We will do that by using a release pipeline that is triggered by a successful build. Click on Pipelines > Releases > Create a new pipeline. In the select a template dialog choose "Azure App Service deployment".

Name the stage and then click on "Add an artifact".

Choose the source type as "Build" and choose the build pipeline we created earlier. Set the default version to latest and click add.

Click on the "Continuous deployment trigger" and set it to enabled, this way every time there is a successful build the release pipeline will be triggered.

Go into the stage settings by clicking on the link "1 job, 1 task". Choose your Azure subscription and change the app type to "Web App on Linux". This should load the app service we created earlier, select it from the drop down and leave the start up command blank.

Click on the "Deploy Azure App Service Task", here we can rename it to something more relevant, or leave it as is. The important thing that needs to change is the "Package or folder" location. Click on the 3 dots and you will see the artifacts that the build pipeline published. Navigate to the api folder and choose that as the folder to deploy. We can leave all the other settings and save our release pipeline.

To test it create a release from the latest build. Once the release completes, check that everything worked by going to <yourwebsitename>.azurewebsites.net.

Summary

We have now successfully deployed our web app to Azure. Every time we push to the master branch on Azure DevOps our new code will be released to production. This can be both very useful and very dangerous. We need to ensure that every person on the team understands that all of their changes to the master branch will be immediately available for people to see and use. The code should be well tested and there should be some sort of pull request model in place. In this way we don't have to fear pushing to master and breaking something in production.

Forms, Web Components and Redux

Gergana Young — Tue, 17 Dec 2019 07:14:20 +0000

My team and I have been working on a project using the PWA Starter Kit since the beginning of this year. I've learned so much on this project, how to create performant web components, how to embrace Redux (instead of fighting it) and how to secure the whole application using Azure Active Directory, just to name a few. Although we did get stuck a number of times on a few different things (which we plan to write about in the future), nothing stumped me more than building a form with validation. I think this was because I was thinking along the lines of "But it's just a form, we build forms on the web every other day". By the end of this task, I went to my team with a big smile on my face and said "I Reduxed the hell out of that form".

In this post I'd like to share with you what I did to get our form to work the way we wanted it.

The end goal is to be able to create or update an object named Quest which consists of one or more Missions. Our object structure will look something like this:

Quest {
  goal: string
}

Mission {
  name: string,
  description: string
}

The first mission

The first component will be an HTML form that creates a mission object for us, it will have a property to store errors and it will not allow you to submit the form if there are errors on the page. The code for this looks as follows:

import { LitElement, html } from 'lit-element';

export class MissionsForm extends LitElement {
  constructor() {
    super();
    this.errors = [];
  }

  static get properties() {
    return {
      errors: Array
    };
  }

  render() {
    const hasError = (name) => (this.errors.indexOf(name) >= 0 ? 'error' : '');

    return html`
      <style>
        .error {
          border: 1px solid red;
        }
      </style>

      <form @submit="${(e) => this.submit(e)}">
        <div>
          <label>Name: </label>
          <input class="${hasError('name')}" type="input" name="name"/>
        </div>
        <div>
          <label>
            Description:
          </label>
          <textarea class="${hasError('description')}" name="description"></textarea>
        </div>
        <div>
          <button type="submit">Save</button>
        </div>
      </form>
    `;
  }

  submit(e) {
    e.preventDefault();
    let form = e.target;
    this.errors = this.checkForErrors(form);

    if (!this.errors.length) {
      let mission = {
        name: form.name.value,
        description: form.description.value
      };

      //save mission here
      form.reset();
    }
  }

  checkForErrors(form) {
    let errors = [];

    if (!form.name.value) {
      errors.push('name');
    }

    if (!form.description.value) {
      errors.push('description');
    }

    return errors;
  }
}

customElements.define('missions-form', MissionsForm);

This works fine, because every time we trigger the submit, if there are any errors, we update the property which causes the page to re-render and show us those errors. However, it's not great that the only time the errors will disappear is if the user clicks submit again. We want them to know that the error is fixed as soon as they have fixed it. In order to do that we must listen to the change event on the form:

<form @submit="${(e) => this.submit(e)}" @change="${(e) => this.formValueUpdated(e)}">
  <!--...-->
</form>

We can now remove the errors as soon as they are fixed by implementing the method:

formValueUpdated(e) {
  let errorList = [...this.errors];
  if (!e.target.value) {
    errorList.push(e.target.name);
  } else {
    let indexOfError = errorList.indexOf(e.target.name);
    if (indexOfError >= 0) {
    errorList.splice(indexOfError, 1);
    }
  }
  this.errors = [...errorList];
}

Adding more missions

What we need to do next is implement the //save mission here method. In order to do that we will first make a new component, this new component will have our list of missions and it will also contain our form component. The basic outline will look like this:

import { LitElement, html } from 'lit-element';

import './missions-form.component';

export class MissionsList extends LitElement {
  constructor() {
    super();
    this.missions = [];
  }

  static get properties() {
    return {
      missions: Array
    };
  }

  render() {
    return html`
      <h2>Missions</h2>
      <ul>
        ${this.missions.map(
          (m) =>
            html`
              <li><strong>${m.name}:</strong> ${m.description}</li>
            `
        )}
      </ul>
      <missions-form></missions-form>
    `;
  }
}

customElements.define('missions-list', MissionsList);

We are going to use Redux to update our list of missions whenever save is clicked in the form component. If you are using the PWA Starter Kit, then you already have all of the Redux plumbing set up for you. If you started from scratch, follow this tutorial to help you set it up. The following is the first version of our reducer:

import { MISSIONS_UPDATED } from "../actions/missions-updated.action";

const INITIAL_STATE = {
  missions: []
};

export const editor = (state = INITIAL_STATE, action) => {
  switch (action.type) {
    case MISSIONS_UPDATED:
      return {
        ...state,
        missions: action.missions
      }
    default:
      return state;
  }
}

This reducer imports an action, let's implement that action:

export const MISSIONS_UPDATED = 'MISSIONS_UPDATED';

export const missionsUpdated = (missions) => {
  return {
    type: MISSIONS_UPDATED,
    missions
  };
};

Now, whenever save is clicked we will need to dispatch that action. This means that we will need to change our MissionsList component to connect it to the Redux store:

export class MissionsList extends connect(store)(LitElement) {
  //...
}

And our MissionsForm component will also need to be connected:

export class MissionsForm extends connect(store)(LitElement) {
  //...
}

Both of these components need to implement the stateChanged method:

stateChanged(state) {
  this.missions = state.missions;
}

Here we are accessing the missions directly from the state. In my project we use reselect, which is a middleware for creating optimised selectors. To see some of the things we did to improve our performance and make our code less complex, checkout my colleague's article on Wrangling Redux.

The last thing left to do is to replace that comment with a call to our action and update the list of missions:

store.dispatch(missionsUpdated([...this.missions, mission]));

Quest

Our next component will be in charge of gathering information about the quest. In our example the quest only has one property, however, the code is written in such a way that this can be extended. Let's create a QuestEditor component:

import { LitElement, html } from 'lit-element';
import { connect } from 'pwa-helpers';
import { store } from '../store';
import { questUpdated } from '../actions/quest-updated.action';
import { errorsDetected } from '../actions/errors-detected.action';

export class QuestEditor extends connect(store)(LitElement) {
  constructor() {
    super();
    this.errors = [];
  }

  static get properties() {
    return {
      quest: Object,
      errors: Array
    };
  }

  render() {
    const hasError = (name) => (this.errors.indexOf(name) >= 0 ? 'error' : '');

    return html`
      <style>
        .error {
          border: 1px solid red;
        }
      </style>

      <form @change="${(e) => this.formValueUpdated(e)}" @submit="${(e) => e.preventDefault()}">
        <div>
          <label>Goal:</label>
          <input class="${hasError('goal')}" name="goal" type="text" />
        </div>
      </form>
    `;
  }

  formValueUpdated(e) {
    let errorList = [...this.errors];
    if (!e.target.value) {
      errorList.push(e.target.name);
    } else {
      let indexOfError = errorList.indexOf(e.target.name);
      if (indexOfError >= 0) {
        errorList.splice(indexOfError, 1);
      }
    }

    let quest = {
      ...this.quest,
      [e.target.name]: e.target.value
    };

    store.dispatch(errorsDetected(errorList));
    store.dispatch(questUpdated(quest));
  }

  stateChanged(state) {
    this.quest = state.quest;
    this.errors = state.errors;

    if (!this.quest) {
      this.quest = {
        goal: ''
      };
    }
  }
}

customElements.define('quest-editor', QuestEditor);

This component is very similar to the one we created for missions, the big difference is that this component does not have a save button. This is because we want to save the quest and missions at the same time (which we will do in another component in a moment). The QuestEditor component also has two new actions errorsDetected and questUpdated. We can implement them as follows:

export const ERRORS_DETECTED = 'ERRORS_DETECTED';

export const errorsDetected = (errors) => {
  return {
    type: ERRORS_DETECTED,
    errors
  };
};

and

export const QUEST_UPDATED = 'QUEST_UPDATED';

export const questUpdated = (quest) => {
  return {
    type: QUEST_UPDATED,
    quest
  };
};

We also need to update our reducer to cater for these two actions, first we change our INITIAL_STATE to:

const INITIAL_STATE = {
  quest: {},
  missions: [],
  errors: []
};

Then add two more cases to our switch statement:

case QUEST_UPDATED:
  return {
    ...state,
    quest: action.quest
  }
case ERRORS_DETECTED:
  return {
    ...state,
    errors: action.errors
  }

Putting it all together

We have to combine what we have done in one "main" component, this component will be called Quest and will look as follows:

import { LitElement, html } from 'lit-element';
import { connect } from 'pwa-helpers';
import { store } from '../store';
import { errorsDetected } from '../actions/errors-detected.action';

import './quest-editor.component';
import './missions-list.component';

export class Quest extends connect(store)(LitElement) {
  render() {
    return html`
      <h1>Create Quest</h1>
      <quest-editor></quest-editor>
      <missions-list></missions-list>
      <div>
        <button type="button" @click="${() => this.saveQuest()}">Save</button>
      </div>
    `;
  }

  saveQuest() {
    let errors = this.pageValid();
    if (!errors.length) {
      //save quest and missions here
    }
    store.dispatch(errorsDetected(errors));
  }

  pageValid() {
    let errors = [];

    if (!this.quest.goal) {
      errors.push('goal');
    }

    if (!this.missions.length) {
      errors.push('missions');
    }

    return errors;
  }

  stateChanged(state) {
    this.missions = state.missions;
    this.quest = state.quest;
  }
}

customElements.define('my-quest', Quest);

The Quest component is in charge of saving the things we have filled in. It needs to know about both the quest and the missions. However, you may have noticed that this component does not have any of its own properties, this is because we do not need to re-render it when quest, missions or errors change. We also need to make sure we have filled in all of the details correctly, the pageValid method is doing that for us. Lastly, if there are no errors, we can save everything (//save quest and missions here).

Some cleaning up

We are almost done, there are a few more small things we have to handle. Let's start by displaying the missions error in the MissionsList component. To do that we need to:

Add errors as a property:

  static get properties() {
    return {
      missions: Array,
      errors: Array
    };
  }

Initialise it to an empty array in the constructor:

  constructor() {
    super();
    this.missions = [];
    this.errors = [];
  }

Set it in the stateChanged method:

  stateChanged(state) {
    this.missions = state.missions;
    this.errors = state.errors;
  }

Create a new method to render our error message:

  hasError() {
    return this.errors.indexOf('missions') >= 0
      ? html`
          <div class="error">There must be at least one mission in every quest!</div>
        `
      : html``;
  }

Call that method inside our render method:

  render() {
    return html`
      <style>
        .error {
          color: red;
        }
      </style>

      <h2>Missions</h2>
      ${this.hasError()}
      <ul>
        ${this.missions.map(
          (m) =>
            html`
              <li><strong>${m.name}:</strong> ${m.description}</li>
            `
        )}
      </ul>
      <missions-form></missions-form>
    `;
  }

The last thing we have to do is some cleaning up in our MissionsForm component so that it follows the same pattern as the others. To do this we need to change:

The stateChanged to get the errors from state:

  stateChanged(state) {
    this.missions = state.missions;
    this.errors = state.errors;
  }

The formValueUpdated method to dispatch an action instead of changing the property directly:

  formValueUpdated(e) {
    let errorList = [...this.errors];
    if (!e.target.value) {
      errorList.push(e.target.name);
    } else {
      let indexOfError = errorList.indexOf(e.target.name);
      if (indexOfError >= 0) {
        errorList.splice(indexOfError, 1);
      }
    } 
    store.dispatch(errorsDetected(errorList));
  }

And the submit method to do the same:

  submit(e) {
    e.preventDefault();
    let form = e.target;
    let errors = this.checkForErrors(form);

    if (!errors.length) {
      let mission = {
        name: form.name.value,
        description: form.description.value
      };

      store.dispatch(missionsUpdated([...this.missions, mission]));
      form.reset();
    }

    store.dispatch(errorsDetected(errors))
  }

Summary

That's all we need to get our forms working with LitElement and Redux. From here on it is possible to implement any other CRUD operations. You can take a look at the full example on my GitHub repo. The example will be updated with editing and deleting missions as well as editing quest.

Web Bluetooth by example

Gergana Young — Tue, 17 Dec 2019 07:13:05 +0000

The Web Bluetooth API is quite a powerful feature of the web. It has a lot of potential and some really cool capabilities. However, getting started with it can be a little daunting. In this post I'd like to take you through building an example using the API with the nRF52 dongle. Before starting with the code, if you would like to know more about the theory of the Web Bluetooth API, checkout my previous post: BLE and GATT and other TLAs.

We are going to be following a very similar example to my WebUSB by example post, with most of the code adapted from Lars Knudsen's GitHub repo. We will be building a website, from scratch, that will connect to the Bluetooth device, send data to it to update the colour of the LED and receive data that will be displayed on the screen.

What you will need

nRF52 dongle
nrfutil: for flashing the firmware onto the device
http-server (or something similar): for starting up our website locally
A computer which has a built in Bluetooth module, or a USB Bluetooth dongle. I use this one.

The hardware

We need to ensure that the nRF52 dongle is flashed with the correct software so that we can access its Bluetooth capabilities. If you have already done this, by following the instructions in my WebUSB post, then you can skip to the next section. If you have not, then please keep reading.

We will be using Zephyr, which is an operating system for small embedded devices. If you are interested in building the firmware and installing it yourself then you can take a look at the instructions on Getting Started with Zephyr. Once you have it setup you will have to build the firmware and flash it onto the dongle by following the instructions here.

This tutorial is going to focus on the Web Bluetooth API, so you can just download the packaged zip file from the repo and flash it onto the device with the command nrfutil dfu serial -pkg pkg.zip -p /dev/ttyACM0 replacing the /dev/ttyACM0 with the USB port that your device is plugged into.

The Setup

As mentioned earlier, we are going to be building this website almost completely from scratch. Since the parts we are concentrating on will be the JavaScript, you can download the HTML and CSS and just use them. There are a few things that need to be mentioned in the HTML file that we will be referring to when we get to writing our JavaScript:

Three buttons with IDs: connectButton, disconnectButton and colourButton.
A div with ID connected.
An input of type color with ID colourPicker.
A span with ID deviceHeartbeat.

Let's create an index.js file and start by getting references to all of the elements we defined in our HTML:

const connectButton = document.getElementById('connectButton');
const disconnectButton = document.getElementById('disconnectButton');

const colourPicker = document.getElementById('colourPicker');
const colourButton = document.getElementById('colourButton');

const connect = document.getElementById('connect');
const deviceHeartbeat = document.getElementById('deviceHeartbeat');

Next we should create variables for the service and characteristic unique identifiers for our BLE device. We have one service which we will be connecting to, and two characteristics, one for sending data and one for receiving data.

const primaryServiceUuid = '12345678-1234-5678-1234-56789abcdef0';
const receiveCharUuid = '12345678-1234-5678-1234-56789abcdef1';
const sendCharUuid = '12345678-1234-5678-1234-56789abcdef3';

Now we are ready to connect to the device. When we click the connect button our browser will scan for the device, once connected we can show the rest of the elements on the page so we can interact with the device:

let device, sendCharacteristic, receiveCharacteristic;
connectButton.onclick = async () => {
  device = await navigator.bluetooth
            .requestDevice({ 
                filters: [{ 
                    services: [primaryServiceUuid] 
                }] 
            });

  const server = await device.gatt.connect();
  const service = await server.getPrimaryService(primaryServiceUuid);

  receiveCharacteristic = await service.getCharacteristic(receiveCharUuid);
  sendCharacteristic = await service.getCharacteristic(sendCharUuid);

  device.ongattserverdisconnected = disconnect;

  connected.style.display = 'block';
  connectButton.style.display = 'none';
  disconnectButton.style.display = 'initial';
};

In the code above, we are also listening for the ongattserverdisconnected event. This can happen if the Bluetooth device goes out of range, or is disconnected for any reason. When we disconnect from the device we want to clean up, we do this as follows:

const disconnect = () => {
  device = null;
  receiveCharacteristic = null;
  sendCharacteristic = null;

  connected.style.display = 'none';
  connectButton.style.display = 'initial';
  disconnectButton.style.display = 'none';
};

Another way we can disconnect from the device is by clicking the disconnect button. We can reuse the disconnect method above, but we also need to tell the browser we are disconnecting from the device:

disconnectButton.onclick = async () => {
  await device.gatt.disconnect();
  disconnect();
};

Start up your website using http-server. If the device is powered on and your Bluetooth is turned on, you should be able to connect to it and disconnect from it. When connected, you should see a little Bluetooth logo in your browser tab:

Changing the LED colour

The first thing we are going to do is send data from our computer to the device. We are going to send different colours to the on-board LED and watch it change. Since we are using a standard HTML colour picker we need to convert the hex values it gives us to RGB values. The below function will take care of this for us:

const hexToRgb = (hex) => {
  const r = parseInt(hex.substring(1, 3), 16); //start at 1 to avoid #
  const g = parseInt(hex.substring(3, 5), 16);
  const b = parseInt(hex.substring(5, 7), 16);

  return [r, g, b];
};

We have our send characteristic, so let's use it to send data:

colourButton.onclick = async () => {
  const data = new Uint8Array([1, ...hexToRgb(colourPicker.value)]);
  sendCharacteristic.writeValue(data);
};

All we have to do is create a data array which starts with a 1, because that is what the firmware is expecting and is followed by the three values for red, green and blue. Try running the code again and check that your LED can change colours.

Device heart beat

As long as the device is connected it will continue to send a heart beat, we just need to listen for it. To do that, we will implement a listen() method:

const listen = () => {
  receiveCharacteristic
    .addEventListener('characteristicvaluechanged', 
      (evt) => {
        const value = evt.target.value.getInt16(0, true);
        deviceHeartbeat.innerText = value;
      });

  receiveCharacteristic.startNotifications();
};

This method does two things, it adds an event listener for characteristicvaluechanged, so that every time the characteristic updates its value we will update the text on our page. It also uses the startNotifications() method which tells the device that it can start sending us data. The last thing we need is to call the listen method from within the connectButton.onclick method, that way as soon as we connect to the device we can start listening to its heart beat.

That's all we need to be able to send and receive data to and from the nRF52 dongle over bluetooth. Start up your app one last time to check that everything is working. If something has gone wrong you can checkout the full solution.

Conclusion

I hope this example has made using the Web Bluetooth API a little less daunting. You can find a whole range of examples on the Web Bluetooth Community Group's GitHub. There are also a few Bluetooth devices I've been playing with in my GitHub repos:

There are so many Bluetooth devices out there, and being able to connect to them using just our browsers is really exciting. With the use of Bluetooth 5 on the rise, I am hoping that support for the Web Bluetooth API will increase, and soon we will be able to use it everywhere. This technology is really exciting and I can't wait to see where it takes us next.

DEV Community: Gergana Young

Authorisation with Casbin and Koa Part 3

The exceptions

The middleware

Using the middleware

A note on testing

Summary

Authorisation with Casbin and Koa Part 2

Koa

Node-Casbin

The middleware

Using the middleware

Summary

Authorisation with Casbin and Koa Part 1

Casbin

RBAC

Config definition

Policy definition

Summary

Puppeteer in an AWS Lambda Function Part 2

Creating your layer

Using your layer

Conclusion

Puppeteer in an AWS Lambda Function Part 1

Using puppeteer-core

Conclusion

Online card games for quarantine

The frontend

The server

Communication

Data

Hosting

Security

Conclusion

Smart Lightsaber - Bluetooth peripherals with JavaScript

Installing bleno

Building the characteristics

Light characteristic

Button Characteristic

Adding in our service

GATT Profile

Summary

Lights and sounds with the Raspberry Pi Zero

Sound

Lights

Button

Summary

Lightsaber prototyping with the Nordic Thingy:52

Connecting and battery

Button and LED

Sound

Accelerometer

Summary

Restify, lit-html and Azure Pipelines

The app

The web app

The build pipeline

The release pipeline

Summary

Forms, Web Components and Redux

The first mission

Adding more missions

Quest

Putting it all together

Some cleaning up

Summary

Web Bluetooth by example

What you will need

The hardware

The Setup

Changing the LED colour

Device heart beat

Conclusion

Using `puppeteer-core`