DEV Community: Adam K Dean

BBC Micro:bit — a story of curiosity

Adam K Dean — Sat, 12 Nov 2022 20:22:17 +0000

I have a six year old (let's call him v2.0). His brain never stops, and my work as an engineer (primarily software) seems to be of great interest to him. He asks lots of questions, but also knows during incidents that I need to focus. After a particularly difficult incident a few months ago, he congratulated me on fixing the "glitch" and called me a "glitch hunter".

I've been looking at ways to introduce him to the wonderful world of building things: programming, making, hacking, the amazing world that we live in where we both learn and create things on a daily basis. I thought perhaps Python, but the IDEs are too complicated. I thought JavaScript, but should a six year old be introduced to NPM and dependency hell at such a young age? (Yes officer, this parent here.)

I decided to search, to scour the internet, for something that would fulfil requirements I'd only know when I saw them, and came across the BBC micro:bit, a pocket-sized computer designed for 11-12 year olds (UK year 7, US 6th grade).

BBC Micro:bit

The microbit is a great little device, the spiritual successor to the BBC micro from the days of yore, designed to introduce children to both software and hardware.

Onboarding starts with a Scratch-like block coding language, and progresses onto Python (MicroPython, in fact). Microsoft provide an online IDE MakeCode and there's a wealth of tutorials and guides.

But you can do more with it. You can use JavaScript, Pascal, Simulink, C++, Forth, Lisp, Rust, Ada, Swift, Basic, and even flash it with Zephyr, a lightweight OS.

While it's really aimed more towards 11-12 year olds, I figured 6 is half of 12 and—armed with this excuse and my own share of excitement (who doesn't love to buy electronics?)—I bought a microbit, and a few extras.

First, the Inventor's Kit, which comprises of 10 experiments, from a physical hello world to handling human device input through to wind power and capacitor charge circuits.

I also decided to buy a robot kit, because robots.

Initial test

The kit arrived and we opened it up. We plugged into the micro-usb and the intro sequence started. I won't spoil it, but it's very cool.

Now it's not until you try to explain something to a six year old that you realise just how much knowledge you've acquired over the year. Most of us sit here day to day with varying degrees of imposter syndrome, but that's because we become unconsciously competent in our various fields.

Having had many discussions about electronics, and completing a few of the experiments in the inventors kit, we're now at the point where v2.0 understands (to an age appropriate level) voltage, current, resistance, and diodes. We're still working on capacitors and transistors, and have much more to get to, in the future.

The experience so far has been amazing. It's been a great educational tool, playing around with the LED matrix, the onboard speaker, the onboard microphone... and then I discovered, the onboard radio.

Radio

Now, I'm a bit of a geek. I've always loved making things, but the thing which has always amazed me is communication. I remember writing my first network program at the age of 11 or 12, and communicating with a friend a few hundred miles away in London via UDP packets. Everything grew from there.

The pandemic wasn't quite the same for me as for most people. I already worked from home, having been fully remote since 2016, and I already baked bread, because carbs are great. So I decided to pick up radio (UK amateur radio, US HAM radio) as a hobby. It's amazing, but a little bit off-topic for this post (though amazing to think that I was able to make contact (albeit one way) with the Halley VI research station in Antartica via a 1mm thick, 40-ish ft long piece of speaker wire!)

So while playing with the microbit, I discovered that it has a radio module built in. It doesn't have a WiFi chip, but it has 2.4 GHz onboard transceiver, and is able to communicate up to 4 dBm (2.5 mW) with other microbits, at a rate of 1-2 mbps. Just think of the possibilities. This seemed to awaken something inside me, and while I'm 33 years old and the microbit is designed for 11-12 year olds, 33 divided by 11 is 3, and you can't argue with maths, so I bought three more microbits and some more components.

And today, today I started tinkering (more on that soon), and somehow 10:00 has turned into 20:21 and I've had a blast. We all need something from time to time to bring back the spark that made us excited all those years ago. So I'll probably be writing about my experiments, my forays into electronics, hardware, software, firmware, all the wares, and buying more wares, while the fire of passion burns.

Here's to curiosity voyages!

10 Tips For New Coders

Adam K Dean — Fri, 07 Oct 2022 13:45:41 +0000

I've been reading and writing code since the year 2000. Before we had Stack Overflow and Codeacademy, we had two real resources for learning: giant dusty tomes filled with scraps of paper, and websites like PlanetSourceCode where you could download zips of entire projects.

I didn't always have access to the internet back then, but I had an old computer, a collection of 3½-inch 1.44 MB floppy disks, and legs, so I'd walk to friends houses and download as much code as I could fit on the floppy disks then go home to take it all apart. When I got stuck, I'd call my mentor who had a seemingly endless supply of both enthusiasm and patience.

I can't recall much of what I created during those formative years, but I do remember the process I followed.

So here are 10 tips for new coders.

1. Don't be afraid to ask for help

Nobody knows everything and everyone was a beginner once. Asking for help is a perfectly normal and acceptable thing to do when coding.

2. Start small

Don’t try to tackle a huge project right off the bat. Start with something small and simple to get the hang of the basics. Building a proof of concept is much easier than building a product, so build small and build often.

3. Read other people's code

One of the best ways to learn how to code is to read other people's code. Not only will you learn new techniques, but you'll also get a feel for how other people think about coding and problem solving.

4. Write your own code

Don't just copy and paste other people's code. Try to write your own code as much as possible. Not only will this help you learn, but you'll also develop your own style and approach to coding.

5. Experiment

Coding is all about trial and error. Don't be afraid to experiment with different code to see what works and what doesn't.

6. Take breaks

Coding can be mentally taxing. Make sure to take breaks every now and then to clear your head and come back refreshed. Here's an interesting article I read recently on why this may be.

7. Organise your code

Keep your code organised and tidy. Not only will this make it easier for you to find things, but it will also make it easier for others to read and understand your code.

8. Use comments

Use comments to explain what your code does. This will help you remember what you did later on, and it will also help others who read your code. Here are two functions I wrote a while back, and without those comments, I'd be lost reading it now.

9. Version control

Version control (also referred to as source code management) is important for keeping track of changes to your code. Make sure to use some form of version control (like Git) so you can easily revert back to previous versions if needed.

In the old days, it was common to make changes locally and upload files via FTP and just overwrite previous versions. It was wild.

10. Have fun!

Coding can be challenging, but it can also be fun. Don't forget to enjoy the process and the satisfaction that comes with solving coding problems.

If you enjoyed this, take a look at some of my other posts and follow me for more coding content.

Slack Notifications with GitHub Actions

Adam K Dean — Fri, 14 Aug 2020 09:30:49 +0000

My Workflow

When I first started working with GitHub about a month ago, there were already quite a few Slack notification actions available, but I found each to either be cumbersome to use, or lacking what seemed like simple features such as status based messages.

So I created adamkdean/simple-slack-notify to try and address those issues, and this post will show you how to use it.

The main features are:

As simple or complex as you need it to be
Status based messages meaning one step handles job successes, failures, and cancellations
JavaScript strings for embedding environment variables or custom logic into notification strings
Easy to add fields based on standard Slack JSON inputs

Example usage

The simplest use would consist of relying on the webhook's defaults and simply providing some text.

- name: Simple notification
  uses: adamkdean/simple-slack-notify@master
  with:
    text: 'This is the simplest notification'

Overriding the channel is sometimes needed, such as to separate out builds, deployments, and alerts perhaps.

- name: Channel specific notification
  uses: adamkdean/simple-slack-notify@master
  with:
    channel: '#alerts'
    text: 'Something is happening and someone should probably panic'

The above works well, but what would really make someone panic is if we make the alert red, right?

You can use danger, warning, good, or a hex code such as #d90000.

- name: Panic inducing notification
  uses: adamkdean/simple-slack-notify@master
  with:
    channel: '#alerts'
    text: 'Something is happening and someone should probably panic'
    color: 'danger'

Perhaps you also want to change the username?

- name: Panic Bot notification
  uses: adamkdean/simple-slack-notify@master
  with:
    channel: '#alerts'
    username: 'Panic Bot'
    text: 'Something is happening and someone should probably panic'
    color: 'danger'

The action also supports fields, but due to the limitations of GitHub actions only passing in inputs as strings, we can't use yaml arrays. So, this is how you'd specify a field:

- name: Specifying what to panic about notification
  uses: adamkdean/simple-slack-notify@master
  with:
    channel: '#alerts'
    username: 'Panic Bot'
    text: 'Something is happening and someone should probably panic'
    color: 'danger'
    fields: |
      [{ "title": "Reason to panic", "value": "Deployed failed halfway through" }]

If there were multiple reasons to panic, you'd add more objects to the fields array:

- name: Specifying what to panic about notification
  uses: adamkdean/simple-slack-notify@master
  with:
    channel: '#alerts'
    username: 'Panic Bot'
    text: 'Something is happening and someone should probably panic'
    color: 'danger'
    fields: |
      [{ "title": "Reason to panic", "value": "Deployed failed halfway through", "short": true },
       { "title": "Timestamp", "value": "${Date.now()}", "short": true }]

Did you notice that some JavaScript snook in? Input strings are evaluated as a JavaScript strings, which means you can put environment variables into your messages, such as the GITHUB_WORKFLOW variable or GITHUB_RUN_NUMBER etc. The environment is stored within the env variable so to access environment variables in your strings, you simply use ${env.GITHUB_REPOSITORY} etc. Here's an example:

- name: Environment variable notification
  uses: adamkdean/simple-slack-notify@master
  with:
    channel: '#example'
    text: '${env.GITHUB_WORKFLOW} (${env.GITHUB_RUN_NUMBER}) has finished'
    fields: |
      [{ "title": "Repository", "value": "${env.GITHUB_REPOSITORY}", "short": true },
       { "title": "Branch", "value": "${env.BRANCH}", "short": true }]

Now, each job has a status, which can be success, failed, or cancelled. Most other notification plugins use multiple blocks with if: success() and if: failed() etc but we don't need to do that. We can simply pass in the status and set status specific text. We use if: always() so that it runs regardless of whether the job is successful or not.

- name: Build notification
  if: always()
  uses: adamkdean/simple-slack-notify@master
  with:
    channel: '#builds'
    status: ${{ job.status }}
    success_text: '${env.GITHUB_WORKFLOW} (${env.GITHUB_RUN_NUMBER}) build completed successfully'
    failure_text: '${env.GITHUB_WORKFLOW} (${env.GITHUB_RUN_NUMBER}) build failed'
    cancelled_text: '${env.GITHUB_WORKFLOW} (${env.GITHUB_RUN_NUMBER}) build was cancelled'
    fields: |
      [{ "title": "Repository", "value": "${env.GITHUB_REPOSITORY}", "short": true },
       { "title": "Branch", "value": "${env.BRANCH}", "short": true }]

Below I show how I'm using this in an actual build & deployment workflow.

Submission Category:

Maintainer Must-Haves

Yaml File or Link to Code

The following example Yaml file shows adamkdean/simple-slack-notify being integrated into a build & deploy pipeline.

name: Build & Deploy
on: push
env:
  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
jobs:
  build:
    name: Build
    runs-on: ubuntu-latest
    steps:
    - name: Build project
      uses: appleboy/ssh-action@master
      with:
        host: ${{ secrets.DEPLOY_HOST }}
        username: ${{ secrets.DEPLOY_USER }}
        key: ${{ secrets.DEPLOY_SSH_KEY }}
        port: ${{ secrets.DEPLOY_PORT }}
        script_stop: true
        script: |
          # Your build commands here
          example_build_project

    - name: Build notification
      if: always()
      uses: adamkdean/simple-slack-notify@master
      with:
        channel: '#builds'
        status: ${{ job.status }}
        success_text: 'Build (#${env.GITHUB_RUN_NUMBER}) completed successfully'
        failure_text: 'Build (#${env.GITHUB_RUN_NUMBER}) failed'
        cancelled_text: 'Build (#${env.GITHUB_RUN_NUMBER}) was cancelled'
        fields: |
          [{ "title": "Action URL", "value": "${env.GITHUB_SERVER_URL}/${env.GITHUB_REPOSITORY}/actions/runs/${env.GITHUB_RUN_ID}"}]

  deploy:
    name: Deploy
    runs-on: ubuntu-latest
    needs: build
    steps:
    - name: Deploy project
      uses: appleboy/ssh-action@master
      with:
        host: ${{ secrets.DEPLOY_HOST }}
        username: ${{ secrets.DEPLOY_USER }}
        key: ${{ secrets.DEPLOY_SSH_KEY }}
        port: ${{ secrets.DEPLOY_PORT }}
        script_stop: true
        script: |
          # Your deploy commands here
          example_deploy_project

    - name: Deploy notification
      if: always()
      uses: adamkdean/simple-slack-notify@master
      with:
        channel: '#deployments'
        status: ${{ job.status }}
        success_text: 'Deployment (#${env.GITHUB_RUN_NUMBER}) completed successfully'
        failure_text: 'Deployment (#${env.GITHUB_RUN_NUMBER}) failed'
        cancelled_text: 'Deployment (#${env.GITHUB_RUN_NUMBER}) was cancelled'
        fields: |
          [{ "title": "Host", "value": "${{ secrets.DEPLOY_HOST }}"},
           { "title": "Action URL", "value": "${env.GITHUB_SERVER_URL}/${env.GITHUB_REPOSITORY}/actions/runs/${env.GITHUB_RUN_ID}"}]

Additional Resources / Info

adamkdean / simple-slack-notify

Slack notification action that just works

Simple Slack Notify

Slack notification action that just works

No longer maintained

Please note that this project is no longer maintained, and has been archived.

Introduction

We've attempted to use a few of the Slack notification actions that are currently available, but they all seem to have limitations or be quite verbose, so we set out to create a simple yet effective action that just does what you need and nothing else. In the examples below, we'll show a few different variations of how the action could be used.

The main features are:

Status based messages meaning one step handles job successes, failures, and cancellations
JavaScript strings for embedding environment variables or custom logic into notification strings
Easy to add fields based on standard Slack JSON inputs

Be sure that you set the SLACK_WEBHOOK_URL environment variable, either in the job or in the step like this:

- uses: edge/simple-slack-notify@master

…

View on GitHub

Notes

I just wanted to point out for anyone new to GitHub Actions that while above I'm referring to the master branch (i.e. adamkdean/simple-slack-notify@master), if you're using this in production, it's always wiser to use a version. As of writing, the latest version is 1.0.4 so we would use adamkdean/simple-slack-notify@1.0.4.

Notes on Remote Working

Adam K Dean — Fri, 27 Mar 2020 15:33:29 +0000

So with the ongoing global crisis, a lot of governments and organisations are recommending that people, if they can, work from home. We're seeing memes flying around, new jokes being created, and also quite a few questions from people who haven't really worked remotely before.

I've worked entirely remotely for almost half a decade, and prior to that, partially remotely for a couple of years. I've had many conversations with people over this time, some who wonder how I ever get work done, some who think I don't have a real job, and others who are envious of what seems like an easy job — I mean, surely I can just eat and watch TV all day?

Over the last couple of weeks, I've been speaking with friends and colleagues who have worked remotely for years, about the challenges that they have faced and continue to face, and how they dealt with these challenges.

How do you get up in the morning?

I've tried all sorts of routines, from having an alarm to having no alarm, waking up early to get work done, to waking up at 10am fresh and ready for a day of ~~watching TV~~ hard work. I've even tried working through the night and sleeping during the day.

There are a few things which determine how you get up in a morning, and one of the main determining factors is how your employer works with schedules.

At Edge for example we follow an asynchronous workflow with a few scheduled calls throughout the week. This means you get your work done in your time, and you accept that your colleagues may not be able to respond to you right away. There are of course situations where you need to synchronise, such as releases and rollouts but these aren't set in stone and you tend to fall into a rhythm with your team. For example, in a morning I often call my CTO for a chitchat. With another team member, we tend to have catch ups through the week to make sure we're on track. None of this is planned.

If your employer has a set schedule, requiring you to be online from 09:00—17:00 for example, then that settles it for you. Often this is called "working from home". But for companies which don't try to fit remote work into an office shaped hole, we call this "remote-first."

So how do I get up in the morning? Currently I have Alexa set to tell me the news at sunrise before wishing me a Happy <day> although recently that's been a little disturbing, I must admit. I ignore it sometimes, and unless there is a pressing need to get up early, some days I take an extra hour or two. But you have to be careful not to oversleep, or you risk being even more tired. Some biological quirk I guess.

Dealing with extended periods of isolation

One of the things which most people will now be experiencing is the feeling of isolation from working from home. This isn't new, and has been something we remote workers have been dealing with for some time. It can be lonely, especially when you live alone. I'm told cats help.

It's important to have a support network of friends and family (or other remote workers) that you can call whenever you're feeling lonely. Sometimes it's useful just to have a chat, so you shouldn't ever feel like you're bothering a colleague by calling. Just ask "got a minute to chat?"

Socialising becomes more of an effort though, so arranging things in advance is useful. We recently started doing informal video chats (where previously we'd stick to audio) so that the team could let off a bit of steam, have a laugh, and not worry too much about talking shop and instead just talk about life. For example, we all laughed at a team member explaining how his government have stolen all the chairs from businesses. Where do the chairs go!?

Another thing which can be a struggle is the lack of background noise. Yesterday we rolled out a large release of a number of systems, so I put together a call with half a dozen of my colleagues invited. It wasn't a meeting, just an open channel to discuss the release, talk, joke, and for some, to have some background noise and chatter. Even engineers who weren't directly involved joined to chat.

How do you motivate yourself to shower?

So this was a question on Twitter, but I've been asked it in person as well. If you work remotely, specifically from home, why shower?

It depends on why you shower, bathe, clean your teeth, brush your hair, etc. Are you doing these things for the sake of others, or is it because you have self-respect? Cleanliness is important regardless of whether you're around others.

Now, just like how not everyone who works remotely wears trousers, neither does everyone who works remotely have a shower in the morning, but showering once a day is important. I generally find that if it seems like everyone smells no matter where I go, then it is in fact likely that I am the smell.

How do you get work done and not watch TV?

This applies to a number of activities, including eating the entire fridge, playing video games, reading great books (if you're a book worm like me), or even finding yourself on a 4 hour YouTube video binge, or in a Wikipedia rabbit hole. Ever more likely is the chance of being stuck in the news cycle, reading and watching the latest on this pandemic.

The most important thing here is to timebox your consumption. Read the news, play a video game, watch some TV with your breakfast, but have a schedule and know when you need to switch it off and start working. Know how long you need to work for or what tasks you need to achieve. The hardest thing is to get started, so by setting arbitrary limits, you can more easily stop doing the easy stuff and get started with the hard stuff.

If I remember correctly, a colleague of mine once announced that he played Dark Souls every morning before work so that no matter what happened, his day got better. That's wisdom.

Don't you get sick of being stuck at home?

Yes, yes we do, but a lot of remote workers still take the time to go for walks, or to go to coffee shops etc. That's obviously an issue right now in a lot of the world. In the UK, you're allowed to go for one walk, run, or cycle a day. I recommend you go for that walk. But hey, that's not always so easy.

One thing a few of my colleagues told me was that it's hard to have the motivation to exercise when your daily commute is 14 seconds and it's so easy to sit down and start working. Forcing yourself to go for a walk as much as possible is important, but if you have the space, indoor exercise equipment is a lot more accessible and you're much more likely to use it.

Before the lockdown, I'd often take a book and get a coffee from a drive through and just relax in the relative peace of the car, usually while the chaos of rush hour went on around me. Sometimes, it's just about a change of scenery.

Knowing when to stop working

This is perhaps one of the biggest challenge I have personally faced. I enjoy the work I do, and blimey, there's a lot of it to do, so often I can find myself working right into the evening, suddenly becoming conscious of my surroundings, thirsty, cold, and in the dark. It's important to know when to stop working. If you live with a partner, it can be really helpful to synchronise your working day with theirs.

In this ever more interconnected world we live in, where we're all isolated together, it can be hard to split work life from home life. Many of us engineers have to be available in the event of outages, but it's important to know the difference between an urgent issue and an asynchronous query. Learn where your line is and stick to it.

Sometimes it can be useful to have a time that you finish work, and other times it can be useful to stop working after an event or piece of work is complete. Just as you may sometimes work until 18:30 to finish a deployment, it's also okay to call it a day at 16:00 after a deployment.

Are you even working?

One thing which many remote workers experience, especially in the beginning, is the question of whether they're actually working. For the older generations, it's hard to accept that someone is home but working. Are they playing on the computer? Aren't they just for games? This is a difficult problem to solve as it's all about comprehension. The best we can do is hope that some good comes of this currently global situation and that remote work is seen by many for the possibilities it holds.

It's important to let family members know that you're working, that this is your livelihood, and that you're not available just because you're home, and that you shouldn't be interrupted as though you were relaxing.

Summary

Remote working is a wonderful thing, but it's not without its challenges. Most of those challenges stem from how we interact with others, so the solution also lies in how we interact with others. Let's support each other, have more calls, and be more understanding.

And listen to your watch, it's time to stand up!

P.S. This was my 200th blog post. Yay!

Setup your own persistent web IRC client

Adam K Dean — Mon, 16 Mar 2020 14:28:05 +0000

These days, Slack is all the rage, but in years gone by, whenever facing world-changing events, IRC has been the place to hangout. These days, you don't seem to hear much about it, and when you try and introduce people to it, the old and somewhat dated clients are hard to sell.

I've used all manner of clients over the years. I think I started with mIRC back in the early 00s, and over time I've used bouncers (these are services that keep you connected to a network which you in turn connect to) and command line clients. The last client I used was an online cloud based IRC client named irccloud. It's great, but it's a few quid a month.

Recently, an online acquaintance setup an IRC server, to help folks working remotely, to have somewhere outside of Twitter to discuss current affairs etc. World changing event in place, I needed an IRC client, a persistent one.

I had a look around and found thelounge. The name doesn't do much for me. I'm not a fan of the word the in a name, but it's okay, breath Adam. The easiest way to test something like this is to spin up an instance. So I went to Scaleway and created a small machine for €2.99/mo, grabbed the IP and setup a domain, let's say irc.example.com. (Setting up the domain simply means creating an A record that points to the IP address of your server.)

Setup

On the server itself, the only setup we need is to install docker.



# apt update
# apt install docker.io -y

... Done

Once we have docker, we'll spin up an automatic reverse proxy service along with an automatic LetsEncrypt certificate generation service. It's super simple, and to learn more, read this award winning article about it right here on dev: Automatic SSL with Let's Encrypt & Nginx.

First, the nginx proxy.



docker run \
  --detach \
  --restart always \
  --publish 80:80 \
  --publish 443:443 \
  --name nginx-proxy \
  --network service-network \
  --volume /var/run/docker.sock:/tmp/docker.sock:ro \
  --volume nginx-certs:/etc/nginx/certs \
  --volume nginx-vhost:/etc/nginx/vhost.d \
  --volume nginx-html:/usr/share/nginx/html \
  jwilder/nginx-proxy

Then the letsencrypt companion service.



docker run \
  --detach \
  --restart always \
  --name nginx-proxy-letsencrypt \
  --network service-network \
  --volume /var/run/docker.sock:/var/run/docker.sock:ro \
  --volume nginx-certs:/etc/nginx/certs \
  --volume nginx-vhost:/etc/nginx/vhost.d \
  --volume nginx-html:/usr/share/nginx/html \
  --env NGINX_PROXY_CONTAINER="nginx-proxy" \
  jrcs/letsencrypt-nginx-proxy-companion

Now we're ready to setup the IRC service. It's so easy.



docker run \
  --detach \
  --restart always \
  --name irc \
  --network service-network \
  --env VIRTUAL_HOST=irc.example.com \
  --env LETSENCRYPT_HOST=irc.example.com \
  --env LETSENCRYPT_EMAIL="webmaster@example.com" \
  --volume ~/.thelounge:/var/opt/thelounge \
  thelounge/thelounge:latest

Configuration

Well that was easy. Now we can head to https://irc.example.com and be met with the initial sign in screen. But, first, we need to configure it and create a user. To do this, on the server, take a look inside the ~/.thelounge/ directory (which maps into the container). You'll see a few files and directories in there. You can read more about the configuration in the thelounge documentation.

You can, if you like, modify the server settings. You do this by editing config.js. The only option I changed here was the channel leave message leaveMessage which defaults to "The Lounge - https://thelounge.chat", which I changed to "".

The main thing to do is to create a user. For this, we need to use the thelounge binary. Easiest way to do that is to hop into the container, so let's do that.



# docker exec -ti 93a08 bash

root@93a08cb52985:/#

Now to create a user we simply run thelounge add yournamehere.



root@93a08cb52985:/# thelounge add example

2020-03-16 13:52:12 [PROMPT] Enter password: *types dogsarebetterthancats*

2020-03-16 13:52:48 [INFO] User example created.
2020-03-16 13:52:48 [INFO] User file located at /var/opt/thelounge/users/example.json.

There may be some additional logging, some warnings, but don't worry about that. You can now login with those credentials at https://irc.example.com and proceed to add networks, setup your IRC users, and so forth. When you open the site, you'll be able to browse your IRC networks. And when you leave, you'll remain connected via this service. This is important because with IRC, if you're not in the channel, you won't get messages.

Now, that's done, welcome to the real social network(s).

Sorting objects with undefined values

Adam K Dean — Thu, 12 Mar 2020 14:46:05 +0000

So take a look at this:

const a = { id: 'a', timestamp: 1570790874500 }
const b = { id: 'b' }

You may have some data like this at some point, and you might try a comparison thinking that a defined value will always be higher and truthier than an undefined one.

You might try and sort them, expecting the undefined timestamps to fall to the bottom. But they won't.

> const c = [b, a]
> c.sort((i, j) => i.timestamp > j.timestamp)

(2) [{…}, {…}]
  0: {id: "b"}
  1: {id: "a", timestamp: 1570790874500}

Let's take a look at some comparisons which don't really help us.

> undefined > 1570790874500

false

> 1570790874500 > undefined

false

The best thing to do in this situation is to check the existence of the timestamp field within the sort predicate and only compare the values when the field exists. Depending on whether you want the objects with the undefined field first or last, you change which object you check for timestamp and return true when they are undefined.

Let's create some data.

> const list = [
    { id: 'a', timestamp: 1535090874500 },
    { id: 'b' },
    { id: 'c' },
    { id: 'd', timestamp: 1570790874500 },
    { id: 'e', timestamp: 1510790874500 }
  ]

So for undefined last, you check the first object passed in.

> list.sort((a, b) => !!a.timestamp ? a.timestamp > b.timestamp : true)

[ { id: 'e', timestamp: 1510790874500 },
  { id: 'a', timestamp: 1535090874500 },
  { id: 'd', timestamp: 1570790874500 },
  { id: 'c' },
  { id: 'b' } ]

And for undefined first, you check the second object passed in.

> list.sort((a, b) => !!b.timestamp ? a.timestamp > b.timestamp : true)

[ { id: 'b' },
  { id: 'c' },
  { id: 'e', timestamp: 1510790874500 },
  { id: 'a', timestamp: 1535090874500 },
  { id: 'd', timestamp: 1570790874500 } ]

And of course, the comparison here, a.timestamp > b.timestamp defines the sort order of the objects where the field is present.

Image Resolutions in CSS

Adam K Dean — Thu, 05 Mar 2020 14:27:56 +0000

This week's post will be a short one. The idea came about while offering suggestions for a dear friend's blog, mostly around typography and space. It's funny, I found it hard to describe typography with words, the irony, but while making suggestions and tweaking the layout, I had an idea.

One of the images on his example blog post was taken during a recent business trip, but at 5184x3888 it had an aspect ratio of 1.33:1 and didn't fit very well within the text. I felt it was too... tall!

Object fit, 60% width

So I thought, well, let's squash it a bit then, and set the height to 60% and used object-fit to ensure it didn't distort. I found this made it fit better within the text, but then I had a further idea.

At 60% height, the 5184x3888 (1.33:1) image retains a width of 5184 but then has a height of 2332. If we normalise this to make it easier to work with, we have an original image of 1000x750. Reducing the height by 40% gives us 450, resulting in an image of 1000x450 and an aspect ratio of 2.22:1.

Cinematic resolution

I looked up the standard cinema resolution, which is 1.85:1. This means that if we want to have a cinematic feel to it, our image should ideally be 1000x540 (1.85:1). Note, 540 is 72% of 750.

As you can see, at 72%, the image isn't ask skinny as the 60% version, but also, it doesn't really feel as cinematic, does it? It's time to add some bars!

Bars

The first thing I tried was to take the original image resolution (1.33:1) and place the 60% resolution (2.22:1) on top of it, but alas, that didn't really give the right result.



.image-1p33-60pc {
  display: flex;
  justify-content: center;
  flex-direction: column;
  background: black;
  margin: 1rem auto;

  // 1.33:1
  width: 52rem;
  height: 40rem;

  img {
    width: 100%;
    height: 60%;
    object-fit: cover;
  }
}

So next, I tried to slim down the bars by putting the 2.22:1 image on top of the 1.85:1 image, with a black background of course to create the bars, but this didn't look right either, it felt too skinny. This required some extra calculation.

1.33:1 is equal to 1000x750
1.85:1 is equal to 1000x540
2.22:1 is equal to 1000x450

So if we want to place a 1.33:1 image into a 1.85:1 container and have it assume the 2.22:1 resolution, we have to divide the 2.22:1 height (450) by the 1.85:1 height (540) which gives us 0.83.



.image-1p85-83pc {
  display: flex;
  justify-content: center;
  flex-direction: column;
  background: black;
  margin: 1rem auto;

  // 1.85:1
  width: 52rem;
  height: 28rem;

  img {
    width: 100%;
    height: 83%;
    object-fit: cover;
  }
}

Next I tried to put a 1.85:1 image into a 1.33:1 container:



.image-1p33-72pc {
  display: flex;
  justify-content: center;
  flex-direction: column;
  background: black;
  margin: 1rem auto;

  // 1.33:1
  width: 52rem;
  height: 40rem;

  img {
    width: 100%;
    height: 72%;
    object-fit: cover;
  }
}

At this point, my friend had already taken my typographical feedback and made some changes and I was left here with a page full of scribbles and calculations, a headache, and a little bit confused as to what I was trying to achieve.

But I also learned that flexbox is great for layouts, especially when you want to do something like add cinematic bars to an image.

Simple SCSS with 11ty

Adam K Dean — Mon, 24 Feb 2020 19:32:03 +0000

Since migrating my blog to DEV, I've been looking at rebuilding my website as a simple signpost to my various activities on the web, and with this I've been looking into static site generators, and focusing mostly on 11ty.

I've been looking through philhawksworth/eleventyone which for the most part gives a nice overview of the workings of 11ty. But one thing it didn't include was SCSS preprocessing. Instead, Phil opted for standard css preprocessing.

One thing I'm good at is searching the internet, but try as I might for almost 5 long minutes, I couldn't find a good solution out there for this. Phil created a gulp-powered SCSS preprocessor, but I didn't really want to add gulp into the mix. He wrote about how as you add in SCSS your complexity increases but I don't think this has to be the case. I carried on looking, and I found someone who was preprocessing SCSS and writing it to a file. I didn't want to generate files before I generate files either, that xzibits¹ too much recursion for my liking.

After looking at everything I'd seen so far, and deciding to do some thinking of my own, I came up with a way that you can easily add SCSS preprocessing to an 11ty project.

The Solution

As I understand it, by having a file with a .11ty.js suffix, you're able to introduce some functionality into an endpoint, the path of which you can define in the permalink field of the data() method in the class you return. You can then define the output of that endpoint by returning a string from the render() methods of that class.

Well, once you realise this, it's quite simple really. Assuming we have a directory structure like so, with the scss files within the includes directory and a JavaScript file within the css directory.

site/
site/_includes/
site/_includes/scss/
site/_includes/scss/main.scss
site/css/
site/css/style.11ty.js
site/index.md

All we need to do is use a scss-to-css module (in this case node-sass-promise), render the scss files into css, and then for good measure, clean it before we send it out.

const path = require('path')
const sass = require('node-sass-promise')
const CleanCSS = require('clean-css')

const inputFile = path.join(__dirname, '../_includes/scss/main.scss')
const outputFile = path.join(__dirname, '../css/style.css')

const comment = `/* This is an example comment */`

module.exports = class {
  data() {
    return {
      permalink: 'css/style.css',
      eleventyExcludeFromCollections: true
    }
  }

  async render() {
    const { css } = await sass.render({ file: inputFile })
    const output = new CleanCSS({}).minify(css.toString()).styles

    return `${comment}\n\n${output}`
  }
}

Now, I'm notorious for adding comments to the beginning of every source file. I'm old school, and also, I feel old. But when I work with a file that doesn't have a banner comment I don't feel right. With this in mind, I've also added the ability to place a header comment in while removing all other chaff from the scss and placing the clean, processed css afterwards.

This all results in the endpoint css/style.css being generated via the render() method and outputting something like this:

/* This is an example comment */

*{margin:0;padding:0}body{padding:1rem;}h1{font-size:3rem;}

So to summarise, you create a file site/css/style.11ty.js in which you read from an input file ../_includes/scss/main.scss, run it through some npm packages (sass, clean-css etc) and then return it as a string so that the 11ty generator can store it somewhere useful.

On the subject of banner comments, I much prefer fancier versions that a simple line of text, such as catsofinternet.org...

            _              __ _       _                       _                    
           | |            / _(_)     | |                     | |                   
   ___ __ _| |_ ___  ___ | |_ _ _ __ | |_ ___ _ __ _ __   ___| |_   ___  _ __ __ _ 
  / __/ _` | __/ __|/ _ \|  _| | '_ \| __/ _ \ '__| '_ \ / _ \ __| / _ \| '__/ _` |
 | (_| (_| | |_\__ \ (_) | | | | | | | ||  __/ |  | | | |  __/ |_ | (_) | | | (_| |
  \___\__,_|\__|___/\___/|_| |_|_| |_|\__\___|_|  |_| |_|\___|\__(_)___/|_|  \__, |
                                                                              __/ |
                                                                             |___/

Or internetdogs.com...



      .-.          /                            /   
      `-'.  .-.---/---.-.  ).--..  .-.   .-.---/--- 
     /    )/   ) /  ./.-'_/      )/   )./.-'_ /     
  _.(__. '/   ( /   (__.'/      '/   ( (__.' /      
          .    `-                     `-            
         /                                          
    .-../ .-._..-.    .      .-.  .-._..  .-. .-.   
   (   / (   )(   )  / \ .-.(    (   )  )/   )   )  
    `-'-..`-'  `-/-'/ ._)`-' `---'`-'  '/   /   (   
             -._/  /                             `-'

Or httprefs.org...


 __ __  ______  ______  ____  ____     ___  _____  _____      ___   ____    ____ 
|  |  ||      ||      ||    \|    \   /  _]|     |/ ___/     /   \ |    \  /    |
|  |  ||      ||      ||  o  )  D  ) /  [_ |   __(   \_     |     ||  D  )|   __|
|  _  ||_|  |_||_|  |_||   _/|    / |    _]|  |_  \__  |    |  O  ||    / |  |  |
|  |  |  |  |    |  |  |  |  |    \ |   [_ |   _] /  \ | __ |     ||    \ |  |_ |
|  |  |  |  |    |  |  |  |  |  .  \|     ||  |   \    ||  ||     ||  .  \|     |
|__|__|  |__|    |__|  |__|  |__|\_||_____||__|    \___||__| \___/ |__|\_||___,_|

But I mean, that's not really important is it? If you do like banner comments like this, the patorjk taag site is super useful and has been for many years.

Check it out.

¹ xzibit joke not typo

Debugging Docker Containers

Adam K Dean — Thu, 20 Feb 2020 11:43:00 +0000

Since my post about docker and SSL made the top 7 last week, I've been thinking about things which I may possibly know but take for granted, things which may be useful to other people. There are several things I could discuss like the inner workings of containers, the more practical side of using docker, how docker elbows its way into your iptables etc, but I think for this week's post I'm going to stick to something simple, though powerful and super easy: debugging docker containers.

One thing I've noticed while observing new users of docker over the years is that for all intents and purposes, containers appear to be these little black boxes connected together via all manner of tubes and pipes, with nothing to indicate what's going on other than a couple of lights and a lack of fire.

I imagine it looks something a little bit like this.

Now, the first you can do to find out what's going on is to check the logs of a container. Most folks know this, but you can do that using the docker logs command, like this.



$ docker logs website

[WEBSITE] Connecting to API...
[WEBSITE] Metal Tube connected to API
[WEBSITE] Loading blog posts via metal tube from API
[WEBSITE] Blog posts loaded
[WEBSITE] Listening on port 80

Sometimes, things take a bit longer. You kinda just have to wait for it, but that's fine, we can watch and wait using the -f or follow flag.



$ docker logs -f api

[API] Connecting to database...
[API] Connection established via glass pipe
[API] Request received from website, loading data... 
[API] Data loaded [ 4%]
[API] Data loaded [11%]
[API] Data loaded [19%]
[API] Data loaded [24%]

This is great for logs that are piped to the standard output (stdout) but you know, sometimes people containerise the strangest of things. I've seen all sorts, including containers within containers.

Sometimes an application will write to a log file, so docker logs won't really help you. Maybe you have a database that isn't really doing anything. Perhaps it's running really slowly. The container is running...



$ docker ps

CONTAINER ID    IMAGE         STATUS          NAMES
1e98dd08aee8    project/web   Up 10 minutes   website
b76207f35778    project/api   Up 10 minutes   api
35f9ba3ed4c8    project/db    Up 10 minutes   database

...but for some reason when we check the container logs, there is nothing apparent going on, and we have to delve deeper.



$ docker logs database

[DATABASE] Loading...
[DATABASE] Listening on port 21000

We need to find the hidden logs. To find them, we must enter the ~~haunted labyrinth~~ running container, which we can do easily by using the docker exec command. The way it works is that you specify a command and a container to run it in, and docker executes that command within the container for you. If we run an interactive shell then we can have a poke around the container. Psst, here's a link so you can read more about docker exec.



$ docker exec -ti database bash

root@35f9ba3ed4c8:/#

We're in. You'll notice that we used the -ti flags. According to the docker docs, -t or --tty allocates a pseudo-TTY and -i or --interactive keeps stdin open even if not attached. All we need to know is that it hooks us up to the container so we can have ourselves a little look around, and find those pesky logs.

This is a really powerful trick, and I've used it so many times over the years. Sometimes you need to look at an application that is running, or find some logs, or perhaps you need to *gulp* edit some code while an application is running without going through the standard build process.

Once you've attached to the container and spawned a shell, you're free to look around, and that's so useful. You can easily read logs.



$ docker exec -ti database bash

root@35f9ba3ed4c8:/# tail -f /var/log/database.log

[MyDB][0.0001] started spooling database contraption
[MyDB][0.0004] reading tables from an ancient .dat file
[MyDB][0.0012] unable to read data, tables.dat corrupted...

Bingo.

Sometimes, you might not be able to spawn a bash shell, especially if the container is based on a slimmed down image that's been on a juice detox, but you tend to always be able to rely on sh being available.



$ docker exec -ti database sh

# ps -p $$
  PID TTY          TIME CMD
   56 pts/0    00:00:00 sh

There are a few other useful tricks as well. You might not use these often but every now and then they'll come in handy. First up, docker top. This is like the top we all know well, except it runs within a container. Containers usually only have one process running, but that's not always the case.



$ docker top database

PID                 USER                TIME                COMMAND
2157                999                 0:35                mydb --bind_ip_all

Then you have the docker stats command. You can either run it against a specific container or view all running containers. It's useful when you want to monitor containers under load, or if you think you may have a memory leak somewhere. I've had to shorten the output a little bit, but you get the idea.



$ docker stats api

ID      NAME  CPU %   MEM USAGE / LIMIT     MEM %   NET I/O       BLOCK I/O
b76207  api   0.31%   44.52MiB / 15.64GiB   0.28%   1.75kB / 0B   1.64MB / 14.2MB

Now, when we run docker ps we see some information about a container: the image it's running, its status, when it was created etc, however containers actually have a lot more information behind the scenes such as where their filesystems are stored, what IP address they have assigned, and we can view all of this via the docker inspect command.

The actual output is much larger, so I've truncated it to give you a rough idea of what it looks like. This is perhaps 10% of the information actually available.



$ docker inspect website

[
    {
        "Id": "1e98dd08aee8b1c6e79eccf0551f4af88299a52fec567a8b27ad4711fe2ac287",
        "Created": "2020-02-19T15:25:13.582120171Z",
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false
        },
        "Image": "sha256:6d5319f761c08cb3053c676a274ce6500f31c98fb1c1fcab8ab736e39968a2fe",
        "LogPath": "/var/lib/docker/containers/1e1933e26402e3b062051c63c3fbc6d327641059213192ab6b94d1afb36484ca/1e1933e26402e3b062051c63c3fbc6d327641059213192ab6b94d1afb36484ca-json.log",
        "Name": "/website",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux"
    }
]

If you know what you're looking for, it's easier to extract that information than it is to sift through the entire document. You can do that using the --format argument. For example, getting the IP address of a container.



$ docker inspect --format '{{.NetworkSettings.IPAddress}}' api

172.17.0.3

The last trick I want to show you is less about docker, but it's still super useful. Sometimes you may have running services in containers but you're not able to connect to them. Is it a firewall issue? A networking issue? Is it an application issue? Well, whenever we have issues the best thing to do is a binary search of possible causes, that is to say, half things, and then half them again. If you can connect two processes over a network then you can rule out a network issue, for example.

We can do this using netcat, or nc. It is perhaps one of the most versatile and powerful tools that you can put in your digital toolbox. The concept is simple, on one end you create a listener, and on the other end you create a connector.

Let's create a bridge network and two containers to demonstrate: one alpine, one ubuntu. Sometimes nc is installed (alpine) and sometimes we have to install it (ubuntu) so that's what we'll do.



$ docker network create metal-tube

7507bfb685510ae7e93f808cdc2392bedf25b2912cc77abc3801d6c575795d30

--------------------------------------------------------------------

$ docker run --rm -ti --network metal-tube --name jeff alpine

jeff#

--------------------------------------------------------------------

$ docker run --rm -ti --network metal-tube --name alan ubuntu

alan# apt update -qq && apt install netcat -qqy

alan#

Okay, we're ready. So we have two containers on the same network, which we've called metal-tube. We've given them the names jeff and alan. I've changed the terminal prompts to jeff# and alan# to make this easier to read.

I've previously talked about user defined bridge networks in my post Automatic SSL with Let's Encrypt & Nginx, but the main thing to know is that they have built in DNS servers that allow you to resolve containers by their names. Don't believe me? Let's ping Alan from Jeff.



jeff# ping alan

PING alan (172.21.0.3): 56 data bytes
64 bytes from 172.21.0.3: seq=0 ttl=64 time=0.247 ms

So now, let's see if we can connect from Alan to Jeff. For this, we first setup a listener on Alan with netcat using the -l listen flag, the -p local port flag, and, so that we know what's going on, the -vv very verbose flag.



alan# nc -vvlp 9000

listening on [any] 9000 ...

Now this will hang until it receives data. Over on Jeff, we can attempt to send some data via netcat. The way we'll do this is to echo some data into the netcat process via a | pipe. We'll be using the -vv very verbose flag again.



jeff# echo "hello" | nc -vv alan 9000

alan (172.21.0.3:9000) open
sent 6, rcvd 0

Great, it seems to have worked. If it didn't, we may have have seen something like nc: alan (172.21.0.3:9000): Connection refused. Run the command again and you'll see. Once a connection to a netcat listener is closed, the listener process exits. But let's look at Alan now.



alan# nc -vvlp 9000

listening on [any] 9000 ...
connect to [172.21.0.3] from jeff.metal-tube [172.21.0.2] 39993
hello
 sent 0, rcvd 6

It worked, therefore we know that TCP connections between these two containers are working fine. Now, locally, they're going to work fine, but sometimes you may have services on different physical servers, and this is often a great way to test connectivity between machines. You don't always need to setup a netcat listener either, you can use netcat on it's own to connect to a service to check that it's up and accepting connections. That's super useful too.

There are a number of other tricks you can use to debug containers, such as overriding entry points, mapping volumes to keep an eye on the filesystem, and more, but the above are the ones I use 95% of the time. If you have some cool tricks, why not share them in the comments below?

Automatic SSL with Let's Encrypt & Nginx

Adam K Dean — Thu, 13 Feb 2020 14:34:11 +0000

See update summary at bottom of post for changelog.

Note: December 2020 saw the release of v2 of the letsencrypt-nginx-proxy-companion project. I've updated this article to reflect that but will leave the old v1 code in the footer. If you need to upgrade your existing machines in situ, please refer to the nginx-proxy/docker-letsencrypt-nginx-proxy-companion repository.

Since 2016, certificate authority Let's Encrypt have offered free SSL/TLS certificates in a bid to make encrypted communications on the web ubiquitous. If you've ever bought a certificate, you'll know they're usually quite expensive, the process for verifying them is a pain in the gluteus maximus, and then they expire while you're on holiday causing an outage.

With Let's Encrypt, all of these problems fade away, thanks to the Automated Certificate Management Environment (ACME) protocol that enables you to automate of the verification and deployment of certificates, saving you money and time. ACME is an interesting topic in it's own right, and you can read more about the various verification methods (called challenges) here, but today I'm going to show you how to easily setup a reverse proxy with automagical certificate generation, verification, and deployment.

The first thing we're going to do is create a user defined bridge network called service-network. I'm bad at naming things but this seems applicable.

docker network create --subnet 10.10.0.0/24 service-network

Next, we'll setup a reverse proxy. A reverse proxy simply accepts requests and proxies them onto another service based on routing rules such as which hostnames should go to which service containers. I imagine it looks something a little bit like this:

We'll use an image by Jason Wilder, jwilder/nginx-proxy. This is a well maintained project with a lot of documentation. We map the ports 80 and 443 into the container so that we can handle both HTTP and HTTPS connections. We have a number of volumes too, three are standard docker volumes, and one is the docker daemon UNIX socket.

You might notice that I often use long-form arguments in scripts. While we could easily save space and have this on one line, it doesn't help others who may have to maintain scripts in the future.

docker run \
  --detach \
  --restart always \
  --publish 80:80 \
  --publish 443:443 \
  --name nginx-proxy \
  --network service-network \
  --volume /var/run/docker.sock:/tmp/docker.sock:ro \
  --volume nginx-certs:/etc/nginx/certs \
  --volume nginx-vhost:/etc/nginx/vhost.d \
  --volume nginx-html:/usr/share/nginx/html \
  jwilder/nginx-proxy

Ok, so now we have our reverse proxy, next we need to setup the Let's Encrypt companion, for which we'll be using Yves Blusseau's image jrcs/letsencrypt-nginx-proxy-companion. I've been using this flawlessly now for almost a year.

We map the same volumes to this container, though no ports are published. We set the NGINX_PROXY_CONTAINER environment variable to match the name of our proxy container, and that's about it. This container will run a process whenever new service containers are detected, generating certificates and keeping them up to date.

docker run \
  --detach \
  --restart always \
  --name nginx-proxy-letsencrypt \
  --network service-network \
  --volumes-from nginx-proxy \
  --volume /var/run/docker.sock:/var/run/docker.sock:ro \
  --volume /etc/acme.sh \
  --env "DEFAULT_EMAIL=mail@yourdomain.tld" \
  jrcs/letsencrypt-nginx-proxy-companion

Finally, it's time to add a service container. We'll use the tutum/hello-world image, quite a popular one for testing. For this we're going to need a hostname with the DNS configured. We'll pretend to use hello-world.example.com and pretend that we've setup an A record pointing to the IP address of this server.

docker run \
  --detach \
  --restart always \
  --name hello-world \
  --network service-network \
  --env VIRTUAL_HOST=hello-world.example.com \
  --env LETSENCRYPT_HOST=hello-world.example.com \
  --env LETSENCRYPT_EMAIL="youremail@example.com" \
  tutum/hello-world

So let's take a look at what's going on here. We set up a proxy container which listens on ports 80 and 443. We then set up an ACME certificate container which will manage certs for us. Then we created a hello world container. We set the environment variable VIRTUAL_HOST to our hostname, and this is picked up by the proxy in order to route requests through to this container. The requests are routed through our user defined bridge service-network. We also set the environment variables LETSENCRYPT_HOST and LETSENCRYPT_EMAIL which are picked up by the ACME container and used when acquiring the certificate. FYI, the two hostnames will always have to match (VIRTUAL_HOST and LETSENCRYPT_HOST).

All we have to do is add these three variables to a container, and it'll be detected by the proxy and ACME containers and in short order, it'll work.

Now a few things to note. Port discovery — how does the proxy know which port to use? The hello-world image we use exposes a port in the Dockerfile with EXPOSE 80. This always takes precedence, so if the image has an exposed port, this will be used. But if your Dockerfile doesn't define an exposed port, or if you perhaps configure the port at runtime with an environment variable, e.g. HTTP_PORT=8000, then you can use the --expose argument to let the proxy know which port to use. If there are multiple ports available, you can specify which to use with the environment variable VIRTUAL_PORT.

Just to reiterate, Dockerfile EXPOSE takes precedence over --expose argument.

So now, when you hit https://hello-world.example.com, the DNS server returns an A record with your public IP, the browser then connects to this IP address on port 443 and in turn, the host routes that connection through to the nginx-proxy container. This terminates the SSL and proxies the request through to the hello-world container as a plain HTTP request.

Finally, there is another thing we can do. The proxy handles upgrading from HTTP to HTTPS, which is great, but sometimes you want to handle www/apex domain redirects. You can configure this in some domain registrar control panels, but then you end up with a different IP address for your hostname.

There is a better solution, and this time the image we'll use this time is actually one of mine, adamkdean/redirect.

The way it works is super simple. You simply setup another container with the VIRTUAL_HOST set to the domain you want to redirect away from and configure it's destination. Take for example we have example.com and we want it to always redirect to www.example.com because we love the www.

We have our main image like so, bound to www.example.com.

docker run \
  --detach \
  --restart always \
  --name example-website \
  --network service-network \
  --env VIRTUAL_HOST=www.example.com \
  example/website

We then create the redirect companion container, bound to example.com, with the environment variable REDIRECT_LOCATION being set to our preferred destination and REDIRECT_STATUS_CODE set as applicable.

docker run \
  --detach \
  --restart always \
  --name example-redirect \
  --network service-network \
  --env VIRTUAL_HOST=example.com \
  --env REDIRECT_LOCATION="http://www.example.com" \
  --env REDIRECT_STATUS_CODE=301 \
  adamkdean/redirect

What happens now is requests for example.com hit this redirect container, which responds with REDIRECT_STATUS_CODE and the REDIRECT_LOCATION.

We can of course use Let's Encrypt with these.

docker run \
  --detach \
  --restart always \
  --name example-website \
  --network service-network \
  --env VIRTUAL_HOST=www.example.com \
  --env LETSENCRYPT_HOST=www.example.com \
  --env LETSENCRYPT_EMAIL="youremail@example.com" \
  example/website

docker run \
  --detach \
  --restart always \
  --name example-redirect \
  --network service-network \
  --env VIRTUAL_HOST=example.com \
  --env LETSENCRYPT_HOST=example.com \
  --env LETSENCRYPT_EMAIL="youremail@example.com" \
  --env REDIRECT_LOCATION="https://www.example.com" \
  --env REDIRECT_STATUS_CODE=301 \
  adamkdean/redirect

In this case, the initial request for example.com hits the proxy, which terminates the SSL and proxies it through to the example-redirect container, which responds with a 301 to www.example.com. The next request which is now for www.example.com hits the proxy and is proxied through to the example-website container, something like this:

I hope this helps you. This setup can work for single sites or for a number of sites. I use this in situations where setting up a large container platform is a little bit overkill (kubernetes for a blog is fine, right guys?)

It's super easy to update and deploy, and having used it for over a year now, it's working great. Thanks for reading.

Update (Feb 21, 2020): as requested, here is a docker-compose.yml version.

version: "3"

services:
  web:
    image: example/website
    expose:
     - 8000
    environment:
      HTTP_PORT: 8000
      VIRTUAL_HOST: www.example.com
      LETSENCRYPT_HOST: www.example.com
      LETSENCRYPT_EMAIL: "example@example.com"
    networks:
      service_network:

  web-redirect:
    image: adamkdean/redirect
    environment:
      VIRTUAL_HOST: example.com
      LETSENCRYPT_HOST: example.com
      LETSENCRYPT_EMAIL: "example@example.com"
      REDIRECT_LOCATION: "https://www.example.com"
    networks:
      service_network:

  nginx-proxy:
    image: jwilder/nginx-proxy
    ports:
      - 80:80
      - 443:443
    container_name: nginx-proxy
    networks:
      service_network:
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
      - nginx-certs:/etc/nginx/certs
      - nginx-vhost:/etc/nginx/vhost.d
      - nginx-html:/usr/share/nginx/html

  nginx-proxy-letsencrypt:
    image: jrcs/letsencrypt-nginx-proxy-companion
    environment:
      NGINX_PROXY_CONTAINER: "nginx-proxy"
    networks:
      service_network:
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - nginx-certs:/etc/nginx/certs
      - nginx-vhost:/etc/nginx/vhost.d
      - nginx-html:/usr/share/nginx/html

networks:
  service_network:

volumes:
  nginx-certs:
  nginx-vhost:
  nginx-html:

Update (Feb 28, 2020):

I realised that I made a mistake when talking about VIRTUAL_PORT. I've updated the document but just to clarify, you tell the proxy which port to use with the EXPOSE command in a Dockerfile and the --expose argument on the Docker command line interface. If there are multiple ports exposed, you can then use the VIRTUAL_PORT environment variable to signal which exposed port to use.

Sorry about the mix up!

Update (Apr 22, 2020):

Added some missing backslashes to the secure redirect snippets. Oops!

Update (Dec 11, 2020):

The v1 code for letsencrypt-nginx-proxy-companion.

docker run \
  --detach \
  --restart always \
  --name nginx-proxy-letsencrypt \
  --network service-network \
  --volume /var/run/docker.sock:/var/run/docker.sock:ro \
  --volume nginx-certs:/etc/nginx/certs \
  --volume nginx-vhost:/etc/nginx/vhost.d \
  --volume nginx-html:/usr/share/nginx/html \
  --env NGINX_PROXY_CONTAINER="nginx-proxy" \
  jrcs/letsencrypt-nginx-proxy-companion:v1.13.1

If you enjoyed this article, you might enjoy my next one Debugging Docker Containers, where we delve into a few ways that you can work out just what is going on inside these black boxes we call containers.

A note on injecting docker.sock: the docker daemon UNIX socket (/var/run/docker.sock) gives access to docker, which in other words, is root access. Be sure you know what you're giving this too whenever you're running third party images.

SPI with MCP3008 ADC chip & TMP36 analogue temp sensor

Adam K Dean — Tue, 04 Feb 2020 12:16:20 +0000

I wrote this a while ago (on 15 Dec 2014, according to GitHub) but for some reason I never wrote about it or shared it really. It's a solution for using SPI with the MCP3008 ADC chip & a TMP36 analogue temperature sensor.

I wanted to find a way to use analogue readings (such as from temperature sensors, light dependent resistors etc) but digitally, from within a node application for example. Specifically, I wanted to cut out the Arduino step that most of these projects tend to take.

That's where the MCP3008 ADC chip comes in. The MCP3008 is an analogue–serial convert with an SPI or Serial Peripheral Interface. The code below shows how you'd make use of this with node.

Unfortunately, I can't locate a photo of the circuit but from what I recall, it was fairly straight forward. The code has this note:

You are required to have an SPI-able machine, such as a Raspberry Pi. For this test, I have two TMP36 chips wired up to feed data into CH0 and CH1 of a MCP3008 ADC chip.

I think programming is fun, but I find the intangibility of the things we create frustrating at times. For 20 years now, I've mostly authored digital creations, but I think the knowledge we have as software engineers can so easily be meshed with the physical world. That's why I'm sharing this.

"use strict";

var util = require('util'),
    fs = require('fs'),
    SPI = require('spi');

var device = '/dev/spidev0.0',
    spi;

if (!fs.existsSync(device)) {
    throw 'Error, SPI is not activated';
}

function read(channel, callback) {
    if (spi === undefined) return;

    // to select the channel, we need to compute a mode (4 bits)
    // a mode consists of a single/diff bit and three selection bits (d2, d1, d0)
    // if we want the input configuration to be single-ended, we use 1, for differential, use 0
    // for the channel, if we add that to 8 (which is 0000 1000), we should get the right value
    // ch0 = 1000, ch1 = 1001, ch2 = 1010, ch3 = 1011
    // ch4 = 1100, ch5 = 1101, ch6 = 1110, ch7 = 1111
    // now we need to pad this with 4 bits, to give us a byte:
    // ch0 = 1000 << 4 = 1000 0000
    var mode = (8 + channel) << 4;

    var tx = new Buffer([1, mode, 0]);
    var rx = new Buffer([0, 0, 0]);

    spi.transfer(tx, rx, function(dev, buffer) {
        // logic explained:

        // the buffer will hold 3 8-bit bytes (24 bits) but we only want the last 10 bits
        // this is the last byte and the last 2 bits from the second byte, we ignore the first byte

        // |   0   | |   1   | |   2   |
        // 0000 0000 0000 0000 0000 0000
        //                 ^^^ ^^^^ ^^^^

        // step 1.
        // we take the second byte and bitwise AND it with 3 (0000 0011) to extract the last two bits
        //   1010 0010 (162)  let's say the byte has some junk data and then two at the end
        // & 0000 0011 (3)    we and it with three
        // = 0000 0010 (2)    and we get the value two

        // step 2.
        // we now want to shift these bits 8 to the left to make space for the third byte
        // byte 1 = 0000 0010 (2)  <- 8 = 10 0000 0000 (512)
        // byte 2 = 0000 1111 (15)           | space |

        // step 3.
        // we can now add them together to get two bytes equaling our value:
        // 0000 0010 0000 1111 (527)

        var value = ((buffer[1] & 3) << 8) + buffer[2];
        callback(value);
    })
}

function tmp36_temp(value) {
    // approx 25 C = 750 mV (0.75 V)
    // 25 / 0.75 = 33.33 C/V
    // I think? http://www.analog.com/static/imported-files/data_sheets/TMP35_36_37.pdf

    var volts = (value * 3.3) / 1023;
    var temp = volts * 33.333;
    return temp.toFixed(2);
}

function get_volts(value) {
    var volts = (value * 3.3) / 1023;
    return volts.toFixed(2);
}

spi = new SPI.Spi(device, [], function(s) {
    s.open();
});

// read from ch0
read(0, function(value) {
    console.log('Sensor 1 is %s C (%s - %s v)', tmp36_temp(value), value, get_volts(value));
})

// read from ch1
read(1, function(value) {
    console.log('Sensor 2 is %s C (%s - %s v)', tmp36_temp(value), value, get_volts(value));
})

spi.close();

Blog migration complete

Adam K Dean — Mon, 03 Feb 2020 18:17:53 +0000

Since I first discovered dev.to as a viable blogging platform (or I as I like to think of it, MySpace for Programmers), I've been migrating my blog posts. Initially, and until today, that looked like this.

First, I take an old blog post, such as this one Melatonin Busting Glasses. I'd copy the markdown into this editor, set the title and tags, add an image, and then remove the frontmatter (a YAML header often used for metadata) and save the post.

Each day, I'd do a few, finding new header images, and each time adding a little note saying "Originally posted on x, 2013" etc, with a link to a post explaining it. A lot of the posts were outdated, but, I keep them with me as I move from blog to blog, ten years of writing.

Then, tonight, I forgot to remove the frontmatter from a blog post, and WHAM! the blog post had the date set as 2013. Not only did I find a way to set the date to the original posting date, but I could also make my life easier by creating a generic cover image for every post!

So, for the last hour or so, I've just been copy pasting markdown files with the cover image added, one by one, into the write post. I was worried I was spamming the feed but I think actually that's not the case.

For the few posts which I already posted, I can't seem to now change the date of those, so I'll leave them as they are as they have comments. But, it's fair to say now, that at 191 posts and what felt like 74 hours of copy pasting manually, I've now migrated my blog to dev.to!