DEV Community: Finnian Anderson

Colourising Video with Serverless Machine Learning

Finnian Anderson — Fri, 10 Nov 2017 08:10:19 +0000

This post was originally published on my blog

Earlier this year I saw a post on Mashable which had some amazing photos from World War Two. They really bring the era to life so I was thinking about how we could automate the process of colourising photos and save a ton of ðŸ’° - it can cost up to Â£3k/minute to colourise video professionally.

DockerCon EU was coming up and I really wanted to attend. My idea sounded like the perfect solution to my problem! Enlisting the help of my friend Oli Callaghan, we started writing code. We tried a couple of different approaches, notably writing our own machine learning framework from scratch in C++ and training a tensorflow network. When it came down to it, these solutions were always limited by one thing: time. DockerCon was in a few weeks time and our networks were taking far too long to train (we're talking weeks).

So, we were forced to go with another approach. We decided to build on the work by Richard Zhang et al and use their prebuilt model to colourise our photos. We were able to successfully deploy their Caffe model at scale with the open source serverless framework OpenFaaS. Using OpenFaaS allowed us to concentrate on the integration of the colourisation, instead of the underlying infrastructure.

Having got the colourisation working in a serverless function, we decided to extend it so that our audience at DockerCon could get involved.

Exploiting the flexibility of OpenFaaS, we wrote some extra serverless functions to allow people to tweet black and white images to @colorisebot and for it to reply back with the colourised image.

This is what our function stack looks like:

Some pretty awesome tried out our bot at the DockerCon afterparty, including the one and only Chloe Condon and Jonas Rosland.

// Detect dark theme var iframe = document.getElementById('tweet-920741455253164032-672'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=920741455253164032&theme=dark" }

Okay, so we can colourise photos. However, our main goal was always to colourise videos. How did this go? Pretty well!

Here's how it works. We simply split the video up into frames and then drop them into the OpenFaaS function stack, gather up the colourised frames and stitch them back together with ffmpeg.

Community

The OpenFaaS community has been very supportive of us, helping test, develop and inspire Oli and I to keep improving the project. Although the original idea was ours, the development and deployment has been a group effort.

In particular, we'd like to say a massive thank you to OpenFaaS's project lead, Alex Ellis for his continued help and support. He's been a great mentor, giving us invaluable advice on various different aspects varying from presentation guidance to OpenFaaS setup and configuration. He's even written a neat function which normalises the images to help with the sepia effect which has given great results.

Alex has just launched the idea of "Pods" in the OpenFaaS community. This is an idea to help manage the work on the project and decentralise leadership. Alex writes:

Pods are:

7 +/- 2 people
Mixed ability
Gravitate to areas of the project that interest them the most
Get special mentions in GitHub issues, PRs and release notes
Work collaboratively or independently
Are represented by a Pod Advocate who is willing to commit to give time to the Pod and the project

I'm really excited about this idea and look forward to joining a pod!
The OpenFaaS community is very inclusive and friendly, if you're interested in getting involved just ping an email to alex@openfaas.com and he'll get you added to the Slack group.

Sum up

Feel free to have a play with @colorisebot on twitter and let us know what you think!

The code is all open source (checkout the original post here) so take a look and have a hack.

I've got a cool little grafana dashboard linked up to the OpenFaaS Prometheus metrics so I can see what's going on in real time. Incidentally, this weekend has been a very busy period for the bot, racking up over 1130 colourised images on Twitter! I've put together a collection of the best here: https://storify.com/developius/best-of-colorisebot

Next steps

In the future we'd like to run the conversions on GPUs so that we can leverage the power of graphics processors to run our network even faster. We think we can decrease execution time by a factor of 100 (5s -> 50ms) by using a GPU.

We'd also like to try running our network on a recurrent network, which learns from the frames that came before the current one. This should help with the video conversion which appears to flicker because some of the frames are slightly incorrect.

Our slides are available here and you can watch our DockerCon talk here. Oli has also written a post about the ins and outs (pun intended) of machine learning and also goes over how we adapted our program for video.

OpenFaaS - deploying serverless functions to Docker Swarm via a CLI

Finnian Anderson — Thu, 18 May 2017 21:52:02 +0000

OpenFaaS

Functions as a Service or OpenFaaS (lead by Alex Ellis) is a really neat way of implementing serverless functions with Docker. You can build out functions in any programming language and then deploy them to your existing Docker Swarm.

In this post we'll look at a CLI for making this process even easier.

Demo

OpenFaaS highlights

The only serverless framework for both Docker Swarm and Kubernetes
Easy to use - deploy in 60 seconds, UI and CLI built-in
Supports code or binaries in any language on Windows or Linux

Getting up and running

To run functions with OpenFaaS, you'll first need to deploy an OpenFaaS cluster to a Docker Swarm. This is really simple, just follow the steps over on the deployment guide.

How it works

The diagram below gives an overview of how the OpenFaaS function packages, the Docker image, and the faas-cli deploy command fit together.

To deploy a function onto an OpenFaaS stack, you firstly must write the function itself. This is really easy and you can do it in any language which runs inside Docker (i.e. all of them). At the current time, the OpenFaaS CLI supports Python and Node, so for this example we'll use Python.

My very simple fib function is written in Python:

def fib(n):
    if n <= 1: return n
    else: return fib(n-1) + fib(n-2)

def handle(st):
    n = int(st)

    output = []
    for i in range(n):
        output.append(str(fib(i)))
    print(', '.join(output))

This code is saved in sample/fib/handler.py. It's important to create a requirements.txt file in the same directory (even if your Python code has no requirements from pip) because otherwise pip install will throw an error.

Building the function package

The way FaaS runs the function is by running this inside Python:

from function import handler
handler.handle(stdin)

When you build a function with the OpenFaaS CLI, what it's doing is packaging up your function (handler.py) and building out the parts for Docker to run the container.

The last step is to write the stack.yml file which FaaS will use to build and deploy the function.

provider:
  name: faas
  gateway: http://localhost:8080 # your faas stack url

functions:
  fib:
    lang: python
    handler: ./sample/fib
    image: developius/func_python-fib

To build the package, simply run:

$ faas-cli  build -f ./stack.yml

Here is an example package for our func_python-fib function, located inside faas-cli's build directory:

$ tree build/fib/
build/fib/
â”œâ”€â”€ Dockerfile
â”œâ”€â”€ function
â”‚Â Â  â”œâ”€â”€ handler.py         <-- called by index.py
â”‚Â Â  â””â”€â”€ requirements.txt
â”œâ”€â”€ index.py               <-- Docker runs this
â””â”€â”€ requirements.txt

1 directory, 5 files

The package includes everything to allow Docker to run our function - Dockerfile, index.py and handler.py

Deploying the function

Now that our function is built, we need to deploy it. Before we do this however, we must push our function to the hub so that all the nodes in the swarm can access it.

$ docker push developius/func_python-fib

Now for the deploy! This can be done with one command:

$ faas-cli deploy -f ./stack.yml

When you run this command, the function will be deployed to the remote Docker Swarm. How cool is that?

Testing

The final thing to do is to test your function deployed successfully. You can do this via the OpenFaaS UI or via curl.

Or, via curl:

$ curl -d "10" http://localhost:8080/function/fib
0, 1, 1, 2, 3, 5, 8, 13, 21, 34

High Availability via autoscaling

FaaS supports Docker Swarm's replication feature, which allows auto scaling of functions directly inside Docker. OpenFaaS makes use of this internally to make sure enough function replicas are available to deal with requests. It works by exporting metrics for Prometheus, which then generates alerts when thresholds are reached. The FaaS gateway will then scale up the functions when it receives these alerts.

This can be seen perfectly in the screenshot of my Grafana dashboard below.

Recap

In this post we've seen how to package and deploy a function using the FaaS framework. We've looked at how to write a function, how to build it into a function package and deploy it to a remote Docker Swarm. Pretty neat stuff.

I've built a number of other functions and hope to expand this to include more soon. I wrote a function to shorten URLs, using a service I built, which received a PR from Richard Gee to merge in an armhf Dockerfile for use on Raspberry Pies. This was a big milestone for me as I've never accepted a PR for one of my projects before!

You can find out more about the projects I've been doing with Docker on my blog at finnian.io/blog - including my DockerCon 2017 writeup! Feel free to get in touch on Twitter @developius and I'd love to hear your thoughts below.

Props to Alex Ellis for creating such an awesome tool, I look forward to participating in the development of FaaS in the future. It's worth noting that FaaS is currently considered experimental but you can expect more language templates amongst some other neat features. First... learn Go. ðŸ™ˆ