Deploying an AES-128 Encrypted HTTP Live Stream (HLS)

Ggicci — Tue, 01 Mar 2022 10:57:27 +0000

HTTP Live Streaming (HLS) is an HTTP-based adaptive bitrate streaming communications protocol developed by Apple Inc. and released in 2009.

Some key points of HLS:

Created by Apple.
Consists of a playlist/manifest file (e.g. index.m3u8) and segment video files (e.g. index01.ts).
H264 codec of video + AAC of audio.
Use HTTP, easily leveraging CDN to reach the widest audience without worring about the bandwidth and firewalls.
Adaptive streaming, enables changing the quality of the video mid-stream.
Widely supported across devicies and platforms. PC, mobile, Web, IOS, Andorid, etc.

Preface

Let's start from an sample video, which can be downloaded from vimeo. In this post, we are going to:

Convert the sample video from MP4 to HLS (a .m3u8 playlist file and .ts segment files) with AES-128 encryption method;
Serve the key files on a key server;
Serve the HLS files on a content server;
Test the deployment with VLC player and video.js.

🔄 Transcode (MP4 -> HLS)

The original sample video was in .mp4 format. While we need HLS files to serve with. We can use ffmpeg to do the transcoding.

Without AES-128, we can simply run the following command:

https://gist.github.com/lukebussey/4d27678c72580aeb660c19a6fb73e9ee

ffmpeg -i sample-video.mp4 -codec: copy -start_number 0 -hls_time 10 -hls_list_size 0 -f hls sample-noaes.m3u8

Open sample-noaes.m3u8 with VLC player. You should see it playing well.

With AES-128, we need to firstly generate an encryption key and an optional IV (initialization vector) for the AES algorithm.

openssl rand 16 > enc.key  # Key to encrypt the video
openssl rand -hex 16       # IV
# de0efc88a53c730aa764648e545e3874

And then make a key info file having the following content:

KEY URI
Path to the key file
IV (optional)

Which is used by ffmpeg. e.g. enc.keyinfo:

https://ksm.ggicci.me/e9672408-b38b-4465-ab47-519c554ae402/enc.key
enc.key
de0efc88a53c730aa764648e545e3874

The first line is the URI of the key. Which will be written to m3u8 file. And the player will consult this URI for the key to decrypt the segments files while playing.
The second line is the path to the file containg the encryption/decription key.
The third line is an optional initialization vector.

Now then, we can feed it to ffmpeg to do the transcoding again. But this time, we should get an encrypted version of the output:

ffmpeg \
  -i sample-video.mp4 \  # input file
  -hls_time 9 \ # 9s for each chunk
  -hls_key_info_file enc.keyinfo \ # encryption key
  -hls_playlist_type vod \ # video on demand mode
  -hls_segment_filename "sample-%d.ts" \ # name the segment files in pattern
  sample.m3u8 # HLS playlist (aka. HLS manifest)

Open sample.m3u8 with VLC. It won't play. Because VLC tried to retrieve the key file from this URI https://ksm.ggicci.me/e9672408-b38b-4465-ab47-519c554ae402/enc.key as noted in sample.m3u8. But it failed. Since we don't have this key server ready, yet.

🔑 Start a Key Server

Let's serve our key file enc.key on a web server and make it accessible from the URI above. Then go back to check if VLC can play this sample.m3u8.

I recommend using Caddy to start up a web server. Which should be easy to learn. Sample site configuration in the Caddyfile:

ksm.ggicci.me {
  log {
    level INFO
  }

  tls {
    alpn http/1.1
  }

  file_server {
    root /var/www/ksm.ggicci.me
  }
}

Copy file enc.key to our web server in the path specified by the URI. And open sample.m3u8 again. This time it should work, since the key URI became accessible.

🎞️ Start a Content Server

So far, our encryption key file has been served well on our key server. As long as we move our video content to a web server, too. We can consume our videos online with assistance of video players. We call it a Content Server.

Again, we use Caddy to achieve our goal:

v.ggicci.me {
  log {
    level INFO
  }

  tls {
    alpn http/1.1
  }

  file_server {
    root /var/www/v.ggicci.me
    index index.html
  }
}

Copy the playlist file and segment files to the web server. Under /sample folder in my case. And it's now accessible at https://v.ggicci.me/sample/index.m3u8. Try open it with VLC player (Media > Open Network Stream...). It should work like a charm.

📺 Use of video.js

Now that we have set up servers to host our stream. Why not host a video player, too? Thus we can consume our videos through web browsers!

Here we will try video.js - the world's most popular open source HTML5 player framework.

With a little research on its official documentation. We can compose a test HTML like this:

<!DOCTYPE html>
<head>
  <link href="https://vjs.zencdn.net/7.11.4/video-js.css" rel="stylesheet" />
</head>

<body>
  <video
    id="my-video"
    class="video-js"
    controls
    preload="auto"
    height="420"
    poster="/sample/cover.png"
    data-setup="{}"
  >
    <source src="/sample/index.m3u8" type="application/x-mpegURL" />
    <p class="vjs-no-js">
      To view this video please enable JavaScript, and consider upgrading to a
      web browser that
      <a href="https://videojs.com/html5-video-support/" target="_blank"
        >supports HTML5 video</a
      >
    </p>
  </video>

  <script src="https://vjs.zencdn.net/7.11.4/video.min.js"></script>
</body>

Since we need to read encryption key files from ksm.ggicci.me on v.ggicci.me. We will meet CORS problem. Solving it by adding corresponding headers to ksm.ggicci.me's site configurations:

ksm.ggicci.me {
  log {
    level INFO
  }

  tls {
    alpn http/1.1
  }

  file_server {
    root /var/www/ksm.ggicci.me
  }

  header {
    Vary "Origin"
    Access-Control-Allow-Origin "https://v.ggicci.me"
    Access-Control-Allow-Methods "OPTIONS, HEAD, GET"
  }
}

Visit https://v.ggicci.me/sample to see the result. Also try it with your mobile devices :)

Architecture

References

Decode HTTP Query Params into a Struct in Go

Ggicci — Tue, 24 Aug 2021 05:54:34 +0000

Many people use net/http package directly in Go to deal with HTTP requests, including reading URL parameters, HTTP headers, and the request body. It's straightforward and efficient, though. We can still get bored writing so much tedious code for just reading and parsing the URL params. Especially when we were maintaining a service with hundres of APIs.

Let's see a piece of code for dealing with HTTP requests by using net/http package:

// GET /v1/users?page=1&per_page=20&is_member=true
func ListUsers(rw http.ResponseWriter, r *http.Request) {
    page, err := strconv.ParseInt(r.FormValue("page"), 10, 64)
    if err != nil {
        // Invalid parameter: page.
        return
    }
    perPage, err := strconv.ParseInt(r.FormValue("per_page"), 10, 64)
    if err != nil {
        // Invalid parameter: per_page.
        return
    }
    isMember, err := strconv.ParseBool(r.FormValue("is_member"))
    if err != nil {
        // Invalid parameter: is_member.
        return
    }

    // Read database and return the user list.
}

It's okay if there are only a few (usually less than 3) parameters to be parsed. But what if more? Both the number of lots of local variables and the writing of statements of parsing string URL params to target types can kill us. Some "evil" guys even spread these statements all over in an API handler. 🤒

For such cases, what can we do to save a clean and tidy code base?

Use `ggicci/httpin`

httpin - 🍡 HTTP Input for Go - Decode an HTTP request into a custom struct

ggicci/httpin is an awesome package which helps you easily decoding HTTP reqeusts from:

Query string (URL parameters), e.g. ?name=john&is_member=true
Headers, e.g. Authorization: xxx
Form data, e.g. login=john&password=*****
JSON/XML body, e.g. POST {"name": "john", "is_member": true}
Path variables, e.g. /users/{username}
File uploads

You don't need to write parsing code for each parameter by yourself. You only care about the input struct and in which handler it will be used.

Let's see an example:

// 1. Define you input struct
type ListUsersInput struct {
    Page     int  `in:"form=page"`
    PerPage  int  `in:"form=per_page"`
    IsMember bool `in:"form=is_member"`
}

// 2. Bind this input struct with an HTTP handler
func init() {
    http.Handle("/users", alice.New(
        httpin.NewInput(ListUsersInput{}),
    ).ThenFunc(ListUsers))
}

// 3. Use the input in the handler, all the parsing stuff are handled by httpin
func ListUsers(rw http.ResponseWriter, r *http.Request) {
    input := r.Context().Value(httpin.Input).(*ListUsersInput)
}

You will find that with the help of httpin, you can get the following benefits:

⌛️ Saving developer time
♻️ Lower code repetition rate
📖 Higher readability
🔨 Higher maintainability

httpin is well documentated at https://ggicci.github.io/httpin/.

❤️ Have a try with it :)

DEV Community: Ggicci