Fish Cam : Lights, Camera, Swim!

So far in this series we have looked at the general background and how to get your Raspberry PI setup. Now it is time to have a little more fun with what you can do with a Web Cam, a Raspberry PI and AWS. For me, my initial fun was creating a live stream of two Goldfish, Goldie and Star, swimming around their tank. I wanted to capture live footage stream it real-time to anywhere in the world (well where ever my daughter is on my iPhone!)

Initial Web Cam Setup

Firstly plug your web cam into a USB port on your PI. Then via the desktop or remote access, run raspi-config and enable the camera.

I did this via SSH:

ssh pi

sudo raspi-config

This brings up the main menu and please select the Inference Options:

Then the Camera option and then enable it:

Your then your see a message confirming it is enabled (hopefully):

Then your need to reboot:

Once rebooted, then ssh (or whatever you use) to go back into your PI and test it out. To test I used the standard fswebcam util to capture a frame:

sudo apt install fswebcam

fswebcam -r 1280x720 image.jpg

Then open up another connection, copy the file to your local machine and open the image:

scp pi:image.jpg .
image.jpg                                                                                             100%  401KB   5.9MB/s   00:00 
open image.jpg

You should see something or even your self in the image:

If you want more details then please see the official docs

Capturing Video

To stream live video then you need to push the output from the Web Cam to a Kinesis Video Stream. To do this AWS have a C producer lib/SDK that you need to compile on your PI, configure and then run. This uses the GStreamer Framework and here is link to producer project located on GIT Hub:

https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/releases

Create Kinesis Video Stream

In order to use the producer library first you need to create a stream. You can do this in the console or via the CLI (plus many more ways). I decided to use the CLI s it is simple enough:

aws kinesisvideo create-stream --stream-name FishCam --region us-west-2

If this is successful then the output contain a ARN for the stream and we will use the ARN in the next section. Therefore please take a note of it.

Create IAM User

Next we will need create a IAM user for the library to use to authenticated against AWS. Now to be honest I would prefer to use an assumed role here and we might revisit this later in other articles in the series when using GreenGrass. For now however please create an IAM user and generate an Access Key and Secret Key, but please make sure you lock it down with something like the attach policy using the ARN (we noted down) as the resource:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kinesisvideo:DescribeStream",
                "kinesisvideo:Get*",
                "kinesisvideo:PutMedia"
            ],
            "Resource": [
                "arn:aws:kinesisvideo:us-west-2:123456789011:stream/FishCam/1634380180064"
            ]
        }
    ]
}

Install Prepresquities

The Producer Library is C, so we have to compile and build the library. This requires some prepresquities and you can installvia apt:

sudo apt install pkg-config cmake m4 git

Now to use GStreamer we need to install some more:

sudo apt-get install libssl-dev libcurl4-openssl-dev liblog4cplus-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev gstreamer1.0-plugins-base-apps gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-tools

Clone And Build

Now we can compile and build the Producer Library, this can take sometime so I recommend doing it via a stable SSH or VNC connection.

First clone the project from GIT:

git clone https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp.git
mkdir -p amazon-kinesis-video-streams-producer-sdk-cpp/build
cd amazon-kinesis-video-streams-producer-sdk-cpp/build

Then run CMake and Make. The GStreamer plugin is optional however it is essential for our streaming. So make sure you pass it as an option to CMake as per below:

cmake -DBUILD_GSTREAMER_PLUGIN=TRUE ..
make

Testing it out

So let’s test it out and see if we can get the stream running. First lets check the step-up by running the following:

cd ..
export GST_PLUGIN_PATH=`pwd`/build
export LD_LIBRARY_PATH=`pwd`/open-source/local/lib
gst-inspect-1.0 kvssink

This should result in an output like:

Factory Details:
  Rank                     primary + 10 (266)
  Long-name                KVS Sink
  Klass                    Sink/Video/Network
  Description              GStreamer AWS KVS plugin
  Author                   AWS KVS <[e-mail]>

Plugin Details:
  Name                     kvssink
  Description              GStreamer AWS KVS plugin
  Filename                 /home/pi/amazon-kinesis-video-streams-producer-sdk-cpp/build/libgstkvssink.so
  Version                  1.0
  License                  Proprietary
  Source module            kvssinkpackage
  Binary package           GStreamer
  Origin URL               http://gstreamer.net/

However if you see something like the statement below it means something failed and your need to look into the make logs to kind out why. Mostly it is the GStreamer dependancies not being present so check they installed correctly.

No such element or plugin 'kvssink'

Start The Stream

To start the streaming you need to take the IAM users credentials, the Stream name plus AWS Region name and replace the place holders in this command and then run it on your PI:

$ gst-launch-1.0 autovideosrc device=/dev/video0 ! videoconvert ! video/x-raw,format=I420,width=640,height=480,framerate=30/1 ! 
                x264enc bframes=0 key-int-max=45 bitrate=500 ! video/x-h264,stream-format=avc,alignment=au,profile=baseline ! 
                kvssink stream-name=FishCam storage-size=512 access-key="YOUR_ACCESS_KEY" secret-key="YOUR_SECRET_ACCESS_KEY" aws-region="YOUR_AWS_REGION"

This should result in log that looks like below that tell you bytes streamed:

2021-10-15 22:50:04 [1237316704] DEBUG - fragmentAckReceivedHandler invoked
2021-10-15 22:50:04 [1237316704] DEBUG - postReadCallback(): Wrote 1024 bytes to Kinesis Video. Upload stream handle: 1
2021-10-15 22:50:04 [1237316704] DEBUG - postWriteCallback(): Curl post body write function for stream with handle: PICam and upload handle: 1 returned: {"EventType":"PERSISTED","FragmentTimecode":1634334601102,"FragmentNumber":"91343852333348846620194705670290539091726766250"}

2021-10-15 22:50:04 [1237316704] DEBUG - fragmentAckReceivedHandler invoked
2021-10-15 22:50:04 [1237316704] DEBUG - postReadCallback(): Wrote 303 bytes to Kinesis Video. Upload stream handle: 1
2021-10-15 22:50:04 [1237316704] DEBUG - postReadCallback(): Wrote 296 bytes to Kinesis Video. Upload stream handle: 1
2021-10-15 22:50:04 [1237316704] DEBUG - postReadCallback(): Wrote 696 bytes to Kinesis Video. Upload stream handle: 1
2021-10-15 22:50:04 [1237316704] DEBUG - postReadCallback(): Wrote 427 bytes to Kinesis Video. Upload stream handle: 1
2021-10-15 22:50:04 [1237316704] DEBUG - postReadCallback(): Wrote 871 bytes to Kinesis Video. Upload stream handle: 1
2021-10-15 22:50:04 [1237316704] DEBUG - postReadCallback(): Wrote 551 bytes to Kinesis Video. Upload stream handle: 1

If you see something like below it means your not replaced the placeholders correctly (watch the quoting):

2021-10-15 22:03:50 [1389352032] DEBUG - describeStreamCurlHandler(): DescribeStream API response: 
2021-10-15 22:03:50 [1389352032] INFO - describeStreamResultEvent(): Describe stream result event.
2021-10-15 22:03:50 [1389352032] WARN - curlCompleteSync(): curl perform failed for url https://kinesisvideo.YOUR_AWS_REGION.amazonaws.com/describeStream with result Couldn't resolve host name: Could not resolve host: kinesisvideo.YOUR_AWS_REGION.amazonaws.com
2021-10-15 22:03:50 [1380959328] WARN - curlCompleteSync(): HTTP Error 0 : Response: (null)
Request URL: https://kinesisvideo.YOUR_AWS_REGION.amazonaws.com/describeStream
Request Headers:
    Authorization: AWS4-HMAC-SHA256 Credential=YOUR_ACCESS_KEY/20211015/YOUR_AWS_REGION/kinesisvideo/aws4_request, SignedHeaders=host;user-agent;x-amz-date, Signature=40201726f3815617c82ecdc62d5b9476936d19f81ce6cdda43fb3a05605bfc6c
    content-length: 37
    content-type: application/json
    host: kinesisvideo.YOUR_AWS_REGION.amazonaws.com
    user-agent: AWS
2021-10-15 22:03:50 [1380959328] DEBUG - describeStreamCurlHandler(): DescribeStream API response: 
2021-10-15 22:03:50 [1380959328] INFO - describeStreamResultEvent(): Describe stream result event.
2021-10-15 22:03:50 [1380959328] WARN - curlCompleteSync(): curl perform failed for url https://kinesisvideo.YOUR_AWS_REGION.amazonaws.com/describeStream with result Couldn't resolve host name: Could not resolve host: kinesisvideo.YOUR_AWS_REGION.amazonaws.com
2021-10-15 22:03:51 [1389352032] WARN - curlCompleteSync(): HTTP Error 0 : Response: (null)
Request URL: https://kinesisvideo.YOUR_AWS_REGION.amazonaws.com/describeStream
Request Headers:
    Authorization: AWS4-HMAC-SHA256 Credential=YOUR_ACCESS_KEY/20211015/YOUR_AWS_REGION/kinesisvideo/aws4_request, SignedHeaders=host;user-agent;x-amz-date, Signature=40201726f3815617c82ecdc62d5b9476936d19f81ce6cdda43fb3a05605bfc6c
    content-length: 37
    content-type: application/json
    host: kinesisvideo.YOUR_AWS_REGION.amazonaws.com
    user-agent: AWS

Handy Bash Script

To make things easier I then created the following script to start up the streaming when I needed. This saves me having to remember all the steps:

cd ~/amazon-kinesis-video-streams-producer-sdk-cpp
export GST_PLUGIN_PATH=`pwd`/build
export LD_LIBRARY_PATH=`pwd`/open-source/local/lib

gst-launch-1.0 autovideosrc device=/dev/video0 ! videoconvert ! video/x-raw,format=I420,width=640,height=480,framerate=30/1 ! x264enc bframes=0 key-int-max=45 bitrate=500 ! video/x-h264,stream-format=avc,alignment=au,profile=baseline !  kvssink stream-name="PICam" storage-size=512 access-key="XXXXX" secret-key="YYYYY" aws-region=us-west-2

More details using the library can be found here :

https://docs.aws.amazon.com/rekognition/latest/dg/streaming-using-gstreamer-plugin.html

Hopefully you got it working and let's leave it running whilst we work on how to view the stream.

Viewing the Live Streaming Video

That all good! but hay we still cannot see Goldie and Star swimming around that lovely tank we got them. Also let's remember the key requirement is seeing them doing their thing from wherever my daughter is on holiday using our tablets or phones. Therefore I decided to use HLS as most modern devices support it and you can uses it within a HTML5 webpage. Luckily Kinesis Video Streams supports HLS and they have a ready made HTML5 page, see here.

To make this work we need to reuse IAM user and hardcode the same credentials into the page. This is not good, so I did a look at Cognito and I did setup a role etc. However it just returned a 403 all the time and it took me a few hours of digging into to find out why. Here the shocker! Authentication via Cognito does not support all aws services see Access Policies in this doc. Plus I have no idea why Kinesis Video Streams is not on that list, especially given Kinesis Data Streams are. So it means we are stuck with the IAM user and hardcoded credentials :-(.

HTML5 client

This resulted in a HTML 5 like the following:

<script src="https://cdn.jsdelivr.net/npm/hls.js@latest"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/aws-sdk/2.278.1/aws-sdk.min.js"></script>
<link rel="stylesheet" href="https://vjs.zencdn.net/6.6.3/video-js.css">
<script src="https://vjs.zencdn.net/6.6.3/video.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/videojs-contrib-hls/5.14.1/videojs-contrib-hls.js"></script>

<video id="video"  controls autoplay></video>
<script>

  var protocol = 'HLS';
  var streamName = 'FishCam';

  // Step 1: Configure SDK Clients
  var options = {
    accessKeyId: 'AAAAAAAAA',
    secretAccessKey: 'BBBBBBBBBB',
    region: 'us-west-2'
  }
  var kinesisVideo = new AWS.KinesisVideo(options);
  var kinesisVideoArchivedContent = new AWS.KinesisVideoArchivedMedia(options);

  // Step 2: Get a data endpoint for the stream
  console.log('Fetching data endpoint');
  console.log(AWS.config.credentials);

  kinesisVideo.getDataEndpoint({
    StreamName: streamName,
    APIName: "GET_HLS_STREAMING_SESSION_URL"
  }, 
  function (err, response) {

    if (err) { return console.error(err); }
    console.log('Data endpoint: ' + response.DataEndpoint);
    kinesisVideoArchivedContent.endpoint = new AWS.Endpoint(response.DataEndpoint);

    // Step 3: Get a Streaming Session URL
    var consoleInfo = 'Fetching ' + protocol + ' Streaming Session URL';
    console.log(consoleInfo);

    kinesisVideoArchivedContent.getHLSStreamingSessionURL({
      StreamName: streamName,
      PlaybackMode: 'LIVE',
      DiscontinuityMode: 'ALWAYS',
      MaxMediaPlaylistFragmentResults: '1000',
      Expires: '30000',
      HLSFragmentSelector: {
        FragmentSelectorType: 'SERVER_TIMESTAMP'
      }
    }, function (err, response) {

        if (err) { return console.error(err); }
        console.log('HLS Streaming Session URL: ' + response.HLSStreamingSessionURL);
        var video = document.getElementById('video');
        if (Hls.isSupported()) {
          var hls = new Hls();
          hls.loadSource(response.HLSStreamingSessionURL);
          hls.attachMedia(video);
          hls.on(Hls.Events.MANIFEST_PARSED, function () {

            video.play(function (err) {
              if (err) { return console.error(err); }
            })
          });
        }
        // hls.js is not supported on platforms that do not have Media Source Extensions (MSE) enabled.
        // When the browser has built-in HLS support (check using `canPlayType`), we can provide an HLS manifest (i.e. .m3u8 URL) directly to the video element through the `src` property.
        // This is using the built-in support of the plain video element, without using hls.js.
        // Note: it would be more normal to wait on the 'canplay' event below however on Safari (where you are most likely to find built-in HLS support) the video.src URL must be on the user-driven
        // white-list before a 'canplay' event will be emitted; the last video event that can be reliably listened-for when the URL is not on the white-list is 'loadedmetadata'.
        else if (video.canPlayType('application/vnd.apple.mpegurl')) {
          video.src = response.HLSStreamingSessionURL;
          video.addEventListener('loadedmetadata', function () {
            video.play();
          });
        }
        else if(playerName === 'VideoJS') {
            var playerElement = $('#videojs');
            playerElement.show();
            var player = videojs('videojs');
            console.log('Created VideoJS Player');
            player.src({
              src: response.HLSStreamingSessionURL,
              type: 'application/x-mpegURL'
            });
            console.log('Set player source');
            player.play();
            console.log('Starting playback');
        }
    })
  })
</script>

Hosting the Webpage

We need somewhere to host our HTML5 client and to be honest I did not want to spend too much money in doing so. Therefore I created a new S3 bucket with relatively meaningful name and then enabled static hosting:


I then named the the page index.html and uploaded it to s3 so that is was served as root page of the web address.

Lastly I took the static hosting URL and placed it my browser and on my phone and tested it out. It can take a few seconds to load but if successfull you should see your fish swimming around or whatever your subject is

Conclusion

This was great fun and works surpassingly well. My daughter was able to view her fish when she has been on holiday in Cornwall and Scotland. I do also have a Philips Hue light bulb in the room so if it is too dark then I can turn the light on as well. I hope you enjoyed this and please get touch if you have any feedback.

Come back soon for more fishy fun!