DEV Community: Rohan Sawant

Installing Docker and Docker-Compose on the Jetson Nano 4GB/2GB in 2 simple steps!

Rohan Sawant — Thu, 22 Oct 2020 06:56:41 +0000

You know what Docker is. You know what a Jetson Nano is. Let me make sure they hit it off together. 😎🤝🏽

This is probably one of the quickest and easiest ways to get Docker and Docker Compose running on the Jetson Nano
(Tested with Jetson Nano 4GB Development Kit)

After searching the Inter-Webs for hours and having several things not work for me, I decided something needed to be done about it.

Steps

How to install Docker and Docker Compose on the Jetson Nano, the right way!

1. Download install script

I have already created a very simple shell file which installs the prerequisites needed to run Docker as well as Docker-Compose itself. Simply download it.

wget -O install-dc.sh https://gist.githubusercontent.com/CT83/6dbe0d9df3fd3ba4d57fd3a5347e5105/raw/4975eb302be84cf1755378523343004451c7260d/install-dc.sh

2. Run the script

The script might need you to prompt "yes" by pressing "Y" a couple of times, do play along with it.

sudo sh ./install-dc.sh

3. Test Docker installation

docker run hello-world

Boom! 🔥 It's done!

Important Notice about Docker on the Jetson Nano

Jetson Nanos use the ARM architecture, and as a result, won't be compatible with all containers out of the box. Images will need to be built from an ARM base image. But, most of these images can easily be found on Docker Hub

Appendix

Some things you might find interesting.

Shell Script

I would encourage you to take a look at the install-dc.sh script.

#!/bin/bash
sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt-get install curl python3-pip libffi-dev python-openssl libssl-dev zlib1g-dev gcc g++ make -y
curl -sSL https://get.docker.com/ | sh
sudo pip3 install docker-compose
sudo docker-compose --version

The story of how I (kinda') fixed AWS Kinesis Producers, helped a student, and saved humanity hundreds of man-hours!

Rohan Sawant — Mon, 05 Oct 2020 09:04:14 +0000

After investing about 20 hours, I finally managed to get AWS Kinesis Video Streaming Producers to work on my Raspberry Pi, despite of the confusing instructions, Github repos which were widely different from the official docs & too many alternative ways to run things - this is the story of how I did that.

👦🏽 How did it start?

It all started with an innocent looking WhatsApp message, from a student who was trying to get AWS Kinesis Video Streams to work for a college project. He said he saw my commits to the AWS Kinesis repo and messaged me asking for help to get it up and running.

I arrogantly thought, "Ah, newbies can't even follow the docs properly." Back then, little did I know how deep this rabbit hole went, what followed were 3 entire days of research, googling and total mind-funkery, all just to get a simple "send data to cloud script" working.

🖤 You stare into the abyss...

What we wanted to achieve was pretty simple, we just wanted a way to stream video frames to AWS Kinesis from /dev/video0 on Linux. Instinctively I looked up the AWS docs and here is where the trouble started

The Problems.

GStreamer?!

Turns out there are mainly 2 ways to get the AWS Kinesis Producer libraries to working. One is using the compiled C++ code from a Github repo and other is by compiling a GStreamer plugin, but that's not all. The GStreamer command changes slightly depending on what platform you are on, this means there are in all 2 x 3 (Mac/Win/Linux) ways to get this up and running that you are greeted with right when you first load up their hello-world example. One could patiently try to read through all of this but the docs even lack a Prerequisites Section, this makes getting started even more difficult.

Suggested Fixes - Adding a Prerequisites Section will easily fix this.

Docker to the rescue?...No

The next way to get the Kinesis Video Producer up and running is by using Docker, one would imagine that this would be lesser of the two evils and it would work similarly across all platforms but you would still be wrong. The docs briefly mention that there is some kind of Docker way to run the Producer but towards the end they leave you off with - Start streaming from the camera using the gst-launch-1.0 command that is appropriate for your device.

The commands GStreamer commands they wanted me to run here kept on complaining that the GStreamer Plugin to support Kinesis-stuff was not installed.

This essentially meant even Amazon's own Docker Images are not working.

Suggested Fixes - Installing things using Docker is a separate way to get the producer up, it should have come with **complete instructions on what we are supposed to do, one should not have to run back and forth between several pages.

Inequality among platforms

Now, on Windows and Mac you cannot get the web camera streaming working with Docker, but on Linux you can. There is no table which tells you which way allows you to do what, what are the advantages and what are the disadvantages of that way and so on. This makes it all really a hit-and-miss approach while deciding what to do.

Suggested Fixes - All the 2 different ways (Docker and GStreamer) and ways for different platforms are all sprinkled in with each other a better way would be to create a separate and complete tutorial for each platform, and then add a comparison sheet to summarize everything nicely.

This was just the start, these were all of the glaring issues I could see right off the bat when I took a look at AWS's documentation, Azure on the other hand in one of their Azure IOT tutorials runs us through each scenario completely from start to finish. They also have several videos to show us how to get things up.

🌪 Smoke & Mirrors

Back to the problem we were trying to solve - "how to get a AWS Kinesis Producer running on Linux"

Initially I just assumed that the docs were difficult to understand but correct, I just needed to follow them precisely. But, boy! was I wrong. I soon came to the realization that the docs were flat out lying. Yes, let me explain.

The Lying Docs #1

In Using the C++ Producer SDK on Raspberry Pi the docs tell you to run install-script, but I have look around and if one were to follow the docs to the point, and I have you won't be able to find any such file. The closest thing you can find to something like this in a previous version of the GitHub repo in one of the older commits.

Yes. You see the problem? Not only that, this is not where this ends.

The only reason I was able to find this out was because the docs referred to using git clone --recursive, I was skeptical what that meant but, I thought it was because the repo, once used to have embedded repos. Now the docs still say it does but in reality it does not.

The Lying Docs #2

In Step 3: Run and Verify the Code the docs talk about cd-ing into amazon-kinesis-video-streams-producer-c but, that is impossible because one of the commits removed this embedded github repository (Read More)

The only way to get this to work is by checking out the older code from one of the previous commits - This is what I do here

I will cease to ramble more on about this but the temporary solution for this is to checkout the repo at one of the earlier stages that fixes all of the out-of-sync issues of the docs with the repos.

Suggested Fixes - Checkout commit 824de900c2d4b42a403319ace5de9d68ec88b171

🔥 My Solution

I knew something had to be done about this. I could not let everyone waste just as much time as I had wasted, so I wrote a very short technical blog about this. How to stream video using AWS Kinesis Video Streaming - Docker on Linux? - It gives you just enough information to get this to work, and nothing else.

But, hopefully some one some day will find great value in it.

Finally, how did I get the streaming to work, how can you?

docker-compose up --build

I wrote this blog summarizing my approach - How to stream video using AWS Kinesis Video Streaming - Docker on Linux?

That's all you need to do. 😎

This will send the video stream from /dev/video0 straight to your AWS Kinesis Video Stream

Now, the approach I propose in the blog may not be the best one, it might not be the most efficient one, but that's not the goal of it. The official docs can go in more details on how to speed things up, but the initial setup I believe should be way easier than it is right now.

👩🏽‍🚀 Conclusion

What did we learn here? And why was I extremely mad when I first encountered this issue?

AWS Kinesis is actually great

Personally, I think this enormous level of complexity to get things up and running with AWS Kinesis makes the entire pipeline look very difficult to use and because I have used AWS Kinesis before I know for a fact that this is not the case, AWS Kinesis is pretty cool and paired up with SageMaker it is capable of doing really impressive things. (expensive, yes but impressive things)

The Space-Time Continuum

I am also confident that Rohan from 2 years ago would have certainly given up on trying to get this working, and because all the ways to get an AWS Kinesis Producer up and running are considerably difficult Amazon is actually losing out on a large chunk of new users which would have otherwise used the service but now fail to do so.

It is also really surprising to note that the ways I tried to get the Producer up and running are also the ways most web developers would try to get things running and all of those failed, I had to literally go back in time to get the Producer working. When customers have to bend the space-time continuum to get your service working it is rarely a good thing.

Currently no way to get it up

The issue disproportionately ended up affecting all of the new users who were trying to use AWS Kinesis, virtually without being very very smart, and without using Github commit history I cannot think of a way someone would be able to get the Producer running, this means there will be very few people are able to get it up.

So, yeah this was it. That was how I was able to get AWS Streaming to work.

AMZ, Slide in to the DMs

Amazon, you are watching and want me to build docker images to add support for RTSP streaming too don't be shy. My DMs are open for you. If I was too hard on you that's only because you made me waste a week's worth of time on you, nothing against you, man. You are okay. 🤝🏽

How to stream video using AWS Kinesis Video Streaming - Docker on Linux?

Rohan Sawant — Sat, 03 Oct 2020 02:54:25 +0000

This article talks about how you can stream video using AWS Kinesis Video from one machine to AWS. We will use the AWS Kinesis Producer library for this and then view the livestream on AWS.

📚 Prerequisites

This article assumes that you already know

Basics of AWS, AWS Kinesis, Python and Linux
Cursory understanding of Docker and Docker Compose

Let's first get some of the basics out of the way.

AWS Kinesis - essentially a tunnel which lets you send things
Kinesis Producer - the thingy that creates stuff to send through the tunnel

This means to get video streaming to work we have to first setup our Producer make sure it is sending things to Kinesis, that's what we will be doing here.

Note - Here, the video source mounted at /dev/video0 will be streamed to AWS, this can be updated in the docker-compose.yml file. Before starting with the next steps make sure /dev/video0 is the desired video source and that it's working correctly.

👷🏽‍♂️ Steps

1. Create a Video Stream Using the AWS CLI

Install & setup the AWS Command Line Interface
Run the following Create-Stream command in the AWS CLI:



aws kinesisvideo create-stream --stream-name "Producer-Stream" --data-retention-in-hours "1"

Send Video to the Kinesis Stream

Clone my Github Repo git clone https://github.com/CT83/AWS-Kinesis-Video-Streaming.git
Create .env with client id, & secret for your IAM user Refer the .example.env - Read More
docker-compose up --build

3. View the Video Stream

Login to AWS Console
Navigate to Kinesis Video Streams, Video Streams and Producer-Stream
Check different regions if you can't see anything in your current one. Mine in us-west-2

🐛 Appendix

Now, let's get a little bit of a background on what this repository is actually doing, and it is not doing much.

Dockerfile

First, let's take a look at the Dockerfile, it's simply starting from a base debian image, installing build tools and Gstreamer and building the C++ producer. In the last step we are simply running this producer when the container starts.



FROM debian:buster

# Install Requirements

RUN apt-get update -y && apt-get install git build-essential cmake g++ wget curl -y

RUN apt-get install libssl-dev libcurl4-openssl-dev liblog4cplus-1.1-9 liblog4cplus-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev gstreamer1.0-plugins-base-apps gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-tools -y

# Build Producer

RUN git clone https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp.git

WORKDIR /amazon-kinesis-video-streams-producer-sdk-cpp

RUN git checkout 824de900c2d4b42a403319ace5de9d68ec88b171

WORKDIR /amazon-kinesis-video-streams-producer-sdk-cpp/kinesis-video-native-build

RUN ./min-install-script

CMD ["/amazon-kinesis-video-streams-producer-sdk-cpp/kinesis-video-native-build/kinesis_video_gstreamer_sample_app", "Producer-Stream","/dev/video0"]

References

https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp

docker-compose.yml

The docker-compose file simply sets a container name, mounts the /dev/video0 inside the container and sends the environment variables to the container



version: "3.3"

services:

  producer:

    container_name: aws-kvs-producer

    build: .

    devices:

      - /dev/video0:/dev/video0

    env_file:

      - .env

References

https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/gs-createstream.html

How to setup a local Docker registry with Kubernetes on Windows (includes 1 hidden step that official docs doesn't)?

Rohan Sawant — Mon, 10 Aug 2020 03:57:02 +0000

This post covers how to -

Setup a local Docker registry with Kubernetes on Windows
How to push/pull images to/from the registry

Prerequisites

You have Kubernetes installed
You have Docker installed

Architecture

1A - Steps - Setting up the registry

Start the cluster and allow insecure registries minikube start --insecure-registry "10.0.0.0/24"
Tell minikube to start a registry inside a pod in the Kubernetes cluster minikube addons enable registry
Get the name of the registry pod, in my case it is, (the official docs didn't explain this) registry-s4h7n kubectl get pods --namespace kube-system
Forward all traffic to the registry, run the two commands in separate terminals kubectl port-forward --namespace kube-system registry-s4h7n 5000:5000
Continued - docker run --rm -it --network=host alpine ash -c "apk add socat && socat TCP-LISTEN:5000,reuseaddr,fork TCP:host.docker.internal:5000"
Done. Now, visit. http://localhost:5000/v2/_catalog

1B - Steps - Testing the Setup

git clone https://github.com/CT83/ping-google.git
docker build -t ping-google .
Make sure it works docker run ping-google
Tag it and prepare for push to our newly created registry docker tag ping-google localhost:5000/ping-google
Push it docker push localhost:5000/ping-google
Pull it back down on Kubernetes kubectl create deployment ping-google --image=localhost:5000/ping-google
Ensure it's running kubectl get pods



NAME                           READY   STATUS    RESTARTS   AGE
ping-google-7666ff8964-w5h44   1/1     Running   0          17s

kubectl logs ping-google-7666ff8964-w5h44



PING ns1.google.com (216.239.34.10) 56(84) bytes of data.

64 bytes from ns2.google.com (216.239.34.10): icmp_seq=1 ttl=97 time=73.0 ms

Conclusion

That's basically it, this should greatly simplify your development workflow.

I made an AI Chrome Extension to fight Fake News! - Bunyip

Rohan Sawant — Sun, 10 May 2020 06:20:49 +0000

Bunyip is a Chrome Extension, which allows us to detect AI-generated text, it helps users detect fake news articles which might be generated automatically and not by a real human!

You can install the extension from Chrome Store! - Bunyip - Detect all the Glitter in the Wild

As with several of my older projects the aim with Bunyip, initially was to monetize it.

CT83 / Bunyip

Bunyip is a Chrome Extension, which allows us to detect AI generated text, it helps users detect fake news articles which might be generated automatically and not by a real human!

You can install the extension from Chrome Store! - Bunyip - Detect all the Glitter in the Wild

As with several of my older projects the aim with Bunyip, initially was to monetize it.

Buut, I could not find a market for this, and for now I decided against making it a paid extension on Chrome Store so I Open Sourced it instead.

Working

The selected text is sent to a Serverless Function for classification
The response contains the words with the likelihood of each word being generated by an AI.
The extension then visualizes these words, using different words to correspond to different probabilities.

This project builds on the Giant Language Model Test Room, which enables a…

View on GitHub

Buut, I could not find a market for this, and for now I decided against making it a paid extension on Chrome Store so I Open Sourced it instead.

Working

The selected text is sent to a Serverless Function for classification
The response contains the words with the likelihood of each word being generated by an AI.
The extension then visualizes these words, using different words to correspond to different probabilities.

This project builds on the Giant Language Model Test Room, which enables a forensic analysis of how likely an automatic system has generated a text. - Live Demo

Components

There are about 3 components that makeup Bunyip.

1) Bunyip - Chrome Extension

This simply sends the selected text to the GCP Cloud Function Proxy which then forwards it to GLTR.

2) Serverless Proxy running on Google Cloud Platform

The Algorithmia REST call contains an API Key that is required to make a request to it, so and the only way I could think of to keep it not hardcoded in the Chrome Extension was to use a proxy hence the workaround.

3) Modified Version of GLTR - A tool to detect automatically generated text

This is deployed on Alogrithmia's Serverless Environment and is interacted with - through a REST API, the GCP Function makes a call to this internally and returns the response to the Chrome Extension.

How did I go about making it?

Step 1 - Analyzing the problem statement at hand

To create a Chrome Extension to detect if the selected text was generated by an AI

I created a list of all the things I needed to learn, Chrome Extensions, Serverless Deployment, GCP Cloud Functions, the GLTR Integration.

Step 2 - Getting GLTR up and running locally

This was way easier than I thought it would be, everything worked in a jiffy - installed requirements and started the flask server, used PostMan to test everything locally.

Step 3 - Creating the Chrome Extension

This was the easiest but the most time-consuming part of the process, the UI took longer than I expected to make but the results were impressive!

Step 4 - Deploying the Flask App onto a Serverless Cloud Platform

This was super tricky and I touch more on this in the Challenges section.

Step 5 - Publishing the Extension to Chrome Webstore

Documentation on how to do this was pretty clear, so I was able to power through this.

Challenges

Deployment is always a doozy

Yes, one of the most understated parts of building Bunyip was the overwhelming amounts of extra work that needed to be done to make run in the wild and not just on my laptop. Deploying the entire setup somewhere cheap and scalable was the major challenge.

1. Models cannot be directly deployed onto Serverless functions

I had assumed that I would just be able to directly deploy my entire app to some Serverless Environment and everything would be a breeze, well...

Turns out the PyTorch package which is needed to run the model was over 500 MB big, which meant it was too big for AWS Lambda Functions and GCP Cloud Functions to handle.

Then, I thought about deploying the Flask App to AWS EC2 instances instead.

But, I noticed how I was going to need at least a t2.large instance and it was more than what I wanted to spend on a side project.

Then I stumbled across Algorithmia, which allows you to wrap your Python code in a REST complete with Authentication, Hosting, Logging, client-side libraries for all major languages and so much more!

With a little bit of refactoring and after a few tries I was able to get by App on it. The next step was simply making POST calls to it from my Chrome Extension.

2. Accessing the Algorithmia API without hardcoding the API Keys in the Chrome Extension

Algorithmia requires you to include an API Key every time you make a request to it, traditionally this would mean, the Bunyip - Chrome Extension would have to do do this. But, I didn't think it was wise to just expose my credentials to all of the internets!

The way I went around this was I created a simple proxy function and deployed it as a GCP Cloud Function, the proxy made authenticated calls on behalf of the browser and returned the appropriate responses, this meant, my API Keys were totally private and secure.

Motivation

Andrej Karpathy tweeted this, and I thought, "Yes! That something which I could actually do!".

So I did!

References

This project builds on the strong foundation provided by the Giant Language Model Test Room built by Hendrik Strobelt, Sebastian Gehrmann and Alexander M. Rush. GLTR, enables a forensic analysis of how likely an automatic system has generated a text.
You can find the GLTR instance deployed as an API on Algorithmia - bunyip-gpt-detector
You can find OpenAI's original GPT Detector, deployed as an API here - gpt-detector

Credits

Bunyip would never have become possible without the works of @hen_str, @S_Gehrmann, and @harvardnlp on the Giant Language Model Test Room, they even went out of their way to aid me on Twitter when I hit a few roadblocks!

Go follow them, now!

What API can I build for you?

Rohan Sawant — Sun, 03 May 2020 12:51:42 +0000

I am trying to think of stating my next Dev Adventure and I am looking for ideas, do you have any ideas for an API?

Are there any problems that I can help you with an API? 🤔

Would love to get your 50 cents on this!

I built a Voice Assisted Video Conferencing Platform with React and Flask! - Twilio Hackathon Submission - Hellin' Worki

Rohan Sawant — Thu, 30 Apr 2020 14:48:16 +0000

🐣 Hellin' Worki - Now, Joe's just a shout away

Hellin' Worki is a video conferencing platform which seamlessly dials your coworkers when you call out their name, a "Yes" and you are connected.

CT83 / Hellin-Worki

A video conferencing platform which seamlessly dials your coworkers when you call out their name, a "Yes" and you are connected.

Problem

Telling your coworkers to be constantly connected on 4 hour long video calls is obtrusive, awkward and plain weird. You could stay connected and disable your video, but then they would still hear your chair squeak, the pigeons out the window and the traffic outside. Nothing quite emulates the comradery of your cozy old office, does it?

Nothing quite replaces sitting across the table from each other and collaborating, does it?

Nope, but Hellin' Worki changes that.

Working

You and your coworker Joe join a Room.
Both enter their names
The microphone listens and waits for you to say "Hey Joe!"
When you do, Joe is notified and he can say, "Yes!" to pickup the call.

That's it!

Demo

Tech Stack

Video Conferencing - Twilio Video API
Speech Recognition - Annyang
Backend - Flask - Python
Frontend - ReactJS - JavaScript

How to Run

git clone https://github.com/CT83/Hellin-Worki.git
Create the .env file. (Read more, in the Appendix)
docker-compose up --build
Visit http://localhost:5000!

Infrastructure

Advantages

Calling out your coworker by their names is natural, uninstructive and seamless
Responding to calls is as simple and easy
You don't have to stay connected to the video call for hours on end.
Your privacy is maintained as you are connected only when you respond.

Screenshots

Home Page

Empty Room

Incoming Call

Call in Progress

Why can I just run the live Demo? 😑

Well, Twilio Video costs real money (0.01$ per participant/minute) every time someone makes a call, and hosting it for free would not be cheap. So...yeah.

But, hey! If you need help setting this up send me an email!

CT83 / Hellin-Worki

A video conferencing platform which seamlessly dials your coworkers when you call out their name, a "Yes" and you are connected.

🐣 Hellin' Worki - Now, Joe's just a shout away

Hellin' Worki is a video conferencing platform which seamlessly dials your coworkers when you call out their name, a "Yes" and you are connected.

Problem

Nothing quite replaces sitting across the table from each other and collaborating, does it?

Nope, but Hellin' Worki changes that.

Working

You and your coworker Joe join a Room.
Both enter their names
The microphone listens and waits for you to say "Hey Joe!"
When you do, Joe is notified and he can say, "Yes!…

View on GitHub

License

Apache License 2.0

A permissive license whose main conditions require preservation of copyright and license notices. Contributors provide an express grant of patent rights. Licensed works, modifications and larger works may be distributed under different terms and without source code.

References

https://www.twilio.com/blog/video-chat-react-hooks

Appendix

1. Setup

We need 5 things, add these in an .env file like this

TWILIO_ACCOUNT_SID=XXXXXXXXXXX
TWILIO_API_KEY=XXXXXXXXXXXXXXX
TWILIO_API_SECRET=XXXXXXXXXXXX
TWILIO_AUTH_TOKEN=XXXXXXXXXXXX

Get the API Key and Secret from here - https://www.twilio.com/console/project/api-keys
Get the Account SID and Auth. Toke here - https://www.twilio.com/console

2. Hellin' Worki? What's with the name, Rohan?

I used a random name generator and it generated a random name.

List of Cool Open Source DL/ML repos by the Giants

Rohan Sawant — Mon, 20 Apr 2020 05:25:31 +0000

I am trying to curate a list of interesting Github Repos and engineering blogs released by the Tech Giants!

Here, is what I have got so far.

Ludwig - Uber
Ludwig is a toolbox built on top of TensorFlow that allows to train and test deep learning models without the need to write code. - Read More
Kraken - Uber
Kraken is a P2P-powered Docker registry that focuses on scalability and availability. It is designed for Docker image management, replication and distribution in a hybrid cloud environment. With pluggable backend support, Kraken can easily integrate into existing Docker registry setups as the distribution layer. - Read More

More Coming Soon.

Do you have some interesting Repos, to add to the list?

Criteria:

ML, DL Projects
DevOps
Experimental Repos not included

I built an Overly Complex Random Number Generator with Docker Compose!

Rohan Sawant — Mon, 13 Apr 2020 11:16:02 +0000

This is an Overly Complex Random Number Generator, created to demystify how containers work.

Demo

User sets a Random Seed
Clicks Generate a Random Number
Random Number is generated

CT83 / The-Overly-Complicated-Random-Number-Generator

An Overly Complex Random Number Generator, created to demystify how containers work.

The-Overly-Complicated-Random-Number-Generator

This is an Overly Complex Random Number Generator, created to demystify how containers work.

How to Run?

Easy peasy lemon squeezy. 🍋💦

docker-compose up --build
Visit localhost

Below is an excerpt from my Blog Post about this.

Demo

User sets a Random Seed
Clicks Generate a Random Number
Random Number is generated

Prerequisites

You have an understanding of Docker and Docker Compose
You have heard about Python, React and Redis

Aim

To illustrate how containerization actually looks in the wild
We will be skipping how the actual Python, JavaScript Apps work, as we want to focus on containerization here, you can find all of the code on Github - CT83/The-Complicated-Simple-Container-App

Architecture

Nginx Proxy - Exposed on port 80, Manages networking
React Dev Server - Accepts Input, Supports Live Reloading, Replaced with Nginx in Production
Flask Server - Serves the API
Redis Cache - To maintain the Task Queue
Python…

View on GitHub

Prerequisites

You have an understanding of Docker and Docker Compose
You have heard about Python, React and Redis

Aim

To illustrate how containerization actually looks in the wild
We will be skipping how the actual Python, JavaScript Apps work, as we want to focus on containerization here, you can find all of the code on Github - CT83/The-Complicated-Simple-Container-App

Architecture

Nginx Proxy - Exposed on port 80, Manages networking
React Dev Server - Accepts Input, Supports Live Reloading, Replaced with Nginx in Production
Flask Server - Serves the API
Redis Cache - To maintain the Task Queue
Python Worker - Uses specified Seed to Generate Random Number, stores it in DB
Database - PostGreSQL DB to store generated random numbers

Working

React App is served, user inputs the random seed.
Seed is sent to the Flask API through a POST
Task is added to Queue
Worker picks up the task, generates a random number and store result in the database
Result is returned to the user.

Configuration

Here, is the minified docker-compose-prod.yml

version: "3.3"

services:
  proxy:
    container_name: proxy
    build: 
      ...
    ports:
      - 80:80

  database:
    container_name: database
    build: database/.
    volumes:
      - ./database/db_data:/var/lib/postgresql
    ...

  client:
    container_name: client
    build: 
      ...
    environment:
      ...

  api:
    container_name: api
    build: 
    ...
    volumes:
      - ./api:/app

  worker:
    container_name: worker
    build:
    ...

  redis:
    container_name: redis
    build: 
    ...

You can find the development docker-compose.yml here. It even supports Live Reloading for both, the Flask and React Container!

Next Steps

You could now move on to trying to push the built images to DockerHub, or add TravisCI integration!

Inspirations

Stephen Grider in his Udemy Course Docker and Kubernetes: The Complete Guide built an overly complex simple app, that greatly inspired me.

How to send SMS in 5 seconds right from your Terminal?

Rohan Sawant — Sat, 11 Apr 2020 06:10:50 +0000

📨Shell-SMS-Shender

A Simple command line tool to send SMS from the command line using the Twilio API.

Send SMSes Swiftly right from your Shell

CT83 / Shell-SMS-Shender

A Simple command line tool to send SMS from the command line using the Twilio API.

Getting Started

Clone the repo. git clone https://github.com/CT83/Shell-SMS-Shender.git, cd into it
chmod 775 sss-client
Head to the Twilio console. Either log-in or sign up for a free account, and look for the Account Summary, make sure you have already bought a Twilio Phone Number.
Copy the credentials and your phone number in a new file .config, and add it to the root of the repo, like so.```bash

ACCOUNTSID=XXXXXXXXXXXXXXXXXXXXXXXX
AUTHTOKEN=XXXXXXXXXXXXXXXXXXXXXXXXXX
CALLERID=507-697-0XXX

5. Done! Type `./sss-client` 

### License

MIT License. You are free to do anything. 🙃

### References

I found [this](https://www.twilio.com/labs/bash/sms) too by Twilio while researching, I modified it to suit my needs.

Hellin' Worki - Now, Joe's just a shout away - TwilioHackathon

Rohan Sawant — Sun, 05 Apr 2020 14:43:18 +0000

Hellin Worki is a video conferencing platform which seamlessly dials your coworkers when you call out their name, a "Yes" and you are connected.

Problem

Nothing quite replaces sitting across the table from each other and collaborating, does it?

Nope, but Hellin' Worki changes that.

Working

You and your coworker Joe join a Room.
Both enter their usernames
The microphone listens and waits for you to say "Hey Joe!"
When you do, Joe is notified and he can say, "Yes!" to pickup the call.

That's it!

Advantages

Calling out your coworker by their names is natural, uninstructive and seamless
Responding to calls is as simple and easy
You don't have to stay connected to the video call for hours on end.
Your privacy is maintained as you are connected only when you respond.

Tech Stack

Twilio Video for the actual Video Conferencing
Flask Web Framework with Python for the backend.
Simple HTML and Vanilla JS for the frontend

and more.

The Build!

How far along am I?

I just got the video conferencing demos by Twilio and the Hotword Detection working, so far so good!

CT83 / Hellin-Worki

A video conferencing platform which seamlessly dials your coworkers when you call out their name, a "Yes" and you are connected.

🐣 Hellin' Worki - Now, Joe's just a shout away

Hellin' Worki is a video conferencing platform which seamlessly dials your coworkers when you call out their name, a "Yes" and you are connected.

Problem

Nothing quite replaces sitting across the table from each other and collaborating, does it?

Nope, but Hellin' Worki changes that.

Working

You and your coworker Joe join a Room.
Both enter their names
The microphone listens and waits for you to say "Hey Joe!"
When you do, Joe is notified and he can say, "Yes!…

View on GitHub

Deploy a Module to Azure IOT Edge Runtime on Linux - Minified - 2

Rohan Sawant — Wed, 18 Mar 2020 12:50:09 +0000

After IOT Edge Hub and Runtime are set up, the next step is to deploy a Module to the Edge Device, and that's exactly what this tutorial is about!
The official documentation is very verbose and wordy, which is exactly how it should be, but this tutorial tries a more brief approach.

Prerequisites

You followed Tutorial 1 where we Setup Azure IOT Edge Portal and IOT Runtime on the Raspberry Pi

You have setup the Azure IOT Hub and the IOT Edge Runtime on a device
You have verified that the Edge Device can talk to Azure

You will learn

How to deploy a Module onto the Edge Device from the Azure Portal
How to view the messages sent from the Edge Device to Azure

Key Concepts

Modules - Read More

modules are the smallest unit of computation managed by IoT Edge.
can contain Azure services (such as Azure Stream Analytics) or your own solution-specific code.
modules images exist as container images in a repository.

Working of the Deployment Process - Read More

You tell Azure what Module should be deployed to the Edge Device
Azure tells the Device about it via a configuration json file.
Device reads the json, and pulls the required Modules (container images) from a docker registry

Deploying a Module to IOT Edge Device

1) Sign in to the Azure portal and navigate to your IoT hub from the left pane under Automatic Device Management, select IoT Edge and select the Device you want to deploy to.

2) On the upper bar, select Set Modules, then under IoT Edge Modules, click Add.

3) Select Marketplace Module, search for "Simulated Temperature Sensor", select that, accept the defaults, next, review and create

4) The Device periodically checks Azure for updates and it will automatically deploy the new Module, this might take about 5-10 minutes.

Confirming that the Deploy worked.

1) Wait for the changes to propagate

2) On the terminal on your Edge Device to see list of deployed modules do, sudo iotedge list

3) Ensure that you see 3 modules deployed.

References - Deploy a Module - Quickstart: Deploy your first IoT Edge module to a virtual Linux device

Viewing the Message on Azure IOT Hub

1) Open up the Azure Cli from Azure Portal, you should be already logged in.

2) Install the IOT Hub Extension

   az extension add --name azure-cli-iot-ext

3) View the Logs sent by the Edge Device to Azure

   az iot hub monitor-events -n <NAME-OF-YOUR-HUB>

That's it! You have successfully deployed your first Module from the Azure Portal, via the IOT Hub to the IOT Edge Device!

Note

I know It's frustrating when something works in the tutorial but not when you do it, I would hate if that happened to you. So, please leave a comment, or reach out to me. If any of this does not work for you.