Video Frame Transmission USing Python

Over the years, video streaming across the network has turned out to be a necessity for multiple reasons regarding security, management, or monitoring as such it’s very likely that you’ll find yourself in a situation where you might need video streaming over the network for your platform.

In this article, we will be developing a simple and effective python script to comply with our requirements whilst discussing other approaches along the way.

Getting started

Step-1: Installing requisites libraries

pip install opencv-python
Pip install imagiz

Imagiz is a python library developed explicitly with the intention of live video streaming over the network. While imagiz provides two ways of video transmission (over ZMQ and TCP server) we will be developing our script using ZMQ server, as Zero MQ (ZMQ) is high-performance asynchronous message passing library which specifically focuses on high throughput and low latency applications.

To those who are already familiar with image processing in python OpenCV is not a new concept. I recommend you visit these sites to gain better insight of discussed libraries.

https://www.geeksforgeeks.org/opencv-python-tutorial/ https://pypi.org/project/imagiz/ https://www.pyimagesearch.com/2019/04/15/live-video-streaming-over-network-with-opencv-and-imagezmq/

Step 2: Creating Basic Server and Client

# Server.py
import imagiz
import cv2
import multiprocessing
import os
import signal


def start_server():

    server=imagiz.Server(port=7070) # Starting server instance on port 7070
    while True:
        message=server.receive() # Recieve incoming frame forever with While True loop.
        frame=cv2.imdecode(message.image,1) # Decoding Bytes of Image
        cv2.imshow("Video",frame) # Showing decoded frame
        if cv2.waitKey(1) & 0xFF == ord('q'): # Close visualization panel if 'q' is pressed
          break

    cv2.destroyAllWindows() 
    current_id = multiprocessing.current_process().pid # getting current server process ID

    os.kill(current_id,signal.SIGTERM) # Making sure the server process is terminated and not running in background

if __name__ == '__main__':
    start_server()

Above presented script hosts ZMQ server instance at port 7070. Whilst it is completely okay to host TCP server , I decided to work with ZMQ server for obvious reason as stated in pyimagesearch article. The hosted ZMQ server instance provides connection-oriented transmission. (i.e. Client will send webcam frame only if server is accepting unlike UDP transmission.)

# Client.py
import imagiz
import cv2


client=imagiz.Client("cc1",server_ip="localhost", server_port=7070) # Connect to server ip on 7070 port
vid=cv2.VideoCapture(0) # capturing webcam frames
encode_param = [int(cv2.IMWRITE_JPEG_QUALITY), 90]

while True:
    r,frame=vid.read() # reading captured frame continously for tranmission
    if r:
        r, image = cv2.imencode('.jpg', frame, encode_param) # Encoding captured framed in bytes
        client.send(image) # Sending captured frame over to server side
    else:
        break

The client side script connects to the server on provided port, captures webcam frame and transmits frame over to server side continuously until server is active.

Step-3: Improvising above scripts

Above scripts work fine, however have common issues related to bandwidth and encryption. If you’re familiar with tools like driftnet, you are aware that these transmitted video frames could be captured and seen which violates security concern over transmission. The frame rate transmission and frame size is also not managed which results in lagging issues particularly if you have low bandwidth.

Concerning following reasons above scripts are reformed as.

#Server.py

import imagiz
import cv2
import multiprocessing
import os
import signal
import base64
import numpy
from io import BytesIO


def start_server(port, sv_op):

    server=imagiz.Server(port=port) # Starting server instance on port 7070

    if sv_op: # if sv_op is True the incoming video frame will be saved as Video.avi
      vid_cod = cv2.VideoWriter_fourcc(*'XVID')
      output = cv2.VideoWriter("Video.avi", vid_cod, 10.0, (640,480))


    while True:
        message=server.receive() # Recieve incoming frame forever with While True loop.

        dc_bytes = base64.b64decode(message.image) # Decoding the recieved message using base64 as was encrypted on client side using base64
        df = BytesIO(dc_bytes)
        df = numpy.load(df) # converting bytes into numpy array

        message = cv2.imdecode(df,1)

        # setting resizing parameter as it was compressed to 60% of original size while transmission

        width = int(message.shape[1] / 0.6) 
        height = int(message.shape[0] / 0.6)
        dim = (width, height)

        # resizing frame to original size
        resized = cv2.resize(message, dim, interpolation = cv2.INTER_AREA) 


        cv2.imshow("Video",resized) # showing resized frame

        if sv_op:
            output.write(resized) # Writing frame to file if save option is provided true



        if cv2.waitKey(1) & 0xFF == ord('q'): # Close visualization panel if 'q' is pressed
          break

    try:
        output.release() 
    except:
        pass 

    cv2.destroyAllWindows() # detroy  cv2 window
    current_id = multiprocessing.current_process().pid # getting current server process ID
    os.kill(current_id,signal.SIGTERM) # Making sure the server process is terminated and not running in background

if __name__ == '__main__':
    start_server(7070,True) 

#Client.py

import imagiz
import cv2
import time
from io import BytesIO
import numpy as np
import base64
import os
import multiprocessing
import signal

def send_webcam(port):
    vid=cv2.VideoCapture(0)
    client=imagiz.Client(server_port=port,client_name="cc1",server_ip='192.168.1.17') # establishing connection with the server computer. Note: change serveer_ip to ip of administrative computer
    encode_param = [int(cv2.IMWRITE_JPEG_QUALITY), 90]
    # t_end = time.time() + int(time_)
    frame_rate = 10
    prev = 0
    while True:
        time_elapsed = time.time() - prev
        r,frame=vid.read()
        print('Original Dimension', frame.shape)

        if time_elapsed > 1./frame_rate:
            prev = time.time()
            scale_percent = 60 # percent of original size
            width = int(frame.shape[1] * scale_percent / 100) # compressing frame for easy transmission
            height = int(frame.shape[0] * scale_percent / 100)
            dim = (width, height)

            # resize image
            resized = cv2.resize(frame, dim, interpolation = cv2.INTER_AREA) # resizing frame with provided scale factor

            if r:
                try:
                    r,image=cv2.imencode('.jpg',resized, encode_param) # enconding to bytes

                    np_bytes = BytesIO()

                    np.save(np_bytes, image, allow_pickle=True) # allow_pickle = true allows dimension of numpy array to be encrypted alongside
                    np_bytes = np_bytes.getvalue()

                    en_bytes = base64.b64encode(np_bytes) # base64 encoding of numpy array


                    response=client.send(en_bytes) # sending encoded bytes over to server computer
                    print(response)
                except:
                    break

    vid.release()
    cv2.destroyAllWindows()
    current_id = multiprocessing.current_process().pid
    os.kill(current_id,signal.SIGTERM)

if __name__ == '__main__':
    send_webcam(7070) 

Base64 encoding mechanism has been implemented and an additional option to save incoming frame to a video file is provided. While implementing encryption algorithm for video frame transmission one must also consider encryption speed, as such base64 being simple and effective solution was implemented for this example.

Conclusion

The above presented solution provides with easy and effective way of secured video frame transmission over network using python.