Live-Video-Stream App

Hello everyone,

In this blog we are going to discuss how we can create Live Video Stream app .
Before we start with the demo part first lets discuss the basic terminologies that will help in understanding the blog better.

What is OpenCV?

OpenCV is a library that works with almost all programming languages therefor known as cross-platform library which is mainly used for Image Processing that is used in Computer Vision

What is Socket Programming?

Socket programming is a way to connect two nodes on a network to communicate with each other. for details

What is socket?

A socket is one endpoint of a two way communication link between two programs running on the network.

What are the Network Protocol ?

Protocol are the set of rules that are followed while transmitting data over network.
Here we are going to use TCP( Transmission Control Protocol).

what is TCP?

TCP Stands for transmission control protocol that enables application programs and computing devices to exchange messages over a network, it is designed to send packets across the internet and ensure the delivery of the data. For Detail

Prerequisites:

• Numpy

• OpenCv

• Now lets begin with the code

first we need to create a Server program, Client connects to the server program.

soc = socket.socket()
soc.bind(("", 5678))
soc.listen()
session, address = soc.accept() from anywhere
print(session.recv(2046)) 
camIndex = 'http://IP-of-WebCam:8080/video'
cam = cv.VideoCapture(camIndex) 

def sender():
    while True:
        ret, photo = cam.read()
        photo = cv.resize(photo, (640, 480))
        print(photo.shape)
        if ret:
            session.send(np.ndarray.tobytes(photo))
        else: print("No Frame Found")

def receiver():
    framesLost = 0
    print("Connecton Established")
    while True:
        framesLost += 1 
        data = session.recv(100000000)
        if(data == b'finished'):
            print("Finished")
            cam.release()
            session.close()
        else:
            photo =  np.frombuffer(data, dtype=np.uint8)
            if len(photo) == 640*480*3:
                cv.imshow('Hi I am Client', photo.reshape(480, 640, 3))
                if cv.waitKey(1) == 13:
                    session.send(b'finished')
                    cam.release()
                    cv.destroyAllWindows()
                    break
            else:
                print("Lost {} frames".format(framesLost) )

threading.Thread(target=sender).start()
threading.Thread(target=receiver).start()

now let's understand the code

soc = socket.socket()
soc.bind(("", 5678)) 

Here socket is a function that comes from the socket library, to use it you need to import the socket library first : import socket

Here bind function is use to bind the IP to the port number. In the above code " " means the IP of local system and 5678 is the port number that we have bind with the local host IP.

session, address = soc.accept() #here it will accepts the request from anywhere
print(session.recv(2046)) 
camIndex = 'http://IP_of_IPWebCam:8080/video' # here this IP is of IPCAMERA app 
cam = cv.VideoCapture(camIndex)

Here accept() is use to accept the the request from the client.

session.recv() show the receives data on the socket and store it in a buffer and here 2046 is the size of buffer in bytes.

camIndex variable stores the address of my camera, here I am using my mobile camera, for that I have used the IP WebCam android app which create a server through which we can use the camera of the android device.

VideoCapture() is use to start the camera and captures the video(multiple images)

def sender():
    while True:
        ret, photo = cam.read()
        photo = cv.resize(photo, (640, 480))
        print(photo.shape)
        if ret:
            session.send(np.ndarray.tobytes(photo))
        else: print("No Frame Found")

Here is sender function that sends the client the video.

read() is use to read the captured images

resize() is use to resize the image

send(np.ndarray.tobytes(photo)) is use to send the image in the byte forma, here numpy is use to convert the numpy array that contains the value of image in byte.

def receiver():
    framesLost = 0
    print("Connecton Established")
    while True:
        framesLost += 1 
        data = session.recv(100000000)
        if(data == b'finished'):
            print("Finished")
            cam.release()
            session.close()
        else:
            photo =  np.frombuffer(data, dtype=np.uint8)
            if len(photo) == 640*480*3:
                cv.imshow('Hi I am Client', photo.reshape(480, 640, 3))
                if cv.waitKey(1) == 13:
                    session.send(b'finished')
                    cam.release()
                    cv.destroyAllWindows()
                    break
            else:
                print("Lost {} frames".format(framesLost) )

Here is receive function that will receive the data from the client.

frameLost will count the lost frames during the transmission of the data.

session.recv(100000000) here recv function will receive the data in the limit of 100000000

cam.release() is use to close the webcam.

session.close() is use to close the session between client and the server.

np.frombuffer(data, dtype=np.uint8) here numpy is use to convert the data in byte format to numpy array.

photo.reshape(480, 640, 3)) is use to reshape the image in the following dimensions.

cv.waitKey(1) == 13 here waitKey() is use to hold the window that contains an image/video and here is 13 is the code for Enter Key in the keyboard.

destroyAllWindows() this will destroy or close the window as soon as enter key is pressed.

threading.Thread(target=sender).start() threading.Thread(target=receiver).start() is use to start the thread. Here thread will continue listening for request from the client.

• Thread:

A thread is a sequence of instruction that run independently of the program and of any thread

• Now we need to create program for Client:

Like the code in server program, we need to code the client program.

soc = socket.socket()
soc.bind(("", 1234))  
server_ip = input(" Enter the Server IP adress -> ")
server_port = input("Enter the Server Port number -> ")

soc.connect((server_ip, int(server_port))) 
soc.send(b"connected")  
camIndex = 0 
cam = cv.VideoCapture(camIndex) 

 server: Client <- Server
def receiver():
    framesLost = 0
    print("Connection Established")
    while True:
        framesLost += 1 
        data = soc.recv(100000000) 
        if(data == b'finished'):
            print("Connection END")
            cam.release() 
            soc.close() 
        else:  
            photo =  np.frombuffer(data, dtype=np.uint8)  
            if len(photo) == 640*480*3: 
                cv.imshow('Hi I am Server', photo.reshape(480,640,3))
                if cv.waitKey(1) == 13: 
                    soc.send(b'finished')
                    cam.release() 
                    cv.destroyAllWindows()  
                    break
            else:
                print("Lost {} frames".format(framesLost) )          
def sender():
    while True:  
        ret, photo = cam.read()
        photo = cv.resize(photo, (640, 480)) 
        print(photo.shape)
        if ret:
            soc.send(np.ndarray.tobytes(photo))
        else: print("No Frame Found")
    cam.release()

threading.Thread(target=receiver).start()
threading.Thread(target=sender).start()

All code is explained in the server program and client code also works in the same way.

Watch the Demo 👇

Hope you find this blog insightful.