Video+Audio conferencing in your terminal !

Yeah i know, what you guys are thinking "Why ?", "Why do you do that ?", well "just for fun ?" and a way to mastering "real-time parallelism", this is also my first POST ever made!

Let's jump into it !
First of all, a videoconferencing system is a real-time application which consists in broadcasting to each person connected the audio and video stream of all the other participants with the least possible latency !

As I had to work on the terminal, I only had characters and "bytes" for data entry!

First question : how to transform an image into a perceptible string of characters on a terminal ?

My algorithm is very simple, I first use OpenCv to retrieve the image that the camera /dev/0 device gives me, after that, i convert to black on white using the cvtColor method from OpenCV; then i cut this image into a sort of grid which gives me the color contrast by squares ranging from 0 to 255, I have a sort array equivalence of each pixel! By setting another array of characters ranging from "" to "B", I have a match against the occupancy rate in the pixel !
To complicated ?

Let me explain again, more a pixel is dark, the less dense the character will be, so we will have either a "." or a "," with the color value 10 or 12 unlike "B" or "K" which is closer to 250 or 255 From there, I manage to have my transcription of a color image in characters on a terminal !

Second question: how to retrieve the audio stream and transmit it ?

To do this I use PyAudio which connects to the microphone to retrieve the audio stream, then transmit it in the form of bytes per sequences, at the reception, i do a remodeling according to bit rate !

Third question: how to send and retrieve the stream(video and audio) for each session ?

Here I am using a byte formatting system in json by adding elements allowing me to identify the one sending and in which session it is sending it, when a client retrieves the data, analyzes it, then retrieves the contents of the package

Fourth question: how does security work in such a system ?

I have implemented two types of data encryption, asymmetric encryption and symmetric encryption, for both systems there is key sharing that needs to be done first, but in the version being tested I have disabled all encryption, as this has an impact on the final rendering quality!

AES (Symmetric encryption)	RSA (Asymmetric encryption)

To use s2c, just install it via pip !

pip install s2c

Start the server :

s2c_server -p 1122

Then start the client :

s2c -s session1 -c darker -i 127.0.0.1 -p 1122

Or you can use help to see the manual :

Of course, all this article would be useless without a DEMO: WATCH ME

s2c is Open Source and free !

Source Code : https://github.com/sanix-darker/s2c
Pypi-Package : https://pypi.org/project/s2c/

PS : Any comments or constructive advice are welcome !

Comments

[Nino Filiu]

Amazing!

I think it's visually interesting how people appear as ascii, that could defo be used in a bad hacker movie from the 90s...

However I think it could be a nice feature to have this:

1. Sender pose and expression is captured (no need for AI, a simple Viola-Jones-based library could would work)
2. Pose and expression are transmitted over the network (a lot less bandwith is required! 🎉)
3. Pose and expression are rendered as little ascii humans like

 _____
/ o < \
|  U  |
\_____/
   |
+--+----+

[ogoh cyril]

Just wow
Imagine telling someone "let's video chat through our terminal"

Looking likes something from a SCI-Fi movies

Enjoyed reading

[darker]

Thank you !!!

[Krrish Dhaneja]

Awesome 👍 it is!!

[darker]

Thank you !!!

[Samuel David Suarez Rodriguez]

Mind blowing idea!!! Congrats

[darker]

Thank you !!!