Or, in spanish, "Sistema Quirófano Remoto".
Yes, my BCompSC thesis was my side project because I started a full-time remote job by the end of college and I only could be able to work on my thesis at night and on weekends. I had a tough yet very creative tutor and this was his idea:
"So, we have these analog video recording cards that we can attach to PCI ports and record the video signal of cameras and devices like laparoscopes during a surgical practice. I wonder if we can transmit it somewhere else with high quality so another surgeon could give advice or even teach remotely".
Yes, PCI, not even PCI Express. It was 2008 and there was no money for advanced hardware like that. By that year, commercial internet was also really slow in this corner of the world. 1 Mbps was considered extremely fast and there was no such thing as simultaneous HQ video with that speed.
"But, do you know what Internet 2 is? We have that type of Internet here and so other buildings in the city including the Urology Institute. With Internet 2 you can connect Valencia (Venezuela) and Valencia (Spain) with 56 Mbps and imperceptible delay".
Woohoo! I imagined a new world with that speed! But soon a realized that it isn't commercial so I wouldn't find YouTube on it. 😞
"So your duty, if you accept it, is to build some kind of software where you could transmit and let doctors assist, and review, remote surgeries letting them capture video signal from at least 4 cameras or equipment at the same time, hold video conferencing, and accessible through a web browser"
It took me exactly 12 months to complete what was proposed. Mainly because the lack of time, and also testing phase took longer than what was expected because surgeries can't be scheduled based on my free time so I had to quit my job by the end of the project if I was willing to get my BCompSc degree by 2010.
I also wrote a paper, with my tutor, about this awesome project and you can get it here.
Deciding The Stack
Backend
- PostgreSQL: I wanted so bad to use MySQL, but public university encourage the use of PostgreSQL for reasons that nobody knew and nobody knows. NoSQL was also an option but I had 0 expertise on that field so I played it safe. A database was needed to store user and surgery information so can be acceded later to review a past procedure for teaching or reviewing purposes.
- PHP: Also, playing safe. My job was about building Drupal apps. So, PHP was the smart choice.
Middleware
- Media Server: Well, you just can't stream video without something in the middle that could let you watch live or resume a video stream. You need a media server and there were a few options.
The following benchmark exposes an assessment we made in order to select which media server will be used. The were some open source solutions like Red5 and Wowza but Wowza documentation was so poor we couldn't test it without failing on every attempt.
Wowza |
Red5 |
FMS |
|
| Multiplatform | Yes | Yes | Yes |
| Open Source | No | Yes | No |
| Performance | Limited | Limited | High |
| Codebase | Java | Java | C++ | Documentation | Not available | Poor | Good |
| Has RTMP | Partial | Yes | Yes |
| Supported Video Codecs | FLV, H.264 | FLV | FLV, H.264 |
| Supported Audio Codecs | FLV, MP3, AAC | FLV, MP3 | FLV, MP3, AAC |
| AMF0 and AMF3 Support | Partial | Partial | Yes |
| Clustering | Yes | Yes | Yes |
| Server Side Programming | JS and limited AS3 | JS | AS3 |
| Streaming | Yes | Yes | Yes |
| Shared Objects | Partial | Partial | Yes |
Red5 worked pretty fine with low res videos but it couldn't handle 480p using OnVP6. On the other hand, we had Flash Media Server. Written in C++, it was fast and reliable even with resolutions like 720p (remember, 2008 and PIV computers). The free license of Flash Media Server only was able to handle up to 10 streams at the time and it was enough!
Also, streaming isn't the only thing it was able to handle, also Shared Objects so apps like realtime chat and real time whiteboards were possible to implement with such middleware.
- AMFPHP: this middleware library was responsible for serializing AS3 objects so they could be transferred from and to the backend. It still exists on GitHub. But why am I mentioning Action Script here? Well, you'll find why in the frontend section.
Frontend
Analogous to the media server, we benchmarked 4 scenarios or technologies HTML & JS, Laszlo, Silverlight, and Flex. I won't show that table here, it is bigger than the other one and I don't want to annoy you (more...) 🙈
Before Angular, React or VueJS, there was something called Rich Internet Applications in a world where HTML5 & ECMAScript were not able to deliver desktop like solutions in your browser. Java Applets were already obsolete and Adobe and Microsoft decided to implement runtimes to fill this gap.
Adobe was saying that they Open Sourced an Object-Oriented framework (Adobe Flex) that would let you write amazing apps, using a language very similar to Java (Yes, Action Script 3) and style them using its API or CSS. Also, it would integrate flawlessly with Flash Media Server so we got a winner!
The following code shows how we use AMFPHP inside one of the classes of the project written in AS3. Follow the comments to understand it better.
package assets {
import flash.events.*;
import flash.net.NetConnection;
import flash.net.Responder;
public class PGDBConexion {
//The file gateway.php has a class with a set of public methods prepared to receive an object
private var gateway:String = "http://127.0.0.1/amfphp/gateway.php";
private var conexion:NetConnection = new NetConnection();
public function PGDBConexion() {
this.conexion.addEventListener(NetStatusEvent.NET_STATUS, netStatusHandler);
this.conexion.connect(this.gateway);
}
private function netStatusHandler(event:NetStatusEvent):void {
...
}
public function call(funcion:String, responder:Responder):void {
//The variable 'funcion' is the name of the method inside the php class in gateway.php
//and responder is the callback function that will receive the response
this.conexion.call(funcion, responder);
}
}
}
And when we need to call a particular endpoint or method we can do something like this
package assets {
...
import flash.net.Responder;
public class SelectorStream extends Panel {
...
private var conexion:PGDBConexion = new PGDBConexion();
private var responder:Responder;
...
public function SelectorStream() {
super();
...
this.responder = new Responder(onResult, onFault);
//This is were we invoke the getStreams method from gateway.php
this.conexion.call("pgDB.getStreams", responder);
}
private function onResult(data:Object):void {
//The object 'data' will contain the response already serialized to be used in AS3
...
}
private function onFault(fault:Object):void {
...
}
}
}
Formal Software Engineering
Let's remember this was a formal software engineering challenge so we had to documentate professionally everything so I'm sharing a few diagrams that describe the overall architecture of the solution. I used UML, of course. 🤓
Use Case Diagram
This is the Use Case diagram of one of the three possible roles that a user could have inside the system.
Entity-Relationship Diagram
Class Diagram
This is a small subset showing only two types of relationships and methods
Deployment Diagram
Final words
This Side Project was software engineering as its finest for me!
I did love this project, I researched, I tested, I delivered small increments using the MVP approach without knowing it was called that way.
I did something nobody has done before and that system is still running on a Sun Server at the Urology Institute here, in Valencia, Venezuela.
This was my last side project until 2020, where I started two new side projects. Now I have 10 additional years of experience and technology has changed. I'm willing to share more side projects with you guys!
Unlike the other ones, I do have the source code and binaries of this gem. 💎
Thanks for reading! And if you like it, then like it, clap it or share it! 😜
Top comments (4)
Hi Javier,
I enjoyed the post 😊, tho it was amazingly coding full for me. But I got the message. I have questions tho, I'm currently building a webinar application with mobile app, like you said we have picked red5 as our middleman. Like others webinar we want to be able to streaming to over 10k to 100k viewers. So what do you think and my team agreed to use React and Flutter for the mobile version.
I haven’t used red5 in a while but I think the biggest problem you could challenge is concurrency and delay if the media server isn’t hosted on a powerful machine or nor have load balancing. Hopefully, VPS are faster these days. That was the problem we had with red5 10 years ago, too slow, but hardware has evolved. On the other side, I find interesting you’ll be using react! I’m kinda a VueJS person but react has plenty UI tools like video players that could help you. And one question, why don’t you start a MVP first using YouTube live? Is that a big team? Because you could focus first on the UI and later on the middleware.
I definitively would like to test your product one day. I think it will be amazing!
Thanks for your response.
So we are looking at broadcasting to over a million viewers. At speaking with red5, the price where off the roof😂. What streaming service can help me streaming over 500k users. One to many.
With no joke, YouTube through WebRTC :/