Video assets are hard to cache and manage efficiently
A smart, distributed cache layer that extracts as much data as possible from storage to manage video assets and improve playback latency is a technical challenge that none of the major players in the market (Akamai, Fastly, CloudFront, CloudFlare, CDNetworks, etc) have fully responded to. That's because their infrastructure and networks must respond to a large market and multiple use cases.
At api.video, our use case is video. We are focused on building a high-end, high-performance and smart private CDN to specifically manage video assets, powered by a real-time data pipeline composed of RUM (Real User Monitoring) data, customer analytics data and measurement of the health of the whole infrastructure that make it up.
But why is it so hard to manage videos?
Video is the most consumed and fastest growing asset in the world. According to Techsmith, it's preferred over any other communication method including texts or images. Your smartphone camera is changing very quickly every year and the latest models already support 8K resolution with HDR support and video files get bigger and bigger.
To give you an idea of a video file's size from your favorite smartphone:
- 1 minute footage of 1080p/30fps HDR is around 100MB with h264 compression
- 1 minute footage of 1080p/60fps HDR is around 200MB with h264 compression
- 1 minute footage of 4K/30fps with HDR is around 400MB with h264 compression
- 1 minute footage of 4K/60fps HDR is around 600MB with h264 compression
- 1 minutes footage of 8K/24fps HDR is around 800MB with h265 compression
Now that you have this in mind, scale these numbers to 10 minutes or even 1 hour of video footage and think back decades to the heyday of the Divx format when an entire movie was stored on a 700MB disk.🤯
With the massive deployment of optical fibers and the 5G network deployment all around the world, Ultra High Definition will become more popular and super easy and fast to upload.
So let's get back to our concern with caching content on the edge and now I think you're starting to see the problem...
Think about what happens if you have several thousand videos ranging from a few minutes to an hour long that you need to stream all over the world. Now imagine there are many others like you, doing the same thing with their own videos. Public CDNs must also cache other files that travel over the Internet (images, HTML content, JS code, CSS files, etc.) and since they deal with many different assets, they must build and provide an infrastructure capable of handling any kind of content and can not be deeply specialized in just one. Building a CDN specializing in video processing therefore requires a huge storage capacity with adequate hardware and network and also smart caching policies.
All major public CDNs to date rely on SSDs (Solid State Drive) for data caching. Their latency times can reach up to 200 µs. Some of them also support NVMe (Non-Volatile Memory Express) SSDs for caching, lowering access latencies to around 100 µs. These latencies may seem very low at first glance, but in the context of providing an ultra-low latency, end-to-end live streaming service, the smallest amount of latency added to one of the stages has a cascade effect on all following steps.
The caching servers that rely on SSDs and/or NVMe SSDs are good for assets with long lifespan or with long caching time (for example VOD (Video on Demand)) otherwise you will have to change your SSDs often and that will cost you a lot of money. SSDs are also not suitable for assets that have to be delivered with low latency and a very short lifespan.
For this use case, to deliver everything super fast, it's best to rely on DRAM memory whose access latency with the CPU takes nanoseconds.
Let's get closer
CDN's PoPs (Points of Presence) are the last bricks involved in the dissemination and provision of video assets for reading and reaching the last mile of delivery.
Having PoPs on IXPs (Internet eXchange Point) is fundamental when serving a worldwide audience. However, it's not enough to prevent loss of latency in milliseconds during the last miles of delivery. Netflix, which offers the best video playback experience today, taught us this lesson. We have to build CDN PoPs on ISPs(Internet Service Provider) or close to them to reach the last miles. With that in mind, the only remaining network to travel is between the ISPs of the end-users and the devices of the end-users.
As you have probably understood, despite having the best hardware possible, network constraints are an important part (to say the least) of achieving the goal of ultra low latency. Because if you don't know exactly how the Internet works and don't take care about how the different networks exchange traffic between them, you will miss something critical and you will fail.
It's all about the peering policy
The internet connects a ton of very diverse devices around the world (desktop, laptop, mobile, IoT, servers, etc). Most people think of it as a single network, but in reality it is made up of several little networks that communicate with each other through different ISPs and these exchange traffic with larger scale networks.
These networks have two ways to exchange traffic between them: peering or transit. To simplify a lot, remember that transit is where small networks have to pay to send their traffic to a larger one to send it over the Internet through a contract. On the contrary a peering is a Win-Win commercial contract (in general) where two networks of approximately the same size exchange traffic between them for free.
There are two type of peering:
Private peering: a physical link connects the two entities.
Public peering: several networks provide a single physical link to an IXP which makes it possible to pool the peerings agreements with several entities.
In conclusion, the more a telecom operator has peering agreements (private or public), the better it is interconnected with the other networks constituting the Internet and has a direct link with them which has a direct positive impact on traffic latency.
By relying on high end cloud providers we ensure that their peering policy already plays nice with IXP, ISP and other cloud providers. We negotiate additional peering agreements when needed, according to the audience that we face and by building our own data centers and networks in the future.
Manage private content without loosing caching ability
If you've played with our API before, you probably know that we provide a best in class feature to let you protect video streaming and control your audience. Our unique system is the only one in the streaming world that avoids making developers do something to protect video consumption because at api.video, we love to make things easier for them.
As we handle all the complexity on our side, we have to deal with some specific constraints introduced by our system (we love challenges). Each time a private video has to be displayed, we generate a dedicated, unique url for you. It's cool but it prevents most public CDNs from reusing their cache as they rely on the URL to cache and serve the cached content.
We are able to achieve extremely low latency thanks to a Nginx module that we developed internally. We also achieve low latency by managing and caching assets delivered through our private routing system and reuse them each time private content is consumed with another private token without relying on cookies or another legacy token system that can add overhead and latency. We stay compliant and functional with current privacy policies (blocking of third-party cookies) and keep on top of any new ones implemented in all modern browsers.
Smart caching policies
At api.video we like standards and because of that, we rely on standard HTTP headers for all caching rules that are understood by all CDNs and browsers. In case we need to offload some part of the traffic or even all the traffic in case of a total CDN failure to a third party provider, we are not locked by any external providers. This strategy ensures that we never have a total delivery and caching outage at the CDN level.
By having our own CDN infrastructure, we can leverage all intelligence and data that we have to cache with the highest efficiency possible, not relying on public CDNs that need to consider all use cases, and make their own cost compromise. If a new technology is available, we can implement it as soon as possible without having to wait for it to be implemented. We can adapt our incident mitigation to re-balance requests on the PoP that we know will be more stable and with less latency while we fix something. We can leverage the knowledge of our own caching infrastructure, we can pinpoint the performance bottlenecks on it and fix it, while with an external solution we would have to wait for it to be eventually put in their roadmap.
We do all of that to provide you a high-end video platform with the best streaming performance for you and your audiences.
Authored by Anthony Dantard - CTO
Top comments (0)