What is WebRTC SFU (Selective Forwarding Unit)?
Selective Forwarding Unit or SFU
SFU is an server based architecture that is used in webrtc to conduct multi party video calling, audio calling and data transfer.
The SFU acts an intermediary between devices that recives media streams from multiple devices and then decides which streams must be send to which devices.
What is WebRTC?
WebRTC is an open source project, using which you can enable real time audio, video and data transfer between devices
WebRTC runs in web browsers as well as in apps, WebRTC is supported by almost all modern browsers and is an standard for video, audio communication
How does SFU or Selective Forwarding Unit work
here is a step by step guide on how does an SFU work
Step 1 Client connections and media capture
- Participant connection
Each participant first establishes a connection with the SFU. This process involves a signalling server where session descriptions are exchanged.
the session description contains important information, which is used to establish a session.
2. Media stream capture
On the client side the media streams are captured from the participant devices, like audio from microphone, video from camera etc. In the next step these streams are sent to the SFU
Step 2: Stream Transmission to SFU
- Encoding and sending the stream: The client device encodes the media stream using codecs for video and audio and sends it to the SFU
This is done using the RTP protocol or the SRTP protocol if you need encryption
2. Receipt by SFU
The SFU or the Selective forwarding unit recieves the individual streams from each connected device and maintains a seprate inbound stream for each device.
Step 3 Stream Processing at SFU
- No Mixing
In SFU as compated to MCU all the streams are seprate from each other and there is no mixing
2. Selective Forwarding
The SFU recieves a seprate stream from each device that is connected to the SFU and then the SFU maintains a seprate stream for each participant
Step 4: Stream Distribution by SFU
- Adapting:
Before forwarding the stream to uesrs the SFU performs certain adaptations to the stream, such as chossing different quality video codecs and selecting between different simulcast streams.
These are based on the receiving client CPU and Bandwidth capabilities
2*. Forwarding Streams*
The SFU forwards the stream which each client is asking for.
The client devices could receive multiple of streams from the SFU or even a single stream of video and audio data
- RTP or SRTP Packets
The media is sent in RTP or SRTP packets, and because different client devices might have different cpu and bandwidth limitations.
The SFU optimizes the streams packets to flow such that the stream flows smoothly and the video quality is smooth on the devices
Step 5 Client Reception and Display
Decoding: Each client recieves a RTP streams from the SFU, the client then decodes the stream and renders it on the screen
Synchronization: Audio and video synchronization is important and it is handled at the client level.The client uses the timestamps and the sequence numbers from the RTP stream to sync video and audio playback smoothly
SFU Vs p2p vs MCU
In this section let us consider the distinctions between SFU that is Selective forwarding unit, Peer to Peer or P2p and MCU that is Multipoint Control Unit
1. Selective Forwarding Unit SFU
selective forwarding: The SFU recieves multiple streams and selectively forwards the stream to devices without altering stream content
Server Based: The SFU relies on a server but the server is used for routing the steams and not for anything else.
Advantages:
Scalability: the SFU is highly scalable and it reduces the load on single client devices by managing which stream is sent to any client device
Low processing requirement: Because the SFU just forwards the stream and does not do any processing on the stream, thus reducing the latency and the cpu processing requirement
Stream flexibility:
The client devices can recive multiple streams or just one stream or no stream at all, it all depends on which stream the client device is asking for and needs to display it on the screen
Disadvantages:
Bandwidth Intensive: The SFU are bandwidth intensive on the server side because each stream is recieved by the server and the SFU should have the CPU and the bandwidth to handle all the streams
2. Peer-to-Peer P2P
In P2p architecture each participant connects to another participant directly without the need for an intermediate server
but often a STUN server or a TURN server is required for NAT traversal and to relay the streams around NAT
if you are looking for a turn server you can consider the Metered TURN server service provider
Metered is a Global TURN server provider with TURN server all around the world, that connects to the user nearest to the TURN server for minimum latency and best performance
Disadvantages:
Each participant device must handle multiple streams if there are multiple devices in a meeting.
Which is CPU and bandwidth intensive on all the client devices. For this you can consider the Metered SFU
Multipoint Control Unit
The MCU mixes all incoming streams and creates a single stream or few streams out of a many streams and then sends it to all the clients
The MCU requires a lot of CPU computation and resources on the server, it also creates latency when transmitting streams but it has one advantage, it can greatly reduce the computation and bandwidth requirement on client devices
When in the past the mobile devies were pretty slow and the network infrastructe was also bad, in those days MCU made a lot of sence
But these days with the advance in technology it perhaps provides a bad experience with increased latency and slow video delivery.
Advantages of SFU in WebRTC
Here are some of the advantages of SFU in WebRTC
1. Scalability
the most important advantage of SFU in WebRTC is scalability. You can also scale P2P video calling but it is difficult to create a mesh and scale it
- Reduced Client Bandwidth
When you are video calling with multiple client devices in peer 2 peer format, each client must send their stream to all the other clients and vice versa.
Because of these there is exponentially greater requirement of bandwidth as well as CPU for each client
Some client that do not have capacity for very high bandwidth and cpu compute struggle with the video calling
2. Improved Quality Management
The SFU can make intelligent decisions regarding which streams to send to which clients thus saving imporving quality of video
- Stream Adaptation
The SFU can adapt the stream according to the bandwidth and CPU capacity of the recieving devices.
If the client device has lower bandwidth then the SFU can send low quality streams to that perticular device so that the video runs smoothly without buffering
3. Reduced Processing Load on Clients
The SFU handles the majority of the data processsing that is required for stream management, the client devices themsevles are freed from all the heavy lifting
This is helpful for mobilde devices and devices with low processing power that is CPU power
- Less CPU intensive
Client devices do not face a need for encoding and decoding the multiples streams that are comming to the SFU for processing instead the SFU does this for them
Thus these devices drink less battery run cooler and require less cpu time
- User Experience
The reduction in CPU processing leads to smoother interface, cooler devices and overall better user experience
4. Flexibility in Media handling
SFU offer greater flexibility in how you want to handle the media steams and how you want the video to be displayed on the participants client devices
- Individual Stream Control
The client deives or the meeting participants can choose which streams they want to download and also how they want to display the stream on their screens
They also have the ability to download different quality of strems or make it automatic such that the SFU chooses the best stream based on the client device bandwidth capacity and CPU capability
- Support for advanced featuers
SFU or the Selective Forwarding Unit supports a variety of advanced features like
simulcasting ( ability to send multiple qualites of a single video stream), scalable video codec and many more
5. Enhanced Flexibility and Security
All the streams passing through the SFU are end to end encrypted meaning no one not even the SFU know what streams are passing through it
The encryption protocols such as DTLS and SRTP are used for end to end encryption
the SFU can log the interactions but cannot intercept the streams for compliance purposes as well.
Metered TURN servers
API: TURN server management with powerful API. You can do things like Add/ Remove credentials via the API, Retrieve Per User / Credentials and User metrics via the API, Enable/ Disable credentials via the API, Retrive Usage data by date via the API.
Global Geo-Location targeting: Automatically directs traffic to the nearest servers, for lowest possible latency and highest quality performance. less than 50 ms latency anywhere around the world
Servers in all the Regions of the world: Toronto, Miami, San Francisco, Amsterdam, London, Frankfurt, Bangalore, Singapore,Sydney, Seoul, Dallas, New York
Low Latency: less than 50 ms latency, anywhere across the world.
Cost-Effective: pay-as-you-go pricing with bandwidth and volume discounts available.
Easy Administration: Get usage logs, emails when accounts reach threshold limits, billing records and email and phone support.
Standards Compliant: Conforms to RFCs 5389, 5769, 5780, 5766, 6062, 6156, 5245, 5768, 6336, 6544, 5928 over UDP, TCP, TLS, and DTLS.
Multi‑Tenancy: Create multiple credentials and separate the usage by customer, or different apps. Get Usage logs, billing records and threshold alerts.
Enterprise Reliability: 99.999% Uptime with SLA.
Enterprise Scale: With no limit on concurrent traffic or total traffic. Metered TURN Servers provide Enterprise Scalability
5 GB/mo Free: Get 5 GB every month free TURN server usage with the Free Plan
Runs on port 80 and 443
Support TURNS + SSL to allow connections through deep packet inspection firewalls.
Supports both TCP and UDP
Free Unlimited STUN
Top comments (1)
Thank you for reading. I hope you liked the article