WebRTC is an open standard. Every browser speaks the same handful of protocols — ICE, DTLS, SRTP — and every tutorial walks you through the same clean offer/answer diagram. What none of those tutorials tell you is what actually happens once you stop talking to a browser tab in a demo and start talking to a real, production video-calling backend.
That's more or less how I ended up down this rabbit hole. I built a raw ICE/DTLS/SRTP client from scratch — no browser, no RTCPeerConnection, just the bare protocols — to talk to Microsoft Teams' calling backend directly. And honestly, I did not expect to enjoy debugging a video call this much. There's something genuinely fun about staring at a wall of encrypted UDP packets, having absolutely no idea why video isn't showing up, and slowly, painfully, working out that the server is just... waiting for you to say something first.
Quick disclaimer before we start: I'm still learning this stuff, not a WebRTC veteran. Everything below is what I actually observed while building this, in my own words — so if something here turns out to be slightly off, that's on my understanding, not on Teams or Microsoft. Happy to be corrected.
This is the full lifecycle of a WebRTC connection — from certificate generation to media actually flowing — with the bits that surprised me along the way called out as we go.
1. Setting Up: The Peer Connection and Certificate
Before any negotiation happens, a RTCPeerConnection generates a DTLS certificate — and it's self-signed. The first time I read that I assumed it was a placeholder for "and then you get a real cert," but no, that's it, that's the whole thing. There's no certificate authority anywhere in the picture. Trust instead comes from the certificate's fingerprint being carried inside the SDP, over whatever signaling channel the app already trusts. It's a neat little inversion once it clicks: the channel you already trust vouches for the crypto, instead of the crypto vouching for itself.
Each session also gets its own fresh certificate, which I like a lot as a design choice — one call's keys tell you nothing about any other call.
Building it in Go — pion: generating this per-session cert is one line with pion's
selfsignpackage. Nothing Teams-specific here, this is just what every conformant WebRTC endpoint does.
2. Signaling: Offer, Answer, and the Negotiation Dance
The textbook version: createOffer() → setLocalDescription() → the offer travels over signaling → the other side calls setRemoteDescription() → it sends back an answer, which travels back the same way. Simple, one round trip, done.
Teams did not do that, and it took me an embarrassingly long time to notice. Its SFU answers with audio only first. Audio comes up fully — keys derived, sound flowing — and then, out of nowhere, a second offer arrives from the SFU itself, adding video. I kept staring at my logs wondering why video was silently missing, before it finally clicked that this wasn't a bug in my code at all — it's a second negotiation I hadn't built a code path for yet. In every session I've tested so far, this two-phase pattern holds consistently.
Sequence diagram of the two-phase negotiation
3. ICE: Finding a Path Between Two Machines
Once SDP has changed hands, both sides gather ICE candidates — host candidates (your real local address), server-reflexive candidates (your public address, discovered via STUN), and relay candidates (via TURN, for when nothing direct works). Both sides run connectivity checks across candidate pairs, and one pair eventually gets nominated as the actual path media will flow over.
The mental model most explanations give you is "trickle ICE" — candidates arrive one at a time as they're found. What I saw from Teams was more of a hybrid: the answer SDP usually already comes bundled with a full set of candidates, not trickled one by one, and then more candidates may show up afterward through separate signaling messages. My first client only expected pure trickle, and it just... missed the whole bundle. Small thing, but it was a genuinely satisfying "oh, that's why" moment once I found it.
4. The DTLS Handshake and Key Extraction
With a candidate pair chosen, DTLS runs over that path to set up encryption. One side sends a ClientHello, the other answers, and a shared secret gets established through the handshake itself. Who goes first is decided by an a=setup value in the SDP — in every session I looked at, Teams' SFU consistently showed up as active, meaning it starts the handshake and my client has to be ready to answer, not initiate.
What I find genuinely elegant here is what happens after the handshake: both sides derive the same master secret, and from that, session keys for the actual media — using a WebRTC-specific extraction label. The expensive asymmetric handshake happens exactly once, and everything after that is fast symmetric encryption. Combined with a fresh cert per call, it means nothing about one call's keys carries over to the next one. I didn't fully appreciate why this layered design existed until I had to implement it myself.
Building it in Go — pion: this is where I hit the sharpest bug of the whole project, and it had nothing to do with Teams at all.
pion/dtlsuses a lazy handshake — it doesn't actually run until you callRead(),Write(), or trigger it explicitly. And after the handshake finishes, pion only keeps reading from the connection while something is actively callingRead()on it — stop, and its internal goroutine quietly exits, silently killing all incoming media even though ICE and SRTP both look perfectly healthy. Took me a while to accept that the fix was genuinely just "have a goroutine that callsRead()forever and throws the result away." A pion quirk, not a Teams one, but it cost me a very confused afternoon.
5. The Media Plane: RTP, RTCP, and Channels
Once keys are in place, media flows as RTP, encrypted as SRTP. Audio and video are logically separate — different SSRCs — multiplexed over the same transport alongside RTCP control messages, all sharing one socket, which means you also need to sort out what kind of packet you're even looking at before you can do anything with it.
Beyond audio and video there are also data channels, running over SCTP-over-DTLS, generally used in production apps for things like chat or reactions alongside the call. I'll be upfront that I didn't dig into what Teams specifically sends over its data channels — treating that part as general WebRTC knowledge rather than something I actually verified myself.
The part that took over my week, though, was recovery. WebRTC has a few ways to recover from lost packets — NACK for a specific missing packet, PLI or FIR to ask for a fresh keyframe, RTX for retransmission over a parallel stream, and congestion feedback (REMB, or the newer TWCC) to tell the sender how much bandwidth is actually available.
Here's what genuinely surprised me: video from Teams' SFU arrives as a short initial burst, and then it goes quiet — and stays quiet unless you keep sending it PLI feedback. I don't actually know for certain whether that very first burst requires a PLI to unlock in the first place; what I can say confidently, because I watched it happen over and over, is that continuing to receive video absolutely requires it. Go without sending a PLI for roughly 11 seconds and the track just stops. Sending one every 2 seconds keeps it alive. It also negotiates congestion control on the side (TWCC), fed on its own separate, faster cadence — a whole rabbit hole I'm still working through myself.
One more thing I noticed: even with a specific codec pinned — say, H264 — recovery packets still show up wrapped in RTX framing rather than as plain retransmitted media. As far as I can tell, RTX just wraps whatever codec you've actually negotiated, rather than being tied to a specific one. I'm still not sure about a couple of things here, though: I haven't pinned down the exact H264 profile values getting negotiated, and I genuinely don't know whether the SFU runs full ICE or ICE-lite — my code handles both, which isn't the same as knowing which one is actually true. Flagging both honestly as things I still need to dig into rather than settled answers.
Full connection-to-media flow, annotated with Teams-specific findings
Compiled Findings
For anyone who just wants the short version:
- Teams' SFU answers audio-only first, then sends a second offer mid-call to add video — not a one-shot negotiation, as far as I've seen every time.
- ICE candidates usually arrive bundled in the answer rather than purely trickled, sometimes with more trickled in afterward.
- The SFU's DTLS role showed up as
activein every session I tested. - Video keeps flowing only with regular PLI feedback — an initial burst, then roughly an 11-second window before it halts without one, and a 2-second cadence to keep it alive. I can't say for sure whether the very first burst needs a PLI to unlock at all.
- Recovery packets stay in RTX framing no matter which codec is pinned, from what I've seen.
- Still digging into: exact H264 profile-level-id values, and whether the SFU runs ICE-lite or full ICE.
Why This Matters
None of this shows up in the spec, and none of it is written down anywhere Microsoft publishes — the only way I found any of it was by building a real client and watching, very closely, what it actually did. And that turned out to matter for more than just curiosity: understanding exactly when to send feedback, at what cadence, and how a specific SFU handles recovery is the groundwork for anything that wants to sit outside the normal browser-to-server path. Environments where the machine rendering the call and the machine best suited to actually process its media aren't the same one — thin clients and remote-desktop setups being a common example — depend on exactly this kind of protocol-level understanding to relocate where media gets handled, without the other end ever needing to know anything changed.
I still have open questions I want to chase down, and I'm sure there's plenty about this stack I haven't even bumped into yet. But that's kind of the fun part.
Top comments (0)