Zero packet loss, 35 freezes: a screen-sharing performance case study

Zero packet loss. 0–1 ms round-trip time. A hardware encoder that dropped nothing. And a viewer that froze 35 times, bottoming out at 3 frames per second.

That was the result of an end-to-end measurement of a macOS screen-sharing stack. The network was not the problem — and also not irrelevant. Understanding how both are true changes what you instrument and what you optimize.

The setup and the symptom

One Mac sharing a 3024×1964 window, H.264 through the hardware encoder, sender and viewer on the same subnet. Over 501 samples, RTT never left the 0–1 ms band and packet loss stayed at 0%. The encoder reported zero dropped frames.

The viewer told a different story: 35 cumulative freezes. Frame rate had a median of 29 fps, a minimum of 3, and a maximum of 48 — including a two-second stretch that delivered three frames. Single-frame encode time was 12.3 ms, roughly a third of a core at 29 fps. Sender CPU averaged 27.9% (41.2% peak); viewer CPU averaged 20.4%. Resident memory stayed inside a stable band on both sides, with no monotonic growth. The bandwidth limit queried from the capture side came back none in all 501 samples.

So: a perfect channel, an idle-looking endpoint, and a picture that periodically stops.

How to measure "stutter"

The word stutter is useless until you define it. I split observation into three planes and required all three to share one timeline.

1. Process and resources. CPU mean and peak, RSS range, and whether memory trends upward. This plane exists to kill the most over-used explanation first: a leak. There was no leak.
2. Capture and encode. Frame rate as a distribution — median, minimum, maximum — not an average. Per-frame encode time, dropped-frame count, capture queue behavior. An average of 29 fps with a minimum of 3 is a localized-stall signature that the mean erases.
3. Transport. RTT, loss, the actual ICE candidate pair type, DTLS-SRTP completion time, and the limit value asked of the capture side at intervals — "what am I being constrained to right now?"

Then the definition that matters: a freeze means the viewer's picture did not update for N consecutive frames. It does not mean a packet was lost. Capture stalls, encoder queueing, renderer drops, and jitter-buffer underruns all land in one bucket. That is why the first reaction to a freeze count should be "check the pipeline," not "check the network."

Three counterintuitive findings

1. A good network is exactly why adaptation never engages

Five hundred and one samples. Five hundred and one limit=none answers.

A bandwidth estimator assumes bandwidth is scarce. At 0–1 ms RTT and 0% loss, it concludes that available capacity is far above the current bitrate and reports no constraint. That is not a bug; it is the design. Adaptation is bandwidth-driven, not quality-driven. Its inputs are channel observations; its outputs can only constrain encoder parameters. When the bottleneck is not in the channel, the controller has no error signal.

The paradox follows: the better the network and the busier the endpoint — same-subnet casting, LAN collaboration, local demos — the less likely adaptation is to act, because the bottleneck sits in capture and encode queueing, upstream of the channel.

It also explains "all green in the lab, users still report stutter": a test matrix that covers only weak networks makes adaptation look excellent, and never covers strong network plus endpoint bottleneck — not a rare corner case.

2. Thirty-five freezes and 0% loss describe the same system

Zero loss proves packets arrived. It does not prove they arrived on time and in a group, and it does not prove the decoder can put them on screen.

Three stages produce "every packet arrived and the picture still will not move." First, capture does not emit evenly: 3 to 48 fps is a 16× spread, because screen content alternates between still and scrolling. A fixed-rate capture piles up queue depth during scrolls. Second, encode time is fixed while queue length is not. 12.3 ms per frame gives a theoretical ceiling near 81 fps, but it never varies with content while demand spikes when content changes. Demand up, throughput flat: queueing is the only possible outcome. Third, a jitter buffer is built for arrival-time jitter, not for the sender simply stopping. If the sender stalls for 200 ms, the buffer underruns and the player freezes while every network counter stays perfect.

Those 35 freezes were not 35 network incidents. They were 35 pipeline-rhythm incidents.

3. The problem was not the degradation order — it was that nobody made the choice

For screen sharing, the right degradation order is frame rate first, resolution second: text and tables depend on resolution for legibility, not frame rate.

But nothing degraded in that order. Resolution stayed at 3024×1964 for the entire run, encode time stayed at 12.3 ms, and limit stayed at none. Three frames per second was a passive outcome, not an active decision: resolution survived because nothing touched it, not because something protected it.

The trap is conflating a preference with a constraint. A preference flag says "if you must choose, keep resolution." It neither prevents frame rate from falling to 3 nor reduces capture load. Expecting a preference to act as a guardrail is expecting a recommendation to enforce itself.

Eight fixes that ship

1. Make capture parameters explicit. Cap width and height, set minimumFrameInterval to 1/15–1/20 s, and keep queueDepth around 3–5. Native resolution at 3 fps buys readability nothing and costs pipeline pressure.
2. Skip unchanged frames. Combine contentHint with frame-difference detection and send only 1–5 fps heartbeat frames while the screen is still. This usually moves freeze counts more than encoder tuning, because it removes uneven frame spacing at the source.
3. Make the frame-rate floor a separate constraint. "Maintain resolution" is a preference. Add an independent never-below-10-fps guardrail that lowers capture resolution when triggered instead of letting frames drop.
4. Evaluate HEVC and screen-content coding. Hardware HEVC saves bitrate at equal subjective quality, and screen-content extensions are worth testing. Keep per-frame encode time in monitoring; it is the pipeline's throughput ceiling and does not move with frame rate.
5. Report receiver-side metrics separately. Freeze count alone is not enough: report duration distribution, and distinguish "no new frame arrived" from "a frame arrived but was never rendered." Users call both stutter; they are different chains.
6. Test the TURN path explicitly. A 0–1 ms direct-path result does not generalize to a relay. Force the relay candidate and rerun the suite.
7. Look at audio alongside video. If the microphone is open, echo and noise are perceived before video problems. Enable AEC and noise suppression conditionally on the mic, not globally disabled to save CPU.
8. Shrink the black-screen window at join time. Connect and first-frame time leave a gap where the user stares at nothing. Send a low-resolution keyframe to light up the picture, then step quality up. Users tolerate blurry-but-immediate better than sharp-but-late.

Measurement beats optimization

The most valuable part was not the eight fixes, but that three individually normal readings only exposed the problem when read together:

• 27.9% CPU says the endpoint is idle.
• 0% loss and 0–1 ms RTT say the network is great.
• 501 of 501 limit=none says adaptation is working.

Stack all three and "35 freezes, minimum 3 fps" becomes visible.

Screen-sharing stutter hides in the frame-rate distribution, not the mean. It hides in frame spacing, not loss. It hides in negative evidence — the constraint that was never set. Get the measurement right first; optimization is one-time, measurement compounds.

I also built a free calculator for this: Screen Sharing Latency & Bandwidth Calculator — plug in resolution, fps, codec and bandwidth, and it shows where your latency budget breaks.

If you want the full checklist and scripts, here's the pack ($9).

Disclosure: I sell a paid pack that expands on this case study.