How do you ensure your monitoring measurements are relevant? In Elecard Boro, we've implemented a dual-verification architecture for MPEG-TS.
1. Path #1 (User Space): The packet stream from the network goes through libpcap to the main analyzer. It operates in user space and counts CC (Continuity Counter) errors.
2. Path #2 (Kernel Space): In parallel, at a lower level, data is intercepted via netfilter and sent to a second, independent analyzer that measures MLR (Media Loss Rate).
Why do we need this? If the readings from both "experts" align—great, we can be confident in the results. But if their graphs diverge—that's a powerful signal: "Attention, something's wrong with the measurement process!" This prevents us from jumping to conclusions about network issues.
Scenario 1: MLR and CC Errors Match
Errors appear synchronously on both graphs.
Likely cause: Actual network problems.
What to do:
Request a dump from the content provider.
Check for drops on switching equipment, deploy probes at the delivery network's input and output points to pinpoint where exactly packets are being lost.
Check CPU performance: RSS might not be enabled, causing one CPU core to be overloaded with data loss, or the entire system might be overloaded, preventing the probe from functioning correctly, or the receive buffer might be overflowing.
Accordingly, you can try increasing the incoming buffer, optimizing system performance, or reducing the load.
Scenario 2: CC Errors Present, but MLR is Normal
What to do:
Your operating system buffer may be overflowing or there may be insufficient performance. Try increasing them.
Check whether a broadcast source (e.g.,
ffmpeg) and the probe are running on the same machine. Process competition for CPU resources can cause such effects. The solution to this problem is to use RPS (Receive Packet Steering) to "pin" processes to different cores.
Scenario 3: MLR Errors Present, but CC Counter is Fine
What to do:
Check whether the probe is running in a virtual machine. We don't recommend using virtual machines for multicast monitoring because we cannot guarantee packets will be measured correctly. You can use them for OTT—measurements will be normal there.
Make sure the server isn't overloaded. If all CPU resources are consumed, the analyzer may simply not keep up with processing the stream, leading to false positives.
Conclusion
Reliable monitoring isn't just about having a good tool, but also a properly configured environment. The dual-verification approach built into the Elecard Boro monitoring system helps our clients quickly separate the real network problems from measurement artifacts.
Top comments (0)