Electronics engineer. 25 years making different systems work together under real conditions: devices, transactions, backend, data, security and operations. Now, edge AI on real hardware.
Location
Ciudad de México
Education
B.Eng. Electronics and Communications, Universidad Iberoamericana
Thanks for putting the p99 next to the per-chunk latency; that's the number I look for first. Was it measured over a long always-on run? That's where temperature catches up. On a Pi 5 in my lab, a 30-minute sustained inference run hit the soft temperature limit at minute 15 (vcgencmd get_throttled flagged it), and yet latency only moved from 127.9 to 128.8 ms. The flag and the slowdown weren't the same thing, so now I log both.
A Pi Zero 2 W in a small case running 24/7 may behave differently. Mine idles at 44–47 °C with no throttle flags, but I haven't put a sustained load on it yet. Have you logged get_throttled alongside the RTF during a long run?
Good catch on the flag/throttle decoupling. Honest answer: no, I didn't log get_throttled alongside the RTF in the article. The 32.2% figure comes from our benchmark run, likely short duration not sustained. I don't have duration metadata for that specific number.
For sustained always-on, your approach of logging both signals is smarter. On the Pi 5 you tested, the flag firing at minute 15 with latency barely moving sounds like soft-limit scaling at 80 C (bit 3), which applies mild clock reduction before the hard throttle at 85 C where latency actually moves. On Pi Zero 2 W you don't have a stock heatsink, the PCB is much smaller with less copper heatspreading, and RAM is PoP-stacked on top of the SoC trapping heat. A small sealed case also kills convection. All of that likely narrows the gap between flag and actual latency hit vs what you see on Pi 5.
If you run a long sustained test on your Pi Zero 2 W, would love to see the data. 32.2% RTF looks like a lot of headroom on paper, but headroom disappears quickly without airflow.
Electronics engineer. 25 years making different systems work together under real conditions: devices, transactions, backend, data, security and operations. Now, edge AI on real hardware.
Location
Ciudad de México
Education
B.Eng. Electronics and Communications, Universidad Iberoamericana
Thanks for the straight answer; a short run is still a useful number as long as it says it's short.
The Zero 2 W details make sense: no heatsink, PoP RAM on top of the SoC and a sealed case all push the throttle point closer. I'll run a sustained test on mine, logging get_throttled, temperature and RTF together, and post the numbers here.
Looking forward to seeing those numbers. If possible, decoding the bitmask into specific bits (bit 2 hard throttle, bit 3 soft limit) rather than just checking non-zero would help separate the two kinds of events. That distinction is where your Pi 5 observation came from.
Electronics engineer. 25 years making different systems work together under real conditions: devices, transactions, backend, data, security and operations. Now, edge AI on real hardware.
Location
Ciudad de México
Education
B.Eng. Electronics and Communications, Universidad Iberoamericana
Will do. I'll log the raw value and decode it in two halves: bits 0–3 for what is happening at the moment of the sample, and bits 16–19 for what has happened since boot. A sampler that reads every few seconds can miss a short throttle event; the sticky bits can't, so a run won't come back clean just because the sample landed at the wrong moment.
Thanks for putting the p99 next to the per-chunk latency; that's the number I look for first. Was it measured over a long always-on run? That's where temperature catches up. On a Pi 5 in my lab, a 30-minute sustained inference run hit the soft temperature limit at minute 15 (vcgencmd get_throttled flagged it), and yet latency only moved from 127.9 to 128.8 ms. The flag and the slowdown weren't the same thing, so now I log both.
A Pi Zero 2 W in a small case running 24/7 may behave differently. Mine idles at 44–47 °C with no throttle flags, but I haven't put a sustained load on it yet. Have you logged get_throttled alongside the RTF during a long run?
Good catch on the flag/throttle decoupling. Honest answer: no, I didn't log get_throttled alongside the RTF in the article. The 32.2% figure comes from our benchmark run, likely short duration not sustained. I don't have duration metadata for that specific number.
For sustained always-on, your approach of logging both signals is smarter. On the Pi 5 you tested, the flag firing at minute 15 with latency barely moving sounds like soft-limit scaling at 80 C (bit 3), which applies mild clock reduction before the hard throttle at 85 C where latency actually moves. On Pi Zero 2 W you don't have a stock heatsink, the PCB is much smaller with less copper heatspreading, and RAM is PoP-stacked on top of the SoC trapping heat. A small sealed case also kills convection. All of that likely narrows the gap between flag and actual latency hit vs what you see on Pi 5.
If you run a long sustained test on your Pi Zero 2 W, would love to see the data. 32.2% RTF looks like a lot of headroom on paper, but headroom disappears quickly without airflow.
Thanks for the straight answer; a short run is still a useful number as long as it says it's short.
The Zero 2 W details make sense: no heatsink, PoP RAM on top of the SoC and a sealed case all push the throttle point closer. I'll run a sustained test on mine, logging get_throttled, temperature and RTF together, and post the numbers here.
Looking forward to seeing those numbers. If possible, decoding the bitmask into specific bits (bit 2 hard throttle, bit 3 soft limit) rather than just checking non-zero would help separate the two kinds of events. That distinction is where your Pi 5 observation came from.
Will do. I'll log the raw value and decode it in two halves: bits 0–3 for what is happening at the moment of the sample, and bits 16–19 for what has happened since boot. A sampler that reads every few seconds can miss a short throttle event; the sticky bits can't, so a run won't come back clean just because the sample landed at the wrong moment.
The sticky-bit split is better than what I suggested. Looking forward to the numbers.