Every embedded Linux engineer has hit it: the system that works for weeks, then crashes on a cold night or after a firmware update. Memory faults are the hardest to diagnose because they hide behind generic symptoms — segfaults, random reboots, corrupted filesystems.
This guide walks through the field-tested steps I use to isolate DRAM problems on embedded boards.
1. Reproduce and classify
First, capture the failure. Run the board under load and at temperature extremes if the failure is thermal:
memtester 128M 5
stress-ng --vm 4 --vm-bytes 256M --timeout 600s
A fault that appears only under heat points to refresh issues. A fault at cold startup points to timing margins.
2. Check the logs
Kernel logs often reveal ECC or machine check events:
dmesg | grep -i -E "ecc|machine check|memory error"
journalctl -k -p err
On boards with ECC memory, an increasing count of corrected errors is a leading indicator of marginal DRAM.
3. Verify the part and its temperature grade
The most common root cause is a commercial-grade part in an industrial environment. Check the marking on the chip and the datasheet:
cat /proc/iomem
Confirm the DRAM is rated for the operating range of the deployment site. Industrial parts are screened for -40°C to +85°C; commercial parts are only guaranteed to 0°C to +70°C.
4. Swap test with a known-good module
If the fault follows the module across boards, the memory is the culprit. Replace with a wide-temperature industrial module and re-run the stress tests.
Summary
Classify the failure, check the kernel logs, verify the temperature grade, and swap-test. Most embedded memory failures trace back to a spec-sheet decision, not a manufacturing defect.
This guide is written by an application engineer at Loongtion, a manufacturer of wide-temperature industrial memory (DRAM DDR3/DDR4, eMMC, industrial SSDs). Datasheets and selection support: loongtion.com
Top comments (0)