The firmware file was sitting in the filesystem. The kernel insisted that it was not. Bluetooth on an i.MX8MQ evaluation board was failing with a -110 error. This meant that the chip (CYW4356) was not able to communicate.
Right after running hciconfig hci0 up there would be messages like setting baudrate failed (-110). After validating that the chip enable was properly being driven, the failures pointed issues in the chip’s communication path.
There are two modes of chip communication: PIO or SDMA. The device tree said that SDMA was being used. Looking closely at the messages indicated that SDMA was not being loaded. The common causes for those on the i.MX8MQ could be that the SDMA channels for Bluetooth were misconfigured or the chip was not working.
dmesg pointed to SDMA accessibility errors with the message We cannot prepare for the TX/RX slave dma!. To narrow this down I commented out the dma and dma-names properties for uart3, aka Bluetooth and voila - Bluetooth was up! However, SDMA firmware loading errors like [2.464348] imx-sdma 302c0000.dma-controller: Direct firmware load for imx/sdma/sdma-imx7d.bin failed with error -2 persisted in dmesg.
Running ls /usr/lib/firmware, confirmed that the bin file was there. It almost looked like the kernel did not want to acknowledge its presence.
The SDMA firmware loader runs when the driver probes. The driver was loaded by default at boot time (CONFIG_IMX_SDMA=y). This meant it was trying to load firmware before the root filesystem was up. The firmware lives in the root filesystem!
You might think why not set the firmware path in the kernel using CONFIG_EXTRA_FIRMWARE. It allows the firmware to be built into vmlinux and avoids the kernel searching for it in the filesystem. This should have worked, but didn’t: darn those yocto dependencies removing it from the kernel build. Faced with the onerous task of threading through all the inc, bb, bbappends, and making multiple builds to determine the root cause, I began to question do we really build the firmware in the kernel. What if, we delayed the SDMA driver load, until something actually needed it: achievable by setting CONFIG_IMX_SDMA=m. By doing this we move the driver initialization out from vmlinux’s init (aka do_initcalls) to it being loaded via modprobe or insmod of a .ko file available on the rootfs. I was betting that as the filesystem exists, the firmware file will be available for the driver to load! Suddenly it was a one line change, just one build. Leading to rebuild, load and success!!
Below is a comparison of what was seen in the logs:
- With
CONFIG_IMX_SDMA=y
[ 2.133882] Bluetooth: HIDP (Human Interface Emulation) ver 1.2
[ 2.139819] Bluetooth: HIDP socket layer initialized
[ 2.464469] imx-sdma 302c0000.dma-controller: Direct firmware load for imx/sdma/sdma-imx7d.bin failed with error -2 << Load at 2.46s
[ 2.483924] imx-sdma 30bd0000.dma-controller: Direct firmware load for imx/sdma/sdma-imx7d.bin failed with error -2
[ 2.500030] imx-sdma 302c0000.dma-controller: Direct firmware load for imx/sdma/sdma-imx7d.bin failed with error -2
[ 2.510503] imx-sdma 302c0000.dma-controller: external firmware not found, using ROM firmware
[ 2.524542] imx-sdma 30bd0000.dma-controller: Direct firmware load for imx/sdma/sdma-imx7d.bin failed with error -2
[ 2.535006] imx-sdma 30bd0000.dma-controller: external firmware not found, using ROM firmware
[ 3.354871] xhci-hcd xhci-hcd.1.auto: xHCI Host Controller
- With
CONFIG_IMX_SDMA=m
root@imx8mq-evk:/etc/systemd/system# modprobe -D imx-sdma
insmod /lib/modules/6.6.52-lts-next-ge0f9e2afd4cf-dirty/kernel/drivers/dma/imx-sdma.ko
root@imx8mq-evk:/etc/systemd/system# dmesg | grep -Ei 'sdma|mounted filesystem|Run /sbin/init'
[ 6.090389] EXT4-fs (mmcblk0p1): mounted filesystem 9cb57070-175f-4761-812a-3656cb092bcd r/w with ordered data mode. Quota mode: none. << Filesystem mounted at 6.09
[ 6.117332] Run /sbin/init as init process
[ 9.328588] imx-sdma 302c0000.dma-controller: firmware found. << load at 9.32s, after FS mount
[ 9.337947] imx-sdma 302c0000.dma-controller: loaded firmware 4.6
[ 9.401308] imx-sdma 30bd0000.dma-controller: firmware found.
Key takeaway: A firmware file being present in the file system does not mean it is available when the driver asks for it. Look out for when the driver tries to load it vs when the filesystem is mounted. As for building the firmware into the kernel: this can wait until something actually requires it.
Next: the same board's WiFi was on a different PCIe controller than the reference device tree claimed.
Top comments (0)