First experiences with Honeycomb LX2K

I last wrote about arm64 two months ago when I was excited for my Honeycomb LX2K board.

Well, it's arrived! and it's glorious! Everything is just working, after some config tweaks and patching. I can stay connected with my team via Slack, write code, and deploy to Graviton2 in the cloud without touching an amd64 machine.

Here are the photos of the board and its installed setup inside the chassis!

Here's the missing manual, written on the very same Honeycomb LX2K arm64 board I did this setup on!

Initial setup

• Set up the hardware
  • Insert the NVMe boot drive
  • Insert RAM sticks
  • Plug in the PicoPSU
  • Plug in the GPU
  • Attach a low-noise fan adapter inline with the CPU fan to avoid the jet engine sounds (PWM isn't working yet).
  • Attach the case button(s) to the power and reset jumpers; reset is more important than power.
  • Do not yet attach a monitor or mouse/keyboard.
• Flash the latest UEFI firmware to a micro SD card
  • Untar the image folder
  • dd if=lx2160acex7_2200_700_2400_8_5_2.img of=/dev/sda where sda is the SD card (make sure it's not mounted, and really is your SD card and not some other load-bearing device)
  • Plug the SD card into the board. Make sure the firmware source jumpers are set off-ON-ON-ON-ON-off (or otherwise as indicated by silkscreen on board).
• Boot for the first time.
  • Plug an Ubuntu ARM64 20.04 USB stick into the board (created with Startup Disk Creator & the latest 20.04 ARM image)
  • Plug in an ethernet cable via USB adapter (the ports on the board won't work yet)
  • Plug in the micro-USB serial console port to an existing machine, and run minicom configured to talk to /dev/ttyUSB0 with hardware/software buffering OFF (very important!)
  • Power on the device by inserting the power cable or turning on the PSU
• Install Ubuntu 20.04
  • If Ubuntu installer GRUB menu doesn't show, hit esc during boot and manually select the USB stick as the boot device.
  • At GRUB menu, select install rather than try live
  • You'll need to update the installer snap when prompted, as the base ISO doesn't install on ARM64 properly
  • Make sure you enable SSH, so you can do the next bit of work comfortably from a desktop environment
  • Disk encryption just works.
  • Reboot when prompted; if it doesn't cleanly shut down, kick the reset button after the prompt to remove the USB stick.
• Boot the second time.
  • Enter the FDE password over serial console
  • Once system is booted, ssh to it from your desktop to make sure everything works.

Prepare to enable graphical mode

• Configure AMD PCIe settings: set GRUB_CMDLINE_LINUX_DEFAULT="amdgpu.pcie_gen_cap=0x00040004" in /etc/default/grub then run sudo update-grub
• Install the PCIe bug patched kernel image PPA: https://launchpad.net/~lizthegrey/+archive/ubuntu/kernel and install the linux-image-unsigned-5.4.0-42-generic package.
• Now, reboot, with display and mouse/keyboard attached. If it doesn't work, you can disconnect display and fall back to serial console to debug (and ask for help in SolidRun Discord)

Enabling graphical mode

• Log in after rebooting (verifying the new kernel is loaded)
• sudo apt install ubuntu-desktop
• mkdir -p ~/.config/environment.d && echo 'export MESA_EXTENSION_OVERRIDE=-GL_ARB_buffer_storage' > ~/.config/environment.d/00-mesa.conf
• If needed, run sudo service gdm3 restart and/or sudo systemctl isolate graphical.target. GDM should be showing. If not, reboot.
• You will want to log in with wayland, not with the default Xorg; click the gear icon in the lower right hand corner to use Ubuntu with Wayland after clicking your name, before entering your password.
• Congratulations, you have a working Gnome3 desktop.

Helpful things to install

• First off, you probably want a web browser. sudo apt install chromium-browser will give you a chromium updated with snap, supported by Ubuntu, for arm64, if you don't like Firefox.
  • sudo update-alternatives --install /usr/bin/x-www-browser x-www-browser /snap/bin/chromium 200 will make it your default.
  • Yes, security keys work out of the box!
  • You can get graphics acceleration by running snap refresh chromium --channel=candidate/vaapi
• Everything else should just work to taste when running apt install -- e.g. golang-1.14, openjdk, etc.

Comments

[Val Packett]

Dang, even v2 silicon needs OS quirks for PCIe still. It's ridiculous that to this day, the Armada8k (macchiatobin) has the least worst PCIe problem, among the affordable devices at least (Ampere eMAG is pretty much fully everything compliant but that's $thousands).

Also it's funny that their board is called the same as your company :D

[Jon Nettleton]

The issue with PCIe is not a quirk, but a bug in the PCIe code that has been in the kernel since 2017. Yes we are exposing the bus in a non-standard way to work around silicon limitations but the bug we exposed is due to the code not being written to the PCIe spec, not what we are doing. The only consumer of the function that needs to be patched is the amdgpu driver. This is made worse by the amdgpu driver not respecting the ACPI tables telling it not to re-assign those resources.

[Val Packett]

Huh, cool. I guess this bug might not be present in other operating systems.

[Jon Nettleton]

Most likely not. The code was an assumption you wouldn't re-assign a bar on the device sitting on the root of the bus. Even though in our implementation this node is not actually the root controller, that assumption is still invalid since root controllers can have their own bars.

[Val Packett]

Do operating systems need to handle the NXP0016 device?

NetBSD does: github.com/NetBSD/src/commit/1a0fb...

but, hm, github doesn't find "NXP0016" in Linux.

[Blake Hartshorn]

Thanks for reporting on this. I'm curious, when you built the UEFI image, did you check out a specific branch or have to do any troubleshooting otherwise? I was having a different set of problems every way I tried it, but your image booted right up.

[uh13719]

Ah ... The UEFI branch has been removed from SolidRun's github.

Out of sheer luck I seemed to have unintentionally downloaded the UEFI branch(lsdk-19.09-sr-uefi) when it was still available back in January. (I don't know the exact date when they removed it)

I was reading somewhere that they might re-release a UEFI branch in a few weeks time.

Having said that, I was shocked when the UEFI build generated an image straight away and the generated image booted into GRUB menu right away. My problems happen after this point.

As mentioned in this article, I only built the boot image using that UEFI branch. The complete/linux build fails on the UEFI branch I have.

I am not sure if this helps.

What issues are you facing by the way ?

[Blake Hartshorn]

I found another user on github who had actually cloned that repo and still had the uefi branch in it, so that's a thing I tried, but the final image didn't seem to do anything. Liz's image boots, but I get memory errors trying to boot any kernel, which I'm thinking has to do with the ram speed that image was built for not matching mine (operating under an u-boot image I built myself is fine). I've spent the last few days hacking my way through the runme script and pulling different branches of things to mixed results, both in and out of docker. Closest I got on the master branch was a 20.04 UEFI loader that starts to boot up and then panics.

Anyway I'm still trying. Lol. I saw on Jon's Twitter new features coming down in it, so it has to be working somewhere.

[Blake Hartshorn]

Actually this must be why:
SoC: LX2160ACE Rev2.0 (0x87360020)

I notice in the latest runme.sh it implies that 19.09 is only for rev1. The image provided here is for 20.04 though, so still very curious.

[uh13719]

So if I understood the directions correctly, if you have managed to get your hands on a brach with UEFI in it, you use that branch to build the BOOTLOADER_ONLY image.

This can be done by either setting the parameter BOOTLOADER_ONLY inside the runme.sh script or passed as a parameter: BOOTLOADER_ONLY=yes ./runme.sh

You will also need to set the DDR_SPEED.

The purpose of this image is only to boot the system with UEFI and not provide a working userspace filesystem (which is the default settings that uses u-boot)

If the image generation goes through, you should be able to boot the board into UEFI shell at the very least.

If you have a ubuntu-server.iso on a USB, it should take you upto the grub2 boot menu. where you chose to install/try ubuntu-server.

[Blake Hartshorn]

I probably should have chimed in that I found out what's going on.

The UEFI image here (which I've found out does work and I'm just having some weird kernel problems) is being passed around by the community at the moment while they get the new branch ready. There's a Discord chat you can get invited to if you reach out to SolidRun's support. Images built with the old UEFI branch don't work with the rev2 board. Currently I can get Fedora to boot on it and working on getting Debian Bullseye to boot.

[uh13719]

Hey Liz,

Great article. But I'm struggling to install this on my HoneyComb LX2K. I'm hoping you or someone on this forum can help.

I'm able to boot to GRUB menu and it looks like the Kernel boots successfully into the ubuntu-installer. But I can't proceed any further because the installer menu/GUI does not display properly on the serial console.

I've tried a bunch of different console applications like screen and minicom but the issue remains the same.

Do you have any suggestions to bypass the installation menu or fix the garbled menu display.

P.S: My LX2160A silicon version is 1.4