DEV Community

potterwhite
potterwhite

Posted on • Originally published at potterwhite.github.io

How I Automated Multi-Platform Docker Image Builds for Embedded SoCs (RK3588, RV1126, RK3568)

#docker #embedded #linux #devops

How I Automated Multi-Platform Docker Image Builds for Embedded SoCs

If you work with embedded Linux boards — RK3588, RV1126, RK3568, or similar ARM SoCs — you've probably hit these problems:

  • Five different Dockerfiles, diverging more with every Ubuntu release
  • Port collisions when running containers for multiple platforms at the same time
  • Ubuntu 24.04 broke your apt mirror setup, pip installs, and UID assignments all at once
  • Image push to Harbor requires glue scripts nobody maintains

I spent months dealing with this and eventually built a tool called HarborPilot to solve it. Here's what I learned.

The Core Problems

Problem 1: Multiple Dockerfiles That Drift Apart

When you start, one Dockerfile per platform seems fine. After 6 months, the RK3588 file has fixes that never made it to the RK3568 file, and Ubuntu 24.04 support requires changes that would break 20.04.

Solution: One monolithic 5-stage Dockerfile. Platform-specific behaviour is injected via ARG/ENV at build time.

Stage 1: base OS (apt sources, packages, user creation)
Stage 2: dev tools (cmake, gdb, CUDA, OpenCV, Node.js)
Stage 3: SDK init (git, symlinks, helper scripts)
Stage 4: env config (proxy, profile variables)
Stage 5: workspace + entrypoint + smoke tests
Enter fullscreen mode Exit fullscreen mode

Problem 2: Port Collisions

When you run Docker containers for RK3588 and RK3568 simultaneously, they'll both try to use port 2109 for SSH. You either document this manually (error-prone) or automate it.

Solution: PORT_SLOT — a single integer that derives all port mappings:

CLIENT_SSH_PORT = 2109 + PORT_SLOT × 10
GDB_PORT = 2345 + PORT_SLOT × 10
Enter fullscreen mode Exit fullscreen mode

Set PORT_SLOT=0 for RK3588, PORT_SLOT=2 for RK3568. Done. Zero conflicts, no documentation needed.

Problem 3: Supporting Three Ubuntu Versions Without Breaking Any

Ubuntu 24.04 has three breaking changes compared to 20.04/22.04:

  1. DEB822 apt format — /etc/apt/sources.list is deprecated
  2. UID 1000 pre-occupied — the default ubuntu user already has it
  3. PEP 668 — pip install now refuses without --break-system-packages

All three are handled with OS_VERSION conditionals in the relevant scripts.

The Three-Layer Config System

The key insight: instead of platform-specific Dockerfiles, use a config inheritance system.

Layer 1: configs/defaults/*.env    ← global defaults (10 domain-scoped files)
Layer 2: configs/common.env        ← project version & constants
Layer 3: configs/platforms/*.env   ← per-platform overrides (≤20 lines)
Enter fullscreen mode Exit fullscreen mode

A real platform file for RK3568 on Ubuntu 20.04:

# configs/platforms/rk3568-rk3568_ubuntu-20.04.env
PRODUCT_NAME="rk3568-rk3568_ubuntu-20-04"
CHIP_FAMILY="rk3568"
OS_VERSION="20.04"
OS_VERSION_ID="20-04"
PORT_SLOT=2
HOST_VOLUME_DIR="/mnt/disk/volumes/rk3568"
HAS_PROXY=true
HTTP_PROXY_IP="192.168.3.152"
Enter fullscreen mode Exit fullscreen mode

That's the complete file. Everything else inherits from Layer 1 defaults.

Adding a new platform takes 15-20 lines. The wizard handles PORT_SLOT collision detection automatically.

One Command to Build, One to Run

Build a platform image:

./harbor
# Interactive: pick platform → build → tag → push to Harbor → verify manifest digest
Enter fullscreen mode Exit fullscreen mode

Start dev container on any Ubuntu host:

./ubuntu_only_entrance.sh start
Enter fullscreen mode Exit fullscreen mode

Create a new platform non-interactively (CI-friendly):

./scripts/create_platform.sh --non-interactive \
    --name rk3566-debian12 --os debian --os-version 12 \
    --harbor-ip 192.168.3.68 --port-slot 6
Enter fullscreen mode Exit fullscreen mode

What I Wish I'd Done Earlier

The envsubst switch was overdue. I originally used sed for template rendering in the Dockerfile stages — brittle, hard to debug. Replacing it with envsubst (from gettext-base) eliminated a whole class of escaping bugs.

The modular ubuntu_only_entrance.sh was also worth the refactor. The original was a 400-line monolith. Splitting it into 6 numbered modules made debugging 10x faster.

Links

Feedback welcome — especially if you're using a different Rockchip or Allwinner SoC.

Top comments (0)