Minwook Je

Posted on Sep 10

Edge Device OTA with real world example

#kubedge #kubernetes #iot #ota

reference

edge computing and IoT
KubeEdge's Over-the-Air (OTA) feature

AS-IS

Traditional cloud computing assumes stable, high-bandwidth connections.
But in reality connectivity may be intermittent
Standard Kubernetes is not edge-ready by default. It assumes persistent cloud connectivity.

KubeEdge was designed to address these gaps by splitting Kubernetes into two logical planes:

CloudCore(center): runs in the cloud
- Manages policies for resources
- Manages Device models
- OTA
- Orchestrate workload
EdgeCore: runs on the edge device
- Auto execute workloads
- Manages HW states
- Persist operation even if the cloud connection is lost

OTA

In real-world edge deployments, the cost of physically updating devices is prohibitive. Imagine dispatching technicians to update 10,000 industrial sensors across factories or retail kiosks across 500 stores. Without OTA, such operations are not feasible.

KubeEdge

Full Architecture

┌────────────────────────────── Cloud (Datacenter/Cloud) ──────────────────────────────┐
│                                                                                     │
│  ┌──────────────┐     ┌───────────────┐                                              │
│  │ Kubernetes   │     │  CloudCore    │                                              │
│  │ API Server   │◀───▶│  (cloudhub,   │                                              │
│  └──────────────┘     │   controller) │                                              │
│         ▲              └───────────────┘                                              │
│         │                   ▲                                                         │
│  kubectl/CI/CD             │ CRDs(Operations, Device, ...), Deployments               │
│         │                   │                                                         │
└─────────┼───────────────────┼─────────────────────────────────────────────────────────┘
          │                   │  (MQTT/WebSocket over TLS, edge↔cloud sync)
          ▼                   ▼
┌──────────────────────────────── Edge Site (매장/공장 등) ─────────────────────────────┐
│  ┌──────────────────────── Edge Node (Jetson/Industrial PC/RPi) ────────────────────┐│
│  │                                                                                │ │
│  │  ┌───────────────┐   ┌──────────────┐   ┌───────────────┐  ┌─────────────────┐ │
│  │  │ EdgeCore      │   │ containerd   │   │   Pods        │  │  DeviceTwin     │ │
│  │  │ (edgehub,     │   │ (CRI runtime)│   │ (앱 컨테이너) │  │  (디바이스 그림자)│ │
│  │  │  edged,       │   └──────────────┘   │  grillx-ctrl  │  └─────────────────┘ │
│  │  │  eventbus,    │        ▲             │  vision-svc   │          ▲            │
│  │  │  devicetwin)  │        │             │  ...          │          │            │
│  │  └───────────────┘        │             └───────────────┘          │            │
│  │        ▲                   │         (이미지 Pull/Run)              │            │
│  │        │                   │                                        │            │
│  │  CloudCore와 동기화  ◀─────┘                                  센서/MCU 상태 동기화 │
│  └──────────────────────────────────────────────────────────────────────┬────────────┘│
│                                                                         │             │
│                  ┌─────────────── Peripheral Devices ────────────────┐  │             │
│                  │  Sensors (temp, gas), actuators, Arduino/MCU     │◀─┘             │
│                  │  (USB/UART/CAN/MQTT)                             │                │
│                  └───────────────────────────────────────────────────┘                │
└───────────────────────────────────────────────────────────────────────────────────────┘

Deploy & OTA

[Dev Push]
   │
   ▼
[CI/CD: build & push image]
   │           (Harbor/ECR/GCR)
   ▼
[kubectl apply / set image]
   │  (Deployment/DaemonSet)
   ▼
[Kubernetes API Server] ──▶ [CloudCore] ──▶ [EdgeCore] ──▶ [containerd] ──▶ [Pods 교체]
                                             ▲               ▲
                                             │ 레이어 Pull   │ 헬스체크/롤백
                                             └───────────────┘

NodeUpgradeJob

[kubectl apply -f NodeUpgradeJob.yaml]
   │
   ▼
[CloudCore] ──▶ [EdgeCore] ──▶ (EdgeCore/런타임/드라이버 등 교체 스크립트 실행)
                                     │
                                     └─ 재시작/검증/결과 보고(status)

TO-BE

With KubeEdge

KubeEdge addresses this with a multi-layered OTA mechanism.

1. Containerized Application OTA

Applications deployed as k8s workloads
Declarative manifests (k8s)
Rolling updates by k8s mechanism

2. Firmware OTA

Manage Firmware Device Custom Resource Definitions
Desired firmware version by Device spec
EdgeCore or Custom mapper pulls binaries and validates checksums, and updates the status back to CloudCore

3. Node OTA (EdgeCore / Runtime upgrade)

For upgrade Meta system itself like EdgeCore, Runtime(CRI-O, Docker ..), Edge Runtime itself (GPU driver, TensorRT, Jetson version..)
NodeUpgradeJob (introduced in v1.19+)

Case Study 1

Smart Retail Kiosk Fleet

A retail enterprise deployed 5,000 smart kiosks nationwide. Each kiosk included:

- Containerized applications for payment, ads display, and inventory sync.
- Firmware-controlled peripherals (barcode scanner, receipt printer).
- An edge runtime running KubeEdge EdgeCore.

Challenges

Connectivity: Network frequent dropouts.
Heterogeneity : Devices had different firmware (multi vendors).
Security: Regulatory requirement that every update must be signed and auditable.

Solution by KubeEdge OTA

DeviceModel (firmware version, display resolution, payment module version)
Deployment (COntainer images)
- CloudCore handled sync and EdgeCore perform rolling updates locally
OTA Job resource with HTTPS OTA server
- EdgeCore validate checksum(data transmission)
- Update Firmware via serial flashing.
- patched the Device CRD status -> new version

DEV Community