DEV Community: George Hertz

🐋 Incremental (+Parallel) Builds + Manifest Lists = ❤️

George Hertz — Fri, 09 Apr 2021 15:41:18 +0000

This is a cross post of my (not really) blog posts from github

Using buildx to build docker images for foreign architectures separately using qemu and publishing as one multi-arch image to docker hub.

Steps involved in words:

Build image for each architecture and push to temp registry
Create a manifest list grouping them together in the temp registry
Use scopeo to copy from temp registry to public registry

These steps are easier said than done, few things need to happen first.

Example project

Let's image a case where we have a project that runs on docker. We would like to build images for the following
platforms.

linux/amd64
linux/arm64/v8
linux/arm/v7
linux/arm/v6
linux/ppc64le
linux/s390x

The build should happen in parallel for each platform, but only publish one "multi-arch" image (in other words a
manifest list).

Here's a sample app

// app.js
const http = require('http');

const port = 3000;

const server = http.createServer((req, res) => {
    res.statusCode = 200;
    res.setHeader('Content-Type', 'text/plain');
    res.end('Hello World');
});

server.listen(port, () => {
    console.log(`Server running at %j`, server.address());
});

And it's complementing (not very good) Dockerfile

FROM node:14-alpine
RUN apk add --no-cache tini
ENTRYPOINT ["/sbin/tini", "--"]
WORKDIR /app
COPY ./app.js ./app.js
CMD [ "node", "/app/app.js" ]
EXPOSE 3000

Step 1.1: Setup

To perform the first step of we need to set-up a few things:

registry
qemu - to emulate different cpus for building
binfmt
buildx builder that has access to all above

Step 1.1.1: registry

First start a v2 registry and expose as an INSECURE localhost:5000.

docker run --rm --name registry -p 5000:5000 registry:2

Step 1.1.2: qemu, binfmt & buildx

Now setup qemu, binfmt configuration to use that qemu and create a special buildx container which has access to
host network.

sudo apt-get install qemu-user-static

docker run --privileged --rm tonistiigi/binfmt --install all

docker buildx create \
                --name builder \
                --driver docker-container \
                --driver-opt network=host \
                --use

docker buildx inspect builder --bootstrap

The tonistiigi/binfmt --install all is a docker container "with side-effects" that will set up binfmt
configuration on your host.

The --driver-opt network=host will allows the buildx container to reach the registry running on host
at localhost:5000.

The buildx inspect --bootstrap will kickoff the contianer and print it's information for us.

Step 1.2: Build

NOTE: Buildx by itself runs the builds in parallel if you provide a comma separated list of platforms
to buildx build command as --platform flag.

The problem for me and the whole reason of writing this post is that if the build with multiple --platforms
fails for one of the platforms then the whole build is marked as failed and you get nothing.

Another use case can also be that together with one multi-arch image maybe you want to push arch specific repositories (
eg: docker.io/app/app, docker.io/arm64v8/app and docker.io/amd64/app).

The other case is that I do builds on multiple actual machines that natively have arm/v6, arm/v7 and arm64/v8
cpus (a cluster of different Pis and similar).

There are probably even more reasons why you would want to build them this way 🤷.

Now we are ready to start building for different architectures with our buildx builder for this example.

The base alpine image supports the following architectures.

linux/amd64
linux/arm/v6
linux/arm/v7
linux/arm64/v8
linux/ppc64le
linux/s390x

lets target all of them 😎

docker buildx build \
        --tag localhost:5000/app:linux-amd64 \
        --platform linux/amd64 \
        --load \
        --progress plain \
        . > /dev/null 2>&1 &
docker buildx build \
        --tag localhost:5000/app:linux-arm-v6 \
        --platform linux/arm/v6 \
        --load \
        --progress plain \
        .> /dev/null 2>&1 &
docker buildx build \
        --tag localhost:5000/app:linux-arm-v7 \
        --platform linux/arm/v7 \
        --load \
        --progress plain \
        .> /dev/null 2>&1 &
docker buildx build \
        --tag localhost:5000/app:linux-arm64-v8 \
        --platform linux/arm64/v8 \
        --load \
        --progress plain \
        .> /dev/null 2>&1 &
docker buildx build \
        --tag localhost:5000/app:linux-ppc64le \
        --platform linux/ppc64le \
        --load \
        --progress plain \
        .> /dev/null 2>&1 &
docker buildx build \
        --tag localhost:5000/app:linux-s390x \
        --platform linux/s390x \
        --load \
        --progress plain \
        .> /dev/null 2>&1 &
wait

Once this is done, the images will be loaded and visible with docker images command

$ docker images

...
localhost:5000/app   linux-arm64-v8    e3ec56e457e6   About a minute ago   115MB
localhost:5000/app   linux-arm-v7      ab770e5be5d1   About a minute ago   106MB
localhost:5000/app   linux-ppc64le     3a328d516acf   About a minute ago   126MB
localhost:5000/app   linux-s390x       73e064c0c3d4   About a minute ago   119MB
localhost:5000/app   linux-amd64       f6260fedf498   About a minute ago   116MB
localhost:5000/app   linux-arm-v6      5a1fb75b0a45   2 minutes ago        110MB
...

There is no need to --load the images to your local docker, you can make buildx directly push to our local registry.

For this example you have to push these images as an extra step

docker push --all-tags -q localhost:5000/app

Step 2: Manifest List

Now we only need to group those images together into one big manifest list.

docker manifest create --insecure
    localhost:5000/app:1.0.0 \
        localhost:5000/app:linux-amd64 \
        localhost:5000/app:linux-arm-v6 \
        localhost:5000/app:linux-arm-v7 \
        localhost:5000/app:linux-arm64-v8 \
        localhost:5000/app:linux-ppc64le \
        localhost:5000/app:linux-s390x

docker manifest push localhost:5000/app:1.0.0

Step 3.1: Skopeo

The one last step is to copy the manifest list and only the blobs that are linked by it. For this we need skopeo, an
amazing tool for working with registries.

You can either build form source or for ubuntu 20.04 we can use the prebuilt kubic packages.

NOTE: You don't have to install it with this script, just follow the install guide
at https://github.com/containers/skopeo/blob/master/install.md

OS="x$(lsb_release --id -s)_$(lsb_release --release -s)"
echo "deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/${OS}/ /" > /etc/apt/sources.list.d/skopeop.kubic.list
wget -qO- "https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/${OS}/Release.key" | apt-key add -
apt-get update
apt-get install -y skopeo

Now because our local registry is insecure skopeo will complain when we try to copy from it, so we need to explicitly
configure it to allow insecure connections to our temp registry.

[[registry]]
location = 'localhost:5000'
insecure = true

Create this file in /etc/containers/registries.conf.d/localhost-5000.conf

Step 3.2: Copy

The final step is to copy only the localhost:5000/app:1.0.0 to lets say hertzg/example:app-1.0.0.

First you might need to authenticate with your target registry

skopeo login docker.io

Now we can finally copy the image

skopeo copy \
        --all \
        docker://localhost:5000/app:1.0.0 \
        docker://docker.io/hertzg/example:app-1.0.0

This might take some time but once it's finished you can check the docker hub or just pull and run the image on target
architectures

docker run --rm -it hertzg/example:app-1.0.0

Thats it.

References

Cover image from https://laptrinhx.com/multi-arch-all-the-things-1320316701/

Hacking BLE Kitchen Scale

George Hertz — Fri, 04 Sep 2020 16:13:57 +0000

TL;DR: Result

hertzg / metekcity

ETEKCITY smart nutrition scale protocol reverse engneering

Backstory

Recently I have been gaining weight and blaming it all on the COV-19 (jira issue).
So I thought I have to manage my food intake and count calories therefore I did what I do best, procrastinate and try to do other stuff while still thinking about the task at hand.
All of this + Amazon and my interest in IoT somehow convinced me to buy Etekcity Smart Nutrition Food Calorie Kitchen Digital Scale.

Having hard time waking next day after late night impulse buying, I tried the device, played a bit with the (meh) app and realized that this is just an overpriced kitchen scale with app function whose sole reason is data mining. After installing in VeSync app and skipping as much of the registration as possible and playing enough with it, I decided to try and somehow gain control of the device without having to use the (meh) app.

Disclaimer

Before I go into technical details I would like to mention that I have never worked with BLE devices before. Being armed with 0 technical knowledge of Bluetooth Low energy I was (not) equipped with all the knowledge I need and (definitely not) ready to start hacking.

Step 1: Take it apart 🛠

Having dabbled in some other IoT devices (ESP) my first instinct was to disassemble the device and try to find how this thing worked. I was hoping I could find the microcontroller name and model or some debug ports exposed and labeled but I was disappointed to see this.

PCB was labeled here and there but it was not too helpful as they were just "component ids" to pick and place. The communications device had some information for me.

The communications module is for Bluetooth 4 which is something that I can start investigating.

Step 2: Maybe there's a lib for it? 🥺

Next step was to try to somehow find how to communicate to this and maybe someone else has already done some hacking 💔 on this but I was not able to find information for this device 💔 . The one of the projects that was relatable to this was oliexdev/openScale

oliexdev / openScale

Open-source weight and body metrics tracker, with support for Bluetooth scales

openScale

Open-source weight and body metrics tracker, with support for Bluetooth scales

Install openScale-dev-build.apk to get the latest development build generated by GitHub Actions. Also be aware that this version may contain bugs and you don't get any automatic updates.

Summary 📋

Monitor and track your weight, BMI, body fat, body water, muscle and other body metrics in an open source app that:

has an easy to use user interface with graphs,
supports various Bluetooth scales,
doesn't require you to create an account,
can be configured to only show the metrics you care about, and
respects your privacy and lets you decide what to do with your data.

Supported Bluetooth scales 🚀

openScale has built-in support for a number of Bluetooth (BLE or "smart") scales from many manufacturers, e.g. Beurer, Sanitas, Yunmai, Xiaomi, etc. (see model list below). Together with our users we constantly improve and extend the set of…

View on GitHub

But it was only targeted towards body weight scales 💔.

I was also able to find a github issue asking about this particular device and model and it was rejected for obvious reason 💔.

Add Support for the ETEKCITY Bluetooth Scale #509

Dipan61241828 posted on Oct 17, 2019

Hi it's great app. it works like a charm for almost all devices. Thanks for this great creation. recently i bought new weight measurement scale of ETEKCITY and it is not supported by this app.

https://www.etekcity.com/product/100334

here the debug log file attached with your debug app

openScale_2019-10-17_12-57.txt

More debug Log, openScale_2019-10-17_16-04_new.txt

Scale information

Let me know if the above is sufficient or should I need to give more.

Thank you.

View on GitHub

Step 3: Down the rabbit hole 🐰

I love JS and Node.JS and I felt confident (for some weird reason) in worst case scenario I could use some linux tools with child_process or even hack something in C to communicate using BLE (via USB). It was already getting late and I was getting delirious :D .

Now I'm here and I want to be able to at least get the measurements read. I quickly googled up a module for node which was a good start.

noble / noble

A Node.js BLE (Bluetooth Low Energy) central module

A Node.js BLE (Bluetooth Low Energy) central module.

Want to implement a peripheral? Checkout bleno

Note: macOS / Mac OS X, Linux, FreeBSD and Windows are currently the only supported OSes. Other platforms may be developed later on.

Prerequisites

OS X

install Xcode

Linux

Kernel version 3.6 or above
libbluetooth-dev

Ubuntu/Debian/Raspbian

sudo apt-get install bluetooth bluez libbluetooth-dev libudev-dev

Make sure node is on your path, if it's not, some options:

symlink nodejs to node: sudo ln -s /usr/bin/nodejs /usr/bin/node
install Node.js using the NodeSource package

Fedora / Other-RPM based

sudo yum install bluez bluez-libs bluez-libs-devel

Intel Edison

See Configure Intel Edison for Bluetooth LE (Smart) Development

FreeBSD

Make sure you have GNU Make:

sudo pkg install gmake

Disable automatic loading of the default Bluetooth stack by putting no-ubt.conf into /usr/local/etc/devd/no-ubt.conf and restarting devd (sudo service devd restart).

Unload ng_ubt kernel module if already loaded:

sudo kldunload ng_ubt

…

View on GitHub

And starting hacking away and logging output. With some luck and more luck I was able to guess the correct service, characteristic and ended up with some notes where I could start looking at the protocol.

And around 4 am in the morning finished writing the README and finally tired enough to just go to bed and rest.

hertzg / metekcity

ETEKCITY smart nutrition scale protocol reverse engneering

ETEKCITY Smart Nutrition Scale

⚠️ Very much work in progress ⚠️

This is a potential project that tries to reverse engineer the BLE protocol that ETEKCITY Smart Nutrition Scale (ESN00) uses.

ETEKCITY Smart Nutrition Scale (ESN00) (DE | US)

BLE Protocol

This section describes the protocol (what was researched so far)

BLE Services & Characteristics

> Service: 00001910-0000-1000-8000-00805f9b34fb
>> Characteristic: 00002c10-0000-1000-8000-00805f9b34fb [READ]
>> Characteristic: 00002c11-0000-1000-8000-00805f9b34fb [WRITEWITHOUTRESPONSE, WRITE]
>> Characteristic: 00002c12-0000-1000-8000-00805f9b34fb [NOTIFY, INDICATE]
> Service: 0000180a-0000-1000-8000-00805f9b34fb
>> Characteristic: 00002a23-0000-1000-8000-00805f9b34fb [READ]
>> Characteristic: 00002a50-0000-1000-8000-00805f9b34fb [READ]
> Service: 00001800-0000-1000-8000-00805f9b34fb
>> Characteristic: 00002a00-0000-1000-8000-00805f9b34fb [READ]
>> Characteristic: 00002a01-0000-1000-8000-00805f9b34fb [READ]

Communication happens on service 0x1910, device to client communication happens on 0x2c12 characteristic and client to device communication on 0x2c12

Protocol

All packets have this structure

Packet

Structure: Data

Payload structure is defined in esn00-packet README

View on GitHub

Next steps

I would like to write (at least half-) decent library to listen and possibly control the big display with nutritional information from outside the app. For now I need an Android device to sniff the packets and analyze the result.

I actually do not know which device to choose so maybe one late night I will pick a random cheap Android phone and invest more in my procrastination or maybe someone will tell me in comments which one to go for ¯\_(ツ)_/¯ .

The end goal would (probably) be to have it integrate with homebridge or home-assistant and have it comfortably enable the nutritional value display based on voice commands.