Wiring Android's CameraX to a Quantized Pose Estimation Model for Real-Time Biomechanics

#programming #webdev

---
title: "CameraX + MoveNet Thunder INT8: Sub-25ms Android Pose Estimation"
published: true
description: "Wire CameraX ImageAnalysis to MoveNet Thunder INT8 with GPU delegate for sub-25ms Android pose estimation, NNAPI fallback, and keypoint thresholding."
tags: [kotlin, android, mobile, performance]
canonical_url: https://mvpfactory.co/blog/camerax-movenet-thunder-int8-pose-estimation
---

What We Are Building

A production-ready pose estimation pipeline that runs MoveNet Thunder INT8 under 25ms on mid-range Android hardware. By the end of this tutorial you will have a CameraX ImageAnalysis setup wired to a TFLite interpreter, GPU delegate with NNAPI fallback, a zero-allocation preprocessing path, and confidence-thresholded keypoints that your downstream biomechanics layer can trust.

Prerequisites

Android project targeting API 24+
TFLite runtime and GPU delegate dependencies in your build.gradle
MoveNet Thunder INT8 .tflite model in assets/
CameraX 1.3.x or later

Step 1 — Set Up CameraX ImageAnalysis

Here is the minimal setup to get this working. Bind ImageAnalysis with STRATEGY_KEEP_ONLY_LATEST — drop frames, never queue them. For biomechanics you want the freshest keypoints, not a backlog.

val imageAnalysis = ImageAnalysis.Builder()
    .setTargetResolution(Size(256, 256))
    .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
    .setOutputImageFormat(ImageAnalysis.OUTPUT_IMAGE_FORMAT_RGBA_8888)
    .build()

imageAnalysis.setAnalyzer(cameraExecutor) { imageProxy ->
    runInference(imageProxy)
    imageProxy.close()
}

OUTPUT_IMAGE_FORMAT_RGBA_8888 is the flag that actually matters here. CameraX handles YUV→RGBA conversion in native code on a hardware-accelerated path. Let me show you the pattern I use in every project: never let an ImageProxy arrive as YUV and convert it in Kotlin — that path is 5–8x slower.

Step 2 — Select the Right Delegate

val gpuDelegate = try {
    GpuDelegate(GpuDelegate.Options().apply {
        inferencePreference = INFERENCE_PREFERENCE_SUSTAINED_SPEED
        precisionLossAllowed = true
    }).also { delegates.add(it) }
} catch (e: Exception) {
    null
}

val options = Interpreter.Options().apply {
    if (gpuDelegate != null) {
        addDelegate(gpuDelegate)
    } else {
        addDelegate(NnApiDelegate())
    }
    numThreads = 2
}

precisionLossAllowed = true permits FP16 intermediate computations, which is the native precision of most mobile GPUs. NNAPI fallback catches devices where GPU delegate initialization fails — common on older Mali GPUs with driver issues.

Step 3 — Allocate Your Input Buffer Once

MoveNet Thunder expects [1, 256, 256, 3] INT8 input with values in [0, 255]. Allocate your ByteBuffer at class level — never inside the analysis loop.

private val inputBuffer = ByteBuffer.allocateDirect(1 * 256 * 256 * 3).apply {
    order(ByteOrder.nativeOrder())
}

fun preprocessFrame(bitmap: Bitmap, buffer: ByteBuffer): ByteBuffer {
    buffer.rewind()
    val pixels = IntArray(256 * 256)
    bitmap.getPixels(pixels, 0, 256, 0, 0, 256, 256)
    for (pixel in pixels) {
        buffer.put(((pixel shr 16) and 0xFF).toByte())
        buffer.put(((pixel shr 8) and 0xFF).toByte())
        buffer.put((pixel and 0xFF).toByte())
    }
    return buffer.rewind() as ByteBuffer
}

Step 4 — Parse and Threshold Keypoints

MoveNet outputs 17 keypoints as [y, x, confidence] triples. For biomechanics, 0.3 is your baseline confidence threshold.

data class Keypoint(val y: Float, val x: Float, val confidence: Float)

fun parseKeypoints(output: Array<Array<Array<FloatArray>>>): List<Keypoint?> {
    val raw = output[0][0]
    return (0 until 17).map { i ->
        val confidence = raw[i][2]
        if (confidence >= 0.3f) Keypoint(raw[i][0], raw[i][1], confidence) else null
    }
}

Returning null for low-confidence keypoints forces your downstream biomechanics layer to handle missing data explicitly — which is correct behavior when computing metrics like knee valgus or hip drop in gait analysis.

Gotchas

Don't normalize INT8 input to [-1, 1]. The docs do not make this obvious, but that normalization applies to float MoveNet variants. Thunder and Lightning INT8 quantized models expect raw [0, 255] byte values. This mistake silently corrupts every keypoint output.

setTargetResolution is a hint, not a guarantee. CameraX picks the nearest resolution the hardware supports. Always explicitly scale your bitmap to exactly 256×256 before passing it to the model.

Per-frame ByteBuffer allocation will kill your frame rate. GC pauses from hot-path allocations dwarf inference latency on mid-range devices. A Snapdragon 695 runs Thunder INT8 in 18–22ms with GPU delegate — a naive allocation strategy adds unpredictable jank on top that no delegate optimization recovers.

Conclusion

The inference model is rarely the bottleneck — the data pipeline around it is. Here is what actually gets you under 25ms:

OUTPUT_IMAGE_FORMAT_RGBA_8888 offloads YUV conversion to the hardware path and cuts preprocessing from 25–40ms down to 4–8ms.
A class-level ByteBuffer eliminates GC pressure on the hot path.
A 0.3 confidence threshold with null propagation keeps physically invalid joint angles out of your metrics.

Further reading: TFLite GPU delegate docs · CameraX ImageAnalysis reference · MoveNet on TF Hub

DEV Community