---
title: "Real-Time OCR on Android: CameraX to TFLite in 45ms"
published: true
description: "Wire CameraX ImageAnalysis to a quantized CRNN/EAST pipeline on Android. The tricks that separate 45ms from 120ms — INT8, GPU delegate, and buffer layout."
tags: android, kotlin, mobile, architecture
canonical_url: https://mvpfactory.co/blog/real-time-ocr-android-camerax-tflite-45ms
---
## What you will build
By the end of this tutorial, you will have a working end-to-end OCR pipeline that processes camera frames in under 45ms on a mid-range Android device. We are wiring CameraX `ImageAnalysis` to a quantized CRNN/EAST model, running inference through the GPU delegate, and decoding character sequences with CTC beam search.
Here is the full pipeline at a glance:
CameraX ImageAnalysis
└─► Frame Buffering (STRATEGY_KEEP_ONLY_LATEST)
└─► YUV → RGB + Normalization
└─► EAST Text Detection (INT8, GPU Delegate)
└─► CRNN Recognition per RoI
└─► CTC Beam Search Decode
└─► Structured Output + Bounding Boxes
## Prerequisites
- Android Studio Hedgehog or later
- TensorFlow Lite runtime with GPU delegate dependency
- CameraX `1.3.x`
- An INT8-quantized EAST/CRNN model (`.tflite`)
- A physical mid-range Android device — emulators will not give you honest numbers
## The thing most teams get wrong
Let me show you a pattern I use in every project. On-device OCR is not a model problem. It is a systems problem. The model accounts for roughly 20% of your latency budget. The remaining 80% is how you move bytes between camera, preprocessor, and inference runtime. Teams spend weeks squeezing the CRNN and ship a 90ms pipeline because YUV conversion and buffer copies consumed the rest of the budget — silently.
Get the plumbing right first.
## Step 1 — Configure CameraX ImageAnalysis
Use `STRATEGY_KEEP_ONLY_LATEST`, not `STRATEGY_BLOCK_PRODUCER`. Under load, blocking the camera producer thread causes frame queue backup and perceived jitter.
kotlin
val imageAnalysis = ImageAnalysis.Builder()
.setTargetResolution(Size(1280, 720))
.setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
.setOutputImageFormat(ImageAnalysis.OUTPUT_IMAGE_FORMAT_YUV_420_888)
.build()
imageAnalysis.setAnalyzer(inferenceExecutor) { imageProxy ->
processFrame(imageProxy)
imageProxy.close() // Always close — memory leak otherwise
}
Resolution matters. 1280×720 gives you enough density for small text without blowing your normalization budget. Going to 4K adds ~18ms before inference even starts.
## Step 2 — INT8 quantization and input normalization
The numbers tell a clear story:
| Precision | Model Size | Inference (GPU) | Inference (CPU) |
|-----------|-----------|-----------------|-----------------|
| FP32 | 42 MB | 68ms | 210ms |
| FP16 | 21 MB | 51ms | 195ms |
| INT8 | 11 MB | 31ms | 112ms |
INT8 cuts model size by ~4× and inference time by 30–50% on devices with no dedicated NPU — which is most of the Android mid-range market.
Here is the minimal setup to get this working. Pre-allocate your input buffer **outside** the analysis callback. Your EAST input tensor expects `[1, H, W, 3]` normalized to `[-1.0, 1.0]`:
kotlin
private val inputBuffer: ByteBuffer = ByteBuffer
.allocateDirect(1 * MODEL_H * MODEL_W * 3)
.order(ByteOrder.nativeOrder())
fun normalizeYuvToBuffer(image: ImageProxy) {
inputBuffer.rewind()
// Convert YUV plane directly — avoid intermediate Bitmap allocation
val yPlane = image.planes[0].buffer
// ... fast YUV→RGB→normalize loop
}
## Step 3 — GPU delegate and memory layout
Initialize once at startup. Never inside a callback.
kotlin
val gpuDelegate = GpuDelegate(GpuDelegate.Options().apply {
setPrecisionLossAllowed(true) // Enables FP16 on GPU, 10-15% faster
setQuantizedModelsAllowed(true)
})
val options = Interpreter.Options().addDelegate(gpuDelegate)
val detector = Interpreter(modelBuffer, options)
Memory layout determines whether you hit 45ms or 120ms. The GPU delegate requires NHWC layout and aligned buffers. Misaligned tensors force a copy on every inference call — that copy alone can add 20ms.
For profiling GPU memory pressure across the full pipeline, Android GPU Inspector gives you per-stage timing, shader occupancy, and buffer transfer costs in a single trace. Use it before trusting any aggregate benchmark number.
## Step 4 — CTC beam search decoding
CRNN outputs a probability distribution over characters per timestep. Greedy decoding is fast but poor on ambiguous text. CTC beam search at width 8 is the sweet spot: accuracy equivalent to width 20 at half the cost, and it only adds 2–3ms over greedy on low-contrast documents.
## Gotchas
Here is the gotcha that will save you hours: **allocating `ByteBuffer` inside the analysis callback**. Calling `ByteBuffer.allocateDirect()` on every frame costs 4–8ms in GC pressure alone. Allocate once at initialization, rewind on each use.
**Skipping `imageProxy.close()`** — the docs mention it, but I have seen it absent in production codebases. You will leak camera buffer memory within minutes of sustained use.
**Trusting inference-only benchmarks.** The model is rarely your bottleneck. Trace the full pipeline: frame acquisition, normalization, buffer transfers, and system overhead together. Android GPU Inspector is the right tool for this.
**The docs do not mention this explicitly**, but misaligned tensors with the GPU delegate force a silent copy on every inference call. Get your buffer layout right at initialization and verify it with a one-time alignment check before shipping.
## Latency budget
| Stage | Target |
|-----------------------------|--------|
| Frame acquisition + YUV→RGB | 4ms |
| Input normalization | 3ms |
| EAST detection (INT8, GPU) | 18ms |
| CRNN recognition (per RoI) | 12ms |
| CTC decode + bbox assembly | 3ms |
| System overhead | ~5ms |
| **Total** | **~45ms** |
That ~5ms system overhead — thread scheduling jitter, `ImageProxy` teardown, output struct construction — shows up consistently across mid-range devices in production traces. Budget for it. It is not optional.
*(If you are deep in pipeline debugging, [HealthyDesk](https://play.google.com/store/apps/details?id=com.healthydesk) is worth keeping in the background — it will remind you to step away before the tension sets in after hour three of frame-timing traces.)*
## Three things to do today
Pre-allocate all buffers outside the analysis callback. `ByteBuffer.allocateDirect()` called on every frame is a GC time bomb. Allocate once, rewind on each use.
Use INT8 quantization with the GPU delegate on any device lacking a dedicated NPU. The accuracy loss on standard document OCR is under 1% ANLS; the latency gain is 30–50%.
Profile the full pipeline — not just inference. Trace every stage with Android GPU Inspector: frame acquisition, normalization, buffer transfers, and system overhead together. The model is rarely your bottleneck.
Top comments (0)