Google Just Open-Sourced ML Drift, the GPU Engine Behind LiteRT. Here's What Changes for You

SignalTech — Sat, 17 Oct 2026 04:05:27 +0000

If you've ever tried to ship an ML model that runs fast on users' GPUs, you know the problem: you have no idea which GPU, driver or graphics API your app will meet. A Pixel, an iPhone, a browser tab and a Windows laptop all speak different dialects.

On 8 October, Google's AI Edge team open-sourced the engine it uses to deal with exactly that. It's called ML Drift, and if you use TensorFlow Lite or LiteRT, it affects you.

What ML Drift is

ML Drift is an open-source (Apache 2.0), cross-platform GPU compute engine built for on-device AI/ML inference. It hides the hardware and low-level API differences across OpenGL ES, OpenCL, Metal and WebGPU.

It's the core GPU acceleration engine inside LiteRT (Google's on-device runtime), and it's also available as a standalone library if you're building your own graphics or inference runtime.

It isn't a research toy. Google says it already runs in production on millions of devices daily, in Chrome, YouTube Shorts, Photos and Meet.

Why it matters

One shader codebase instead of three. The old TFLite GPU delegate needed separately maintained shaders for OpenGL, OpenCL and Metal, with tensors hardcoded to physical GPU objects like textures and buffers. ML Drift introduces tensor virtualization: a tensor's logical layout is separated from how it's physically stored on the GPU, so one set of shader templates works across backends, with minimal runtime overhead according to Google.

5D tensors, finally. The legacy delegate was hardcoded to 4D tensors. ML Drift supports 5D out of the box, which helps 3D convolutional and spatiotemporal models. Google names YOLO 11n, MobileViT v2 and Swin Transformer v2 as examples.

Built with on-device LLMs in mind. LLM inference has a compute-heavy prefill stage and a memory-bandwidth-heavy decode stage. ML Drift switches kernels and layouts depending on which stage it's in. In Google's Gemma benchmarks, it used up to 12% less memory than other frameworks.

Desktop, too (in preview). The WebGPU backend, compiled natively through Dawn (Chromium's WebGPU implementation), lets ML Drift run on Windows and Linux, alongside a native Metal backend on macOS. Mobile and edge remain the main focus.

Custom ops are easier. There's a custom op framework with direct registration APIs, plus a SKILL.md guide that helps coding agents write, register and check custom shaders.

The production numbers Google shares: up to 40% lower average frame latency for YouTube Shorts effects on Android and iOS, up to 30% faster on-device performance for some Adobe Lightroom and Photoshop features, and 30% lower model latency for some Snapchat lens effects.

If you use the TFLite GPU delegate

This is the part to act on. The legacy TFLite GPU delegate will no longer receive new feature updates. Google recommends moving to the LiteRT ML Drift GPU accelerator, which it says is fully backwards compatible with existing models. Google also says models already running on TFLite GPU should maintain or beat their previous benchmarks.

One caveat for Android developers using unbundled runtimes to keep app size down: ML Drift acceleration is available now in the standalone LiteRT packages, but it's still "coming soon" to LiteRT in Google Play Services.

How to get started

The announcement gives two paths:

App developers: use the LiteRT ML Drift accelerator. You get model partitioning, tensor virtualization and LLM acceleration out of the box.
GPU and systems engineers: build directly on ML Drift's standalone C++ APIs from the GitHub repo to build custom GpuModel compute graphs and write your own shaders. There are quickstarts for OpenCL on Android and WebGPU via Dawn.

Source code: github.com/google-ai-edge/ml-drift. Issues and feature requests go on GitHub, and questions go to the Google AI Edge Community Forum.

The takeaway

If your app relies on the TFLite GPU delegate, it won't break today, but it's now frozen. Put a migration to the LiteRT ML Drift accelerator on your roadmap, benchmark your models before and after, and check the Play Services caveat if you ship an unbundled runtime.

Source: ML Drift: Next-Gen GPU AI/ML Inference at the Edge (Google Developers Blog)

I write Signal Tech, a free weekly plain-English briefing on AI, chips and dev tools: https://dailysignaltech.beehiiv.com