Dolby Vision profile 5 is common, and on an Android TV box or phone without a Dolby Vision decoder it plays green and purple. Here is how Yonto Player draws them with the right colours anyway, on top of media3 (ExoPlayer), with the GPU doing the colour work.
Why profile 5 is the hard one
Profile 8 carries an HDR10, HLG or SDR picture under the Dolby Vision layer, so a device that knows nothing about Dolby Vision can still show something sensible. Profile 5 has no such fallback. The picture is coded in Dolby's own IPT-PQ-c2 colour space, and every frame carries an RPU (Reference Processing Unit) with the metadata that turns it back into light. Skip the RPU and decode it as plain HEVC, and you are looking at IPT values as if they were YCbCr, which is where the green and purple come from.
media3 maps Dolby Vision profiles 4, 8 and 9 to an HEVC decoder when no Dolby Vision decoder is present. Profile 5 it leaves alone, which makes sense, since there is no base layer to show. So on most phones and many TV boxes it finds no decoder at all.
The plan
- If the device has a Dolby Vision decoder, use it, as before.
- Otherwise decode the track as HEVC Main 10.
- Read each frame's RPU on its way into the decoder.
- Have the decoder render into a
SurfaceTextureinstead of the screen. - Apply the RPU on the GPU, tone-map to SDR, and draw to the screen.
Choosing the decoder
The video renderer is media3's MediaCodecVideoRenderer with getDecoderInfos overridden. media3's own choice comes first, so a real Dolby Vision decoder always wins. Only when that list is empty and the track is profile 5 (codecs dvhe.05 or dvh1.05) does it fall back to the plain HEVC decoders, keeping only those whose capabilities list the Main 10 profile (or list nothing, which some decoders do).
The codec is then configured as HEVC with KEY_PROFILE set to Main 10.
Reading the RPU
The RPU travels as an HEVC NAL unit of type 62, whose two-byte header is 7C 01. In onQueueInputBuffer the renderer looks at each access unit just before it goes to the decoder, finds the last RPU in it from the end, strips the emulation-prevention bytes, checks its CRC-32 and parses it. The parser is plain Kotlin in the shared core, since nothing in it is Android's.
Two things make this harder than it sounds. B-frames come out of the decoder in a different order than they went in, so the parsed RPU is stored against the sample's timestamp and taken back out by presentation time when the decoder releases that frame. And RPUs refer back to earlier ones: a reshaping can be reused by id, and display metadata can be left out to mean "same as before", so they are resolved in decode order. A missing or broken RPU reuses the previous frame's, and a seek forgets all of it.
Drawing it ourselves
The renderer keeps the screen's Surface it receives through MSG_SET_VIDEO_OUTPUT for itself and hands the codec the input Surface of a SurfaceTexture. For each frame it releases, it tells the drawing side the release time and the RPU that goes with it. The drawing side renders, sets that time with eglPresentationTimeANDROID and swaps. Frame pacing and A/V sync stay media3's.
The one extension that makes it possible
This was the crux. Sampling an external texture with samplerExternalOES gets you RGB: the driver converts from YCbCr and clamps, and the IPT codes are gone before your shader sees them. GL_EXT_YUV_target adds __samplerExternal2DY2YEXT, which hands you the raw Y, U and V codes instead. Declaring it is all the extension asks:
#extension GL_EXT_YUV_target : require
uniform __samplerExternal2DY2YEXT frame; // texture(frame, uv) returns Y, U, V codes
The shader then looks those codes up in a table (below) and writes RGB.
No OpenGL ES 3, no GL_EXT_YUV_target, or no HEVC decoder with Main 10, and the whole path is off. Those devices play it in the app's second player, which says its colours won't be right. Two video players in one app, and the rule that picks one covers that hand-over.
The colour maths
In order, per pixel:
- The RPU's reshaping curves (polynomial and MMR pieces) bring I, P and T back.
- The RPU's matrix and offset take IPT to PQ-coded LMS.
- The PQ curve turns that into linear light.
- LMS to BT.2020.
- ITU-R BT.2390's tone map on the brightest channel, from the scene's peak (the RPU's level 1 metadata, else the mastering display's peak, never below 100 nits) down to SDR white at 203 nits.
- BT.2020 to BT.709, clipped.
- The sRGB curve.
It is written from the published standards and the bitstream syntax.
Too slow per pixel, so bake it
Run in full for every pixel, that took 80 ms for one 1080p frame on a Google TV Streamer's PowerVR GE9215. A 24 fps film needs a frame every 42 ms.
So the maths runs once into a 3D lookup table and every frame is one texture fetch:
- 65 × 65 × 65 points, 10 bits a channel (
GL_RGB10_A2, which every OpenGL ES 3.0 device can render into and filter). - Rebuilt one layer at a time, only when a frame's parameters differ from the last one's, which in practice means at a scene change.
Against the exact maths over every pixel of two frames, the table is within 0.13 of an 8-bit level on average, with under 0.5% of channels more than two levels off, the worst at the edge of the gamut. Spacing the table's points towards the shadows measured no better, since the error is at the gamut's edge, not in the steep start of the curve.
How close it gets
On the Google TV Streamer, with its own Dolby Vision decoder hidden from the renderer so the HEVC decoder and the GPU did the work:
- Frame 48 of the 1080p sample: within 0.5 of a level of the reference implementation on average (worst 9).
- Frame 300, after seeking into another scene: within 0.8 on average (worst 23, with 0.03% of pixels more than 8 off).
- No dropped frames at 1080p, or at 4K at 25 fps.
- Moving the picture to a new
Surfacemid-play works.
For comparison, libplacebo through FFmpeg (on MoltenVK, with BT.2390 chosen) lands within 3.1 and 4.5 levels of ours on the same two frames, about as far as it is from the reference (3.0 and 4.3). Its knee and black point differ; the hues are the same.
What we ruled out
-
media3's
setVideoEffects. Its effects pipeline converts every frame to RGB before any effect runs, through the driver's YCbCr-to-RGB step into RGBA8, or for HDR a clamped BT.2020 path at medium precision. The codes the RPU needs are gone by then. -
Reading frames on the CPU with an
ImageReaderin P010 and uploading the planes as integer textures, for GPUs without the extension. It costs a 25 MB copy per 4K frame, and those GPUs are on the weakest devices, so they keep the second player.
Where it stops
- The output is SDR on every screen, an HDR TV included.
- Dolby's level 2 and level 8 trims are not applied, so contrast and saturation can differ slightly from a licensed Dolby Vision display's own SDR picture.
- The Android emulator has neither a Main 10 HEVC decoder nor
GL_EXT_YUV_target, so this path needs a real device to see.
Read more
- Video format support, what plays where.
- Video formats, how the app picks a player for a file, profile 5 included.
- The changelog for 5.5.1, the release this shipped in.
- Two video players in one app, and the rule that picks one.
- One Kotlin core for every screen, the series article on what is shared across platforms.
Top comments (0)