Enhancing HDR on Instagram for iOS with Dolby Vision
Article Summary
Chris Ellsworth, Cosmin Stejerean, and Hassene Tmar from Meta reveal how Instagram became the first Meta app to support Dolby Vision. The catch? Their initial A/B test showed people watched LESS video with the enhancement.
Meta's engineering team tackled a complex challenge: preserving Dolby Vision and ambient viewing environment (amve) metadata through their entire video pipeline. Since their FFmpeg-based transcoding system was stripping this metadata, HDR videos weren't displaying as intended, especially at low brightness levels. The solution required contributing code to FFmpeg, implementing custom metadata extraction, and overcoming unexpected engagement issues.
Key Takeaways
- First A/B test failed: Dolby Vision added 100 kbps overhead, causing longer load times
- Compressed metadata format reduced overhead from 100 kbps to 25 kbps average
- Implemented 2000+ lines of code for compression/decompression in FFmpeg
- Final test showed increased watch time in low-light environments
- Transcoded HEVC Profile 8.4 to AV1 Profile 10.4 for delivery
Instagram for iOS now delivers Dolby Vision metadata with all AV1 encodings from iPhone HDR uploads, resulting in measurably increased engagement in low-light viewing conditions.
About This Article
iPhone HDR videos include Dolby Vision Profile 8.4 metadata that's built for HEVC delivery. When Meta needed to transcode these videos to AV1 codec instead, they ran into a problem: FFmpeg and Shaka packager didn't have proper specification support for this metadata in AV1.
Meta worked with FFmpeg developers to add full support for Dolby Vision Profile 8 and Profile 10. This made it possible to transcode from HEVC Profile 8.4 to AV1 Profile 10.4 using libaom and libsvtav1 encoders. They also manually extracted metadata for AVSampleBufferDisplayLayer to consume.
Display color analyzer measurements showed that Dolby Vision metadata made image brightness track screen brightness settings more closely at all brightness levels. A/B testing found that viewers watched longer in low-light environments after Meta reduced metadata compression overhead from 100 kbps to 25 kbps.