High Efficiency Image File Format

At WWDC17, Apple had 6 big announcements:

1. watchOS 4
2. macOS High Sierra
3. New Macs
4. iOS 11
5. iPad Pro 10.5-inch
6. HomePod

While those announcements may have made headline news, one of the new features of iOS 11 that I find most interesting will be a new file format for pictures called High Efficiency Image File Format (HEIF). This new photo format will replace JPG on iPhone models that have an A9 processor, namely the iPhone 6s, 6s Plus, the iPhone SE, and iPhone 7 and beyond. A technical overview is provided here. In summary, HEIF use cases include:

• Storage of still images as well as collection of images in a single container file
• Storage of burst photos
• Storage and efficient representation of video animations and cinemagraphs
• Support for simultaneous capture of video and still images, i.e. storing still images and timed image sequences into the same file
• Storage of images derived from other images of the file, either as derived images represented by non-destructive image editing operations or as pre-computed derived images
• Storage of focal and exposure stacks into the same container file
• Support for storing other media, such as audio and timed text, into the same container file with timed image sequences and synchronizing their playback

HEIF files are based on the ISO Base Media File Format (ISOBMFF, a structure for time-based multimedia files) and and includes a rich set of features including storage of other media streams (e.g., timed text and audio). This allows HEIF files to utilize High Efficiency Video Coding (HEVC or H.265) encoded media. Specs and features are also covered in blogs here and here as well as many others.

Aside from the higher efficiency of its storage capacity, HEIC is opening the door to various forms of computational photography--that is, images that are post-processed rather than optically captured. Video animations and cinemagraphs are a couple of examples provided above, but I wonder if high dynamic range (HDR) images will see additional improvements with HEIC. HDR is currently processed within the iPhone's Camera app with no user control over the output, but perhaps 3rd party apps will support bracketed images with 3, 5, or 7 (or more?) exposures that can be blended together with more manual controls for better capture of tonal range in HDR images. I guess time will tell.

I still prefer shooting with my DSLR because of its superior optics and greater number of options for images post-processing (particularly with RAW images) while remaining interested in how mobile phones are innovating with image capture and imaging post-processing.