I am looking to display HDR images on iOS using CAMetalLayer/MTKView. I see CALayers such as AVCaptureVideoPreviewLayer or AVPlayerLayer have the capability to display HDR content. However, it appears these layers might be using EDR APIs to achieve this. However this answer by an Apple Engineer suggests it is not possible to fully display it in MTKView.
I suppose iPhone display may not be fully capable of displaying HDR images and may be doing some kind of tone mapping from HDR to EDR (which is said to be Apple's own HDR representation).
I need to understand if capabilities of MTKView are limited compared to builtin CALayer subclasses such as AVCaptureVideoPreviewLayer or AVPlayerLayer as far as displaying HDR content is concerned? This WWDC video suggests HDR content can be displayed correctly (or as best as any CALayer can display).
Since iOS 16, it is possible to display EDR (or HDR, see below) content with MTKView. The setup is shown in the video you linked at 9:18.
// Opt into using EDR
let layer = self.layer as? CAMetalLayer
layer?.wantsExtendedDynamicRangeContent = true
// Use half-float pixel format
layer?.pixelFormat = .rgba16Float
// Use extended linear display P3 color space
layer?.colorspace = CGColorSpace(name: CGColorSpace.extendedLinearDisplayP3)
Note that you are responsible for rendering content into the view's drawable that matches the set color space. You can also adapt the color space to the content you want to display, e.g., setting it to .itur_2020 if your HDR image is in that color space. Or you can use frameworks like Core Image that can help you with color-matching between the different color spaces and formats.
I also recommend you check out the other WWDC videos on this topic:
Clarification on Apple's EDR
What Apple calls Extended Dynamic Range (EDR) is a bit more than just Apple's representation of HDR. It's their umbrella term for color values that go beyond the standard range (as defined by sRGB). This includes wide gamut color spaces (like Display P3 or BT-2020), as well as extended brightness values (e.g., when the standard white value [1.0, 1.0, 1.0] would be displayed with 500 nits brightness, a bright extended white [2.0, 2.0, 2.0] would be displayed with 1000 nits).
They chose the term EDR because HDR can mean many different things, depending on the context (extended brightness, exposure fusion, color spaces, etc).
The Explore HDR rendering with EDR gives a good explanation on this. I can also recommend checking out the Support Apple Pro Display XDR tech talk for learning more about EDR formats, color spaces, and headroom.