Correctly tonemap an HDR image to SDK using ImageIO

In my app, I am trying to export an SDR sRGB JPEG (which is 8bpc) from an HDR BT.2100 JPEG XL (which is 16bpc), stored inside Apple’s photo library (but this is unimportant here).

I need to correctly tonemap the HDR input colorspace into SDR (either SRGB or P3 colorspaces, suitable for JPG’s low bit depth). This should be possible, as Apple’s color management already supports it. For example, when attempting to display the original JXL image anywhere in macOS that is correctly color-managed on an SDR display will result in the correct output.

Since I would like to support very large (61MP+) 10bpc or 16bpc images, I am trying to use ImageIO (CGImageSource directly to CGImageDestination, if possible) as it is the most memory efficient and performance approach. Before that I used Core Image, which also would not produce correctly tonemapped image, but would balloon memory usage (500-800 MB jump in memory usage for the conversion) and take a considerable time to finish.

Any suggestions would be welcome.

Please note that these files are actual high dynamic range, as opposed to what used to be/is known as “HDR photography”, which would merge several exposures into a conventional SDR image.

This is what I have so far:

guard let data = sourceImageData,
      let source = CGImageSourceCreateWithData(data as CFData, [kCGImageSourceShouldCache: false, kCGImageSourceTypeIdentifierHint: asset.uniformType.identifier as CFString] as CFDictionary),
      let cfOutputData = CFDataCreateMutable(nil, 0),
      let imageDestination = CGImageDestinationCreateWithData(cfOutputData, outputUTType.identifier as CFString, 1, nil) else {
    break
}

var outputOptions: [AnyHashable: Any] = [kCGImageDestinationMergeMetadata: true,
                               kCGImageDestinationLossyCompressionQuality: options.outputLossyCompressionQuality,
                             kCGImageSourceCreateThumbnailFromImageAlways: true,
                               kCGImageSourceCreateThumbnailWithTransform: true]
if case let .scaled(scale) = options.outputResize {
    outputOptions[kCGImageDestinationImageMaxPixelSize] = Double(max(asset.pixelWidth, asset.pixelHeight)) * scale
} else if case let .aspectResized(maxPixelSize) = options.outputResize {
    outputOptions[kCGImageDestinationImageMaxPixelSize] = maxPixelSize
}
if [.jpeg /* and other 8bpc formats */].contains(options.outputDataFormat) {
    outputOptions[kCGImageSourceDecodeRequest] = kCGImageSourceDecodeToSDR
    outputOptions[kCGImageDestinationPreserveGainMap] = false
    outputOptions[kCGImagePropertyNamedColorSpace] = CGColorSpace.sRGB
} else {
    outputOptions[kCGImageSourceDecodeRequest] = kCGImageSourceDecodeToHDR
    outputOptions[kCGImageSourceDecodeRequestOptions] = [
        "kCGFallbackHDRGain": 10000,
        "kCGTargetColorSpace": CGColorSpace.itur_2100_PQ,
        "kCGTargetHeadroom": 1000000
    ]
}

CGImageDestinationAddImageFromSource(imageDestination, source, 0, outputOptions as CFDictionary)

if CGImageDestinationFinalize(imageDestination) {
    imageData = cfOutputData as Data
}

I expected kCGImageSourceDecodeRequest = kCGImageSourceDecodeToSDR to do what I need, tonemapping the HDR image to SDR, but alas, that is not the case.

Interestingly, HDR images taken with iPhones are not in BT.2100 (or other HDR) colorspace, but are composites of a P3 base image with a gain map metadata layer, which I am able to handle correctly with kCGImageDestinationPreserveGainMap = false.

Here is an example of an HDR JXL file:

https://www.dropbox.com/scl/fi/p9bpuuykjgtwaeax7cfyc/orig.jxl?rlkey=w48cx1av2p2uzbt8m39luq2bb&st=bnrcpkg3&dl=0

If you open this in macOS 15.0+’ Preview on a modern MacBook Pro, or with a suitable HDR display, such as Pro Display XDR, you should be able to see the entire dynamic range. On a macOS before Sequoia, Preview was not configured to display HDR images; in that case, you can import the image to Photos, which should then display the HDR range correctly. JXL is supported in macOS 14 and above, iOS 17 and above.

Some (incorrect) output examples:

ImageIO

Since it didn’t change the colorspace at all, this JPG’s colorspace is BT.2100, and you can see the posterization (color banding), stemming from the low bit-depth.

Core Image

Here, the SRGB colorspace was correctly assigned to the image, but no tonemapping was performed, so colors are incorrect.

Photos app

Unsurprisingly, if the Photos app is used to copy (CMD+C) the JXL image, it produces a correctly tonemapped, P3 colorspace JPG with smooth gradients.

Any advice on how to achieve what I need is appreciated. Thanks

Turns out I put the kCGImageSourceDecodeRequest: kCGImageSourceDecodeToHDR option in the wrong place, the destination, rather than the source.

Here is a sample code that works:

guard var imageData,
      let source = CGImageSourceCreateWithData(imageData as CFData, [
        kCGImageSourceShouldCache: false,
        kCGImageSourceTypeIdentifierHint: asset.uniformType.identifier,
        kCGImageSourceDecodeRequest: isRequestSDR ? kCGImageSourceDecodeToSDR : kCGImageSourceDecodeToHDR
      ] as CFDictionary),
      let imageDestination = CGImageDestinationCreateWithData(cfOutputData, outputUTType.identifier as CFString, 1, nil) else {
    return imageData as Any
}

CGImageDestinationAddImageFromSource(_imageDestination, source, CGImageSourceGetPrimaryImageIndex(source), outputOptions as CFDictionary)

if CGImageDestinationFinalize(imageDestination) {
    imageData = cfOutputData as Data
}

return imageData

This solves my original question, but I do feel it is a "magic", where something behind the scenes happens, but it is unclear to me what exactly happens. If I wanted to convert from one HDR format to another, like PQ to HLG, or one HDR colorspace to another, it is still not clear to me how to achieve.