iosavfoundationavcapturesessionavassetwriteravcapturevideodataoutput

AVFoundation record 10 bit HDR video on iPhone 12


iPhone 12/12 pro supports recording Dolby vision HDR video in 10 bit format instead of 8 bits but it is not clear from iOS 14.1 SDK if AVCaptureVideoDataOutput supports delivery of 10 bit sample buffers that can be appended to video file using AVAssetWriter. Has anyone figured it out whether it is possible or not in SDK?

EDIT: A number of apps such as Apple's Clips app have started supporting Dolby Vision 10 bit video recording. But I tried every available API including videoHDREnabled but it doesn't work. So the clear question is how to record HDR (Dolby vision) video using AVFoundation APIs?

EDIT2: I was able to figure out device formats that support 10 bit pixel buffer formats (which is 'x420', NOT the ones with 420v or 420f as media subtype) . On iPhone 12 mini, 4 device formats support 10 bit pixel buffer delivery in kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange, even though AVFoundation documentation says this is not a supported pixel format (Quoting - "On iOS, the only supported key is kCVPixelBufferPixelFormatTypeKey. Supported pixel formats are kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange, kCVPixelFormatType_420YpCbCr8BiPlanarFullRange and kCVPixelFormatType_32BGRA."). Next step is to identify if HDR format for recording can be manually chosen to be Dolby Vision, HLG, or HDR10.


Solution

  • Ok none of the answers given were correct, so I researched after grabbing an iPhone 12 mini in hand and this is what I found.

    AVFoundation documentation is silent and even incorrect at times. One can infer from the documentation that it is not possible to get 10 bit HDR sample buffers, specifically if one reads the documentation of videoSettings property of AVCaptureVideoDataOutput:

       On iOS, the only supported key is kCVPixelBufferPixelFormatTypeKey. 
       Supported pixel formats are kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange, 
       kCVPixelFormatType_420YpCbCr8BiPlanarFullRange and kCVPixelFormatType_32BGRA
    

    It appears from the documentation that one could never get 10 bit frames. But on probing -[AVCaptureDevice formats], one can find 4 formats that are different and have mediaSubtype as 'x420', which is kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange, a 10 bit format. The moment -[AVCaptureDevice activeFormat] is set to one of these 4 formats, AVCaptureVideoDataOutput changes the sample buffer format to kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange! Active Color Space of AVCaptureDevice also changes to AVCaptureColorSpace_HLG_BT2020.