vImage on the Mac: unable to convert from Core Video

Edit: Fixed. A working sample is at https://github.com/halmueller/vImage-mac-sample.

I'm trying to read the feed of a MacBook Pro's Facetime camera to process it with the vImage framework. I'm following the example in Apple's VideoCaptureSample, which is written for iOS.

I'm getting hung up on creating the vImageConverter, which creates an image buffer that vImage can use. My call to vImageConverter_CreateForCVToCGImageFormat() fails, with the console error "insufficient information in srcCVFormat to decode image. vImageCVImageFormatError = -21601".

The same call works on iOS. But the image formats are different on iOS and macOS. On iOS, the vImageConverter constructor is able to infer the format information, but on macOS, it can't.

Here's my setup code:

func displayEqualizedPixelBuffer(pixelBuffer: CVPixelBuffer) {
    var error = kvImageNoError

    if converter == nil {
        let cvImageFormat = vImageCVImageFormat_CreateWithCVPixelBuffer(pixelBuffer).takeRetainedValue()
        let deviceRGBSpace = CGColorSpaceCreateDeviceRGB()
        let dcip3SolorSpace = CGColorSpace(name: CGColorSpace.dcip3)
        vImageCVImageFormat_SetColorSpace(cvImageFormat,
                                          deviceRGBSpace)
        print(cvImageFormat)

        if let unmanagedConverter = vImageConverter_CreateForCVToCGImageFormat(
            cvImageFormat,
            &cgImageFormat,
            nil,
            vImage_Flags(kvImagePrintDiagnosticsToConsole),
            &error) {

            guard error == kvImageNoError else {
                return
            }

            converter = unmanagedConverter.takeRetainedValue()
        } else {
            return
        }
    }

When I run on iOS, I see in the console:

vImageCVFormatRef 0x101e12210:
type: '420f'
matrix: 
    0.29899999499321 0.58700001239777 0.11400000005960
    -0.16873589158058 -0.33126410841942 0.50000000000000
    0.50000000000000 -0.41868758201599 -0.08131241053343
chroma location: <RGB Base colorspace missing>
RGB base colorspace: =Bo

On macOS, though, the call to vImageConverter_CreateForCVToCGImageFormat returns nil, and I see:

vImageCVFormatRef 0x10133a270:
type: '2vuy'
matrix: 
    0.29899999499321 0.58700001239777 0.11400000005960
    -0.16873589158058 -0.33126410841942 0.50000000000000
    0.50000000000000 -0.41868758201599 -0.08131241053343
chroma location: <RGB Base colorspace missing>
RGB base colorspace: Рü

2018-03-13... kvImagePrintDiagnosticsToConsole: vImageConverter_CreateForCVToCGImageFormat error: 
insufficient information in srcCVFormat to decode image. vImageCVImageFormatError = -21601

Note that the image type (4 letter code) is different, as is the RGB base colorspace. I've tried on the Mac using dcip3ColorSpace instead of deviceRGB, and the results are the same.

What am I missing to get this vImageConverter created?

The -21601 error code means that the source CV format is missing chroma siting information (see http://dougkerr.net/Pumpkin/articles/Subsampling.pdf for a nice background of chroma siting). You can fix this by explicitly setting it with vImageCVImageFormat_SetChromaSiting. So, immediately after setting the format's color space, and before creating the converter (i.e. where you have print(cvImageFormat)), add the following:

        vImageCVImageFormat_SetChromaSiting(cvImageFormat,
                                            kCVImageBufferChromaLocation_Center)

Cheers!

simon