swiftcore-imagecvpixelbuffercmsamplebuffer

CIContext render crash, when inserting image into an existing CMSampleBuffer, on 60 FPS


I want to put an image into a CMSampleBuffer, but the following code always crashes on bad access after random time on the line

ciContext.render(compositedImage, to: outputBuffer)

this however happens only on 60FPS, not at 30FPS

any ideas? the function is called from the same thread and directly from the data delegate

PS: tried a CIContext with Metal / CPU, locked, unlocked

func process(sampleBuffer: CMSampleBuffer) -> CMSampleBuffer? {
    guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
        return nil
    }

    // Make a CIImage from input buffer and clamp it to avoid artifacts at edges
    let ciImage = CIImage(cvPixelBuffer: pixelBuffer).oriented(.up).clampedToExtent()

    // Composite overlay image over camera image
    let compositedImage = overlayImage.composited(over: ciImage)

    // Create new pixel buffer for output
    var newPixelBuffer: CVPixelBuffer?
    let pixelBufferAttrs = [
        kCVPixelBufferPixelFormatTypeKey as String: Int(kCVPixelFormatType_32BGRA),
        kCVPixelBufferWidthKey as String: CVPixelBufferGetWidth(pixelBuffer),
        kCVPixelBufferHeightKey as String: CVPixelBufferGetHeight(pixelBuffer),
        kCVPixelBufferIOSurfacePropertiesKey as String: [:]
    ] as CFDictionary

    CVPixelBufferCreate(kCFAllocatorDefault,
                        CVPixelBufferGetWidth(pixelBuffer),
                        CVPixelBufferGetHeight(pixelBuffer),
                        kCVPixelFormatType_32BGRA,
                        pixelBufferAttrs,
                        &newPixelBuffer)

    guard let outputBuffer = newPixelBuffer else { return nil }

    // Render using locked buffer
    let flag: CVPixelBufferLockFlags = CVPixelBufferLockFlags(rawValue: 0)
    CVPixelBufferLockBaseAddress(outputBuffer, flag)
    ciContext.render(compositedImage, to: outputBuffer)
    CVPixelBufferUnlockBaseAddress(outputBuffer, flag)
    
    // Copy timing info from original sample buffer
    var timingInfo = CMSampleTimingInfo()
    CMSampleBufferGetSampleTimingInfo(sampleBuffer, at: 0, timingInfoOut: &timingInfo)

    // Create format description for new pixel buffer
    var formatDescription: CMFormatDescription?
    CMVideoFormatDescriptionCreateForImageBuffer(allocator: kCFAllocatorDefault,
                                                 imageBuffer: outputBuffer,
                                                 formatDescriptionOut: &formatDescription)

    guard let newFormatDesc = formatDescription else { return nil }

    // Create new CMSampleBuffer with modified image
    var newSampleBuffer: CMSampleBuffer?
    CMSampleBufferCreateReadyWithImageBuffer(allocator: kCFAllocatorDefault,
                                             imageBuffer: outputBuffer,
                                             formatDescription: newFormatDesc,
                                             sampleTiming: &timingInfo,
                                             sampleBufferOut: &newSampleBuffer)

    return newSampleBuffer
}

Solution

  • This looks like an Xcode 16.3, 16.4 thread checker issue, as when disconnected from XCode, the crash doesn't happen

    enter image description here