swiftavfoundationcalayermetalcvpixelbuffer

How to achieve better performance in converting UIView to CVPixelBuffer?


I'm wondering if it's possible to achieve a better performance in converting the UIView into CVPixelBuffer.

My app converts a sequence of UIViews first into UIImages and then into CVPixelBuffers as shown below. In the end, I record all these images/frames into an AVAssetWriterInput and save the result as a movie file.

Thank you in advance!

Best, Aibek

func viewToImage(view: UIView) -> CGImage {
  let rect: CGRect = container.frame

  UIGraphicsBeginImageContextWithOptions(rect.size, true, 1)

  let context: CGContext = UIGraphicsGetCurrentContext()!
  view.layer.render(in: context)
  let img = UIGraphicsGetImageFromCurrentImageContext()

  UIGraphicsEndImageContext()

  return img!.cgImage
}
func imageToBuffer(image: CGImage) -> CVPixelBuffer? {
  let frameSize = CGSize(width: image.width, height: image.height)

  var pixelBuffer: CVPixelBuffer?
  let status = CVPixelBufferCreate(kCFAllocatorDefault, Int(frameSize.width), Int(frameSize.height), kCVPixelFormatType_32BGRA, nil, &pixelBuffer)

  if status != kCVReturnSuccess {
    return nil
  }

  CVPixelBufferLockBaseAddress(pixelBuffer!, CVPixelBufferLockFlags(rawValue: 0))
  let data = CVPixelBufferGetBaseAddress(pixelBuffer!)
  let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
  let bitmapInfo = CGBitmapInfo(rawValue: CGBitmapInfo.byteOrder32Little.rawValue | CGImageAlphaInfo.premultipliedFirst.rawValue)
  let context = CGContext(data: data, width: Int(frameSize.width), height: Int(frameSize.height), bitsPerComponent: 8, bytesPerRow: CVPixelBufferGetBytesPerRow(pixelBuffer!), space: rgbColorSpace, bitmapInfo: bitmapInfo.rawValue)

  context?.draw(image, in: CGRect(x: 0, y: 0, width: image.width, height: image.height))

  CVPixelBufferUnlockBaseAddress(pixelBuffer!, CVPixelBufferLockFlags(rawValue: 0))

  return pixelBuffer
}

Solution

  • Converting the UIViews into MTLTextures and recording them into a video file using the Recorder provided by Mayo didn't increase the performance actually.

    However, the recorder is able to write MTLTextures in real-time. That meant for me that I can re-write all the animations using Metal and use the recorder.