c++macosmacos-high-sierrametalkitcore-video

OSX MetalKit CVMetalTextureCacheCreateTextureFromImage failed, status: -6660


I'm trying to make firstly CVPixelBuffer from RAW memory then MTLTexture from CVPixelBuffer, but after running following code I've always got error

CVMetalTextureCacheCreateTextureFromImage failed, status: -6660 0x0

Where is this error came from?

    id<MTLTexture> makeTextureWithBytes(id<MTLDevice> mtl_device, 
                                        int width, 
                                        int height, 
                                        void *baseAddress, int bytesPerRow)
    {

    CVMetalTextureCacheRef textureCache = NULL;

    CVReturn status = CVMetalTextureCacheCreate(kCFAllocatorDefault, nullptr, mtl_device, nullptr, &textureCache);
    if(status != kCVReturnSuccess || textureCache == NULL)
    {
        return nullptr;
    }

    NSDictionary* cvBufferProperties = @{
        (__bridge NSString*)kCVPixelBufferOpenGLCompatibilityKey : @YES,
        (__bridge NSString*)kCVPixelBufferMetalCompatibilityKey : @YES,
    };

    CVPixelBufferRef pxbuffer = NULL;

    status = CVPixelBufferCreateWithBytes(kCFAllocatorDefault,
                                          width,
                                          height,
                                          kCVPixelFormatType_32BGRA,
                                          baseAddress,
                                          bytesPerRow,
                                          releaseCallback,
                                          NULL/*releaseRefCon*/,
                                          (__bridge CFDictionaryRef)cvBufferProperties,
                                          &pxbuffer);

    if(status != kCVReturnSuccess || pxbuffer == NULL)
    {
        return nullptr;
    }

    CVMetalTextureRef cvTexture = NULL;   

    status = CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault,
                                                       textureCache,
                                                       pxbuffer,
                                                       nullptr,
                                                       MTLPixelFormatBGRA8Unorm,
                                                       1920,
                                                       1080,
                                                       0,
                                                       &cvTexture);

    if(status != kCVReturnSuccess || cvTexture == NULL)
    {
        std::cout << "CVMetalTextureCacheCreateTextureFromImage failed, status: " << status << " " << cvTexture << std::endl;
        return nullptr;
    }
    id<MTLTexture> metalTexture = CVMetalTextureGetTexture(cvTexture);

    CFRelease(cvTexture);

    return metalTexture;
}

Solution

  • The error occurs when the CVPixelBuffer isn't backed by an IOSurface. However you can't make an IOSurface backed CVPixelBuffer from Bytes. So despite having the kCVPixelBufferMetalCompatibilityKey set, CVPixelBufferCreateWithBytes (and its planar counterpart) will not back the buffer with an IOSurface.

    2 ways around this (and possible 3rd)

    1. create an empty CVpixelBuffer and use memcpy. You'll need to create a new pixelbuffer each loop, so using a PixelBufferPool is advised.
    CVPixelBufferPoolRef pixelPool; // initialised prior
    void *srcBaseAddress;  // initialised prior
    
    CVPixelBufferRef currentFrame; 
    CVPixelBufferPoolCreatePixelBuffer(nil, pixelPool, &currentFrame);
    
    CVPixelBufferLockBaseAddress(currentFrame,0);
    void *cvBaseAddress=CVPixelBufferGetBaseAddress(currentFrame);
    size_t size= CVPixelBufferGetDataSize(currentFrame);
    memcpy(cvBaseAddress,srcBaseAddress,size);
    
    1. Skip the CVPixelBuffer entirely and write directly to the MTLTexture memory; As long as metal supports your pixel format. However since you're probably rendering to an RGB display, you can write your own metal kernel to convert to RGB. Remember to make the texture first using metalTextureDescriptor.
    id<MTLDevice> metalDevice; // You know how to get this
    unsigned int width; // from your source image data
    unsigned int height;// from your source image data
    unsigned int rowBytes; // from your source image data
    
    MTLTextureDescriptor *mtd = [MTLTextureDescriptor 
    texture2DDescriptorWithPixelFormat:MTLPixelFormatBGRG422 
            width:width
            height:height
            mipmapped:NO];
    
    id<MTLTexture> mtlTexture = [metalDevice newTextureWithDescriptor:mtd];
    
    
    [mtlTexture replaceRegion:MTLRegionMake2D(0,0,width,height) 
            mipmapLevel:0
            withBytes:srcBaseAddress
            bytesPerRow:rowBytes];
    

    A third way might be to convert the CVPixelBuffer into a CIImage, and use a Metal backed CIContext. Something like this;

    id<MTLDevice> metalDevice;
    CIContext* ciContext = [CIContext contextWithMTLDevice:metalDevice
            options:[NSDictionary dictionaryWithObjectsAndKeys:@(NO),kCIContextUseSoftwareRenderer,nil]
        ];
    CIImage *inputImage = [[CIImage alloc] initWithCVPixelBuffer:currentFrame];
    
    [ciContext render:inputImage 
        toMTLTexture:metalTexture 
        commandBuffer:metalCommandBuffer
        bounds:viewRect
        colorSpace:colorSpace];
    

    I successfully used this to render directly to the CAMetalLayer's drawable texture, but didn't have much luck rendering to an intermediate texture (not that I tried very hard to get it working) Hope one of these works for you.