Building a 64bit native OSX (not iOS) ap targeting 10.7+. Somewhat new to dealing with video files in a Cocoa universe.
I would like to be able to open a video file and display the output inside an openGL render (IE, I would like to be able to efficiently access the framebuffer of a video and turn each frame into an opengl texture.)
Conceptually this seems straightforward, but I'm having a difficult time wading through various (old and deprecated)examples and options, all of which seem to have been recently deprecated in favor of AVFoundation. It's possible I missed something, but examples of using AVFoundation with OpenGL seem thin on the ground.
To clarify a bit further, this sample application (QTCoreVideo101 from Apple) does more or less exactly what I want, except that it is built around the deprecated QTKit will therefore not even compile in 64 bit.
I am reading through the AVFoundation docs right now, but I'm still not sure it make sense to try and get a glTexture out of AVFoundation or if I should be looking elsewhere.
This is the solution I ended up going with. "thisLayer.layerSource.videoPlayerOutput" is AVPlayerItemVideoOutput object.
if ([thisLayer.layerSource.videoPlayerOutput hasNewPixelBufferForItemTime:playerTime]){
frameBuffer= [thisLayer.layerSource.videoPlayerOutput copyPixelBufferForItemTime:playerTime itemTimeForDisplay:NULL];
CVReturn result= CVOpenGLTextureCacheCreateTextureFromImage(NULL,
textureCache,
frameBuffer,
NULL,
&textureRef);
if(result == kCVReturnSuccess){
// These appear to be GL_TEXTURE_RECTANGLE_ARB
thisLayer.layerSource.vid_glTextureTarget=CVOpenGLTextureGetTarget(textureRef);
thisLayer.layerSource.vid_glTexture=CVOpenGLTextureGetName(textureRef);
thisLayer.layerSource.vid_glTextureSize=NSMakeSize(CVPixelBufferGetWidth(frameBuffer), CVPixelBufferGetHeight(frameBuffer));
thisLayer.layerSource.vid_ciimage=[CIImage imageWithCVImageBuffer:frameBuffer];
CFRelease(textureRef);
CVOpenGLTextureCacheFlush(textureCache, 0);
}else{
NSLog(@"INTERNAL ERROR FAILED WITH CODE: %i",result);
}
CVBufferRelease(frameBuffer);
}
AVAssetReaderTrackOutput
(added to your AVAssetReader
) will output a CVPixelBufferRef
, of which you can specify your preferred format to upload to OpenGL via glTexImage
or glTexSubImage
.