When using a CIFilter
with an AVVideoComposition
, how can I access the current frame number in the AVAsynchronousCIImageFilteringRequest
callback?
The best I was able to do is estimate the frame number from the time:
^(AVAsynchronousCIImageFilteringRequest * _Nonnull request) {
double seconds = CMTimeGetSeconds(request.compositionTime);
double fps = [[_avAsset tracksWithMediaType:AVMediaTypeVideo] firstObject].nominalFrameRate;
int frame = round(seconds * fps);
// (Calculate filter parameters based on frame number)
}
But this isn't accurate enough for me. Is there a way to access the frame number?
The key is to set the frameDuration
on the AVMutableVideoComposition
to match that of the underlying AVAssetTrack
. if you do this, then the values of compositionTime
will be even multiples of minFrameDuration
and there won't be any rounding issues.
AVMutableVideoComposition *vidComp =
[AVMutableVideoComposition videoCompositionWithAsset:self.avAsset
applyingCIFiltersWithHandler:
^(AVAsynchronousCIImageFilteringRequest * _Nonnull request) {
CMTime frame = self.avAssetTrack.minFrameDuration;
int frameNum = (request.compositionTime.value * frame.timescale) /
(request.compositionTime.timescale * frame.value);
// (Calculate filter parameters based on frame number)
}];
vidComp.frameDuration = self.avAssetTrack.minFrameDuration;
In my case of a ~60 fps MP4 movie, avAssetTrack.minFrameDuration = (1001/60000)
, but vidComp.frameDuration = (1501/90000)
before I update it. This resulted in slow drift of 'compositionTime' in the callback, which resulted in occasional repeated frames.