iosswiftavfoundationavassetreadercmsamplebuffer

When reading frames from a video on an iPad with AVAssetReader, the images are not properly oriented


A few things I want to establish first:

What I am left with is a situation where when I run the following code (condensed from my application to remove unrelated code), I am getting an image that is upside down (landscape) or rotated 90 degrees (portrait). Viewing the video that is processed just prior to this step shows that it is properly oriented. All testing has been done on iOS 11.2.5.

* UPDATED *

I did some further testing and found a few more interesting items:

A condensed version of the code involved:

import UIKit
import AVFoundation

let asset = ...
let assetReader = try? AVAssetReader(asset: asset)

if let assetTrack = asset.tracks(withMediaType: .video).first,
    let assetReader = assetReader {
    let assetReaderOutputSettings = [
        kCVPixelBufferPixelFormatTypeKey as String : NSNumber(value: kCVPixelFormatType_32BGRA)
    ]
    let assetReaderOutput = AVAssetReaderTrackOutput(track: assetTrack,
                                                     outputSettings: assetReaderOutputSettings)
    assetReaderOutput.alwaysCopiesSampleData = false
    assetReader.add(assetReaderOutput)

    var images = [UIImage]()
    assetReader.startReading()

    var sample = assetReaderOutput.copyNextSampleBuffer()

    while (sample != nil) {
        if let image = sample?.uiImage { // The image is inverted here
            images.append(image)
            sample = assetReaderOutput.copyNextSampleBuffer()
        }
    }
    // Continue here with array of images...
}

Solution

  • After some exploration, I came across the following that allowed me to obtain the video's orientation from the AVAssetTrack:

    let transform = assetTrack.preferredTransform
    radians = atan2(transform.b, transform.a)
    

    Once I had that, I was able to convert it to degrees:

    let degrees = (radians * 180.0) / .pi
    

    Then, using a switch statement I could determine how to rotate the image:

    Switch Int(degrees) {
       case -90, 90:
           // Rotate accordingly
       case 180:
           // Flip
       default:
           // Do nothing
    }