This problem didn't occur in Xcode 10.2.1 and iOS 12. It started in Xcode 11.1 and iOS 13
My app records video, when the app goes to the background I stop the capture session from running and remove the preview layer. When the app comes back to the foreground I restart the capture session and add the preview layer back in:
let captureSession = AVCaptureSession()
var previewLayer: AVCaptureVideoPreviewLayer?
var movieFileOutput = AVCaptureMovieFileOutput()
// *** I initially didn't remove the preview layer in this example but I did remove it in the other 2 examples below ***
@objc fileprivate func stopCaptureSession() {
DispatchQueue.main.async {
[weak self] in
if self?.captureSession.isRunning == true {
self?.captureSession.stopRunning()
}
}
}
@objc func restartCaptureSession() {
DispatchQueue.main.async {
[weak self] in
if self?.captureSession.isRunning == false {
self?.captureSession.startRunning()
}
}
}
What happens is when I go to the background and come back the preview layer and ui is completely frozen. But before going to the background if i put a breakpoint on the line if self?.captureSession.isRunning == true and another breakpoint on the line if self?.captureSession.isRunning == false, once I trigger the breakpoints the preview layer and ui works fine.
Upon further research I came upon this question and in the comments @HotLicks said:
Obviously, it's likely that the breakpoint gives time for some async activity to complete before the above code starts mucking with things. However, it's also the case that 0.03 seconds is an awfully short repeat interval for a timer, and it may simply be the case that the breakpoint allows the UI setup to proceed before the timer ties up the CPU.
I did a little more research and Apple said:
The startRunning() method is a blocking call which can take some time, therefore you should perform session setup on a serial queue so that the main queue isn't blocked (which keeps the UI responsive). See AVCam-iOS: Using AVFoundation to Capture Images and Movies for an implementation example.
Using the comment from @HotLicks and the info from Apple I switched over to use DispatchQueue.main.sync and then Dispatch Group and after coming back from the background the preview layer and ui were still frozen. But once I add the breakpoints like I did in the first example and trigger them the preview layer and ui works fine.
What am I doing wrong?
Update
I switched from debug mode to release mode and it still didn't work.
I also tried switching to using DispatchQueue.global(qos: .background).async and a timer DispatchQueue.main.asyncAfter(deadline: .now() + 1.5) like @MohyG suggested but it made no difference.
Upon further inspection without the breakpoint the Background notification works fine but it's the Foreground Notification that's not getting called when the app enters the fg. For some reason the fg notification only triggers when I first put a break point inside the stopCaptureSession() function.
The issue is the foreground notification only fires with the breakpoint I described above.
I tried DispatchQueue.main.sync:
@objc fileprivate func stopCaptureSession() {
if captureSession.isRunning { // adding a breakpoint here is the only thing that triggers the foreground notification when the the app comes back
DispatchQueue.global(qos: .default).async {
[weak self] in
DispatchQueue.main.sync {
self?.captureSession.stopRunning()
}
DispatchQueue.main.async {
self?.previewLayer?.removeFromSuperlayer()
self?.previewLayer = nil
}
}
}
}
@objc func restartCaptureSession() {
if !captureSession.isRunning {
DispatchQueue.global(qos: .default).async {
[weak self] in
DispatchQueue.main.sync {
self?.captureSession.startRunning()
}
DispatchQueue.main.asyncAfter(deadline: .now() + 15) {
self?.previewLayer = AVCaptureVideoPreviewLayer(session: self!.captureSession)
self?.previewLayer?.videoGravity = AVLayerVideoGravity.resizeAspectFill
guard let previewLayer = self?.previewLayer else { return }
previewLayer.frame = self!.containerViewForPreviewLayer.bounds
self?.containerViewForPreviewLayer.layer.insertSublayer(previewLayer, at: 0)
}
}
}
}
I tried Dispatch Group:
@objc fileprivate func stopCaptureSession() {
let group = DispatchGroup()
if captureSession.isRunning { // adding a breakpoint here is the only thing that triggers the foreground notification when the the app comes back
group.enter()
DispatchQueue.global(qos: .default).async {
[weak self] in
self?.captureSession.stopRunning()
group.leave()
group.notify(queue: .main) {
self?.previewLayer?.removeFromSuperlayer()
self?.previewLayer = nil
}
}
}
}
@objc func restartCaptureSession() {
let group = DispatchGroup()
if !captureSession.isRunning {
group.enter()
DispatchQueue.global(qos: .default).async {
[weak self] in
self?.captureSession.startRunning()
group.leave()
group.notify(queue: .main) {
self?.previewLayer = AVCaptureVideoPreviewLayer(session: self!.captureSession)
self?.previewLayer?.videoGravity = AVLayerVideoGravity.resizeAspectFill
guard let previewLayer = self?.previewLayer else { return }
previewLayer.frame = self!.containerViewForPreviewLayer.bounds
self?.containerViewForPreviewLayer.layer.insertSublayer(previewLayer, at: 0)
}
}
}
}
Here is the rest of the code if needed:
NotificationCenter.default.addObserver(self, selector: #selector(appHasEnteredBackground),
name: UIApplication.willResignActiveNotification,
object: nil)
NotificationCenter.default.addObserver(self, selector: #selector(appWillEnterForeground),
name: UIApplication.willEnterForegroundNotification,
object: nil)
NotificationCenter.default.addObserver(self, selector: #selector(sessionWasInterrupted),
name: .AVCaptureSessionWasInterrupted,
object: captureSession)
NotificationCenter.default.addObserver(self, selector: #selector(sessionInterruptionEnded),
name: .AVCaptureSessionInterruptionEnded,
object: captureSession)
NotificationCenter.default.addObserver(self, selector: #selector(sessionRuntimeError),
name: .AVCaptureSessionRuntimeError,
object: captureSession)
func stopMovieShowControls() {
if movieFileOutput.isRecording {
movieFileOutput.stopRecording()
}
recordButton.isHidden = false
saveButton.isHidden = false
}
@objc fileprivate func appWillEnterForeground() {
restartCaptureSession()
}
@objc fileprivate func appHasEnteredBackground() {
stopMovieShowControls()
imagePicker.dismiss(animated: false, completion: nil)
stopCaptureSession()
}
@objc func sessionRuntimeError(notification: NSNotification) {
guard let error = notification.userInfo?[AVCaptureSessionErrorKey] as? AVError else { return }
stopMovieRecordigShowControls()
if error.code == .mediaServicesWereReset {
if !captureSession.isRunning {
DispatchQueue.main.async { [weak self] in
self?.captureSession.startRunning()
}
} else {
restartCaptureSession()
}
} else {
restartCaptureSession()
}
}
@objc func sessionWasInterrupted(notification: NSNotification) {
if let userInfoValue = notification.userInfo?[AVCaptureSessionInterruptionReasonKey] as AnyObject?,
let reasonIntegerValue = userInfoValue.integerValue,
let reason = AVCaptureSession.InterruptionReason(rawValue: reasonIntegerValue) {
switch reason {
case .videoDeviceNotAvailableInBackground:
stopMovieShowControls()
case .audioDeviceInUseByAnotherClient, .videoDeviceInUseByAnotherClient:
stopMovieShowControls()
case .videoDeviceNotAvailableWithMultipleForegroundApps:
print("2. The toggleButton was pressed")
case .videoDeviceNotAvailableDueToSystemPressure:
// no documentation
break
@unknown default:
break
}
}
}
@objc func sessionInterruptionEnded(notification: NSNotification) {
restartCaptureSession()
stopMovieShowControls()
}
I found the bug and it was an extremely WEIRD bug.
The tint color of the images of the buttons are white. Instead of using a regular black background I wanted a blurred background so i used this:
func addBackgroundFrostToButton(_ backgroundBlur: UIVisualEffectView, vibrancy: UIVisualEffectView, button: UIButton, width: CGFloat?, height: CGFloat?){
backgroundBlur.frame = button.bounds
vibrancy.frame = button.bounds
backgroundBlur.contentView.addSubview(vibrancy)
button.insertSubview(backgroundBlur, at: 0)
if let width = width {
backgroundBlur.frame.size.width += width
}
if let height = height {
backgroundBlur.frame.size.height += height
}
backgroundBlur.center = CGPoint(x: button.bounds.midX, y: button.bounds.midY)
}
And I called in viewDidLayoutSubview() like:
lazy var cancelButto: UIButton = {
let button = UIButton(type: .system)
//....
return button
}()
let cancelButtoBackgroundBlur: UIVisualEffectView = {
let blur = UIVisualEffectView(effect: UIBlurEffect(style: .dark))
//...
return blur
}()
let cancelButtoVibrancy: UIVisualEffectView = {
let vibrancyEffect = UIVibrancyEffect(blurEffect: UIBlurEffect(style: .extraLight))
// ...
return vibrancyView
}()
override func viewDidLayoutSubviews() {
super.viewDidLayoutSubviews()
// I did this with 4 buttons, this is just one of them
addBackgroundFrostToButton(cancelButtoBackgroundBlur,
vibrancy: cancelButtoVibrancy,
button: cancelButto,
width: 10, height: 2.5)
}
Once I commented out the above code the foreground notification started firing with no problem and I didn't need the breakpoint anymore.
Since viewDidLayoutSubviews() can get called multiple times the UIVisualEffectView and the UIVibrancyEffect kept compounding on top of each other and for some very WEIRD reason it affected the foreground notification.
To get around it I simply created a Bool to check to see if the blurs were added to the button. Once I did that I had no more problems.
var wasBlurAdded = false
override func viewDidLayoutSubviews() {
super.viewDidLayoutSubviews()
if !wasBlurAdded {
addBackgroundFrostToButton(cancelButtoBackgroundBlur,
vibrancy: cancelButtoVibrancy,
button: cancelButto,
width: 10, height: 2.5)
wasBlurAdded = true
}
}
I don't know why or how this affected the foreground notification observer but like I said, this was an extremely WEIRD bug.