Setting Audio Input node for AVAudioEngine causes outside audio to stop

I'm building an app that will allow users to record voice notes. The functionality of all that is working great; I'm trying to now implement changes to the AudioSession to manage possible audio streams from other apps. I want it so that if there is audio playing from a different app, and the user opens my app; the audio keep playing. When we start recording, any third party app audio should stop, and can then can resume again when we stop recording.

This is my main audio setup code:

    private var audioEngine: AVAudioEngine!
    private var inputNode: AVAudioInputNode!

    func setupAudioEngine() {
        audioEngine = AVAudioEngine()
        inputNode = audioEngine.inputNode
        audioPlayerNode = AVAudioPlayerNode()
        audioEngine.attach(audioPlayerNode)

        let format = AVAudioFormat(standardFormatWithSampleRate: AUDIO_SESSION_SAMPLE_RATE, channels: 1)
        audioEngine.connect(audioPlayerNode, to: audioEngine.mainMixerNode, format: format)
    }

    private func setupAudioSession() {
        let audioSession = AVAudioSession.sharedInstance()
        do {
            try audioSession.setCategory(.playAndRecord, mode: .default, options: [.defaultToSpeaker, .allowBluetooth])
            try audioSession.setPreferredSampleRate(AUDIO_SESSION_SAMPLE_RATE)
            try audioSession.setPreferredIOBufferDuration(0.005) // 5ms buffer for lower latency
            try audioSession.setActive(true)
            
            // Add observers
            setupInterruptionObserver()
        } catch {
            audioErrorMessage = "Failed to set up audio session: \(error)"
        }
    }

This is all called upon app startup so we're ready to record whenever the user presses the record button.

However, currently when this happens, any outside audio stops playing.

I isolated the issue to this line: inputNode = audioEngine.inputNode

When that's commented out, the audio will play -- but I obviously need this for recording functionality.

Is this a bug? Expected behavior?

So, the thing you're running into is actually something iOS does on purpose when it deals with audio sessions, especially with the playAndRecord catagory. When your app starts using the microphone with AVAudioSessionCategoryPlayAndRecord, iOS tends to either pause or lower the volume of other apps' audio so it can focus on recording.

If you want your app to be able to record audio without stoping the music or podcasts from other apps until you actually start recording, and then have it resume after you're done, you can use the mixWithOthers option when setting up the audio session. But keep in mind, once you hit record, you'll need to change the session to stop other audio.

Here's a more detailed breakdown:

First, set up the audio session to allow mixing with others:

private func setupInitialAudioSession() {
    let audioSession = AVAudioSession.sharedInstance()
    do {
        try audioSession.setCategory(.playAndRecord, mode: .default, options: [.defaultToSpeaker, .allowBluetooth, .mixWithOthers])
        try audioSession.setPreferredSampleRate(AUDIO_SESSION_SAMPLE_RATE)
        try audioSession.setPreferredIOBufferDuration(0.005) // 5ms buffer for lower latency
        try audioSession.setActive(true)
        
        // Add obsevers
        setupInterruptionObserver()
    } catch {
        audioErrorMessage = "Failed to set up initial audio session: \(error)"
    }
}

Next, change the audio session catagory when you start recording:

private func startRecording() {
    let audioSession = AVAudioSession.sharedInstance()
    do {
        // Update the audio session to not mix with others during recording
        try audioSession.setCategory(.playAndRecord, mode: .default, options: [.defaultToSpeaker, .allowBluetooth])
        try audioSession.setActive(true)
        
        // Start your audio engine and recording process
        audioEngine.prepare()
        try audioEngine.start()
    } catch {
        audioErrorMessage = "Failed to start recording: \(error)"
    }
}

And then change it back when you stop recording:

private func stopRecording() {
    let audioSession = AVAudioSession.sharedInstance()
    do {
        // Stop the audio engine and recording process
        audioEngine.stop()
        
        // Update the audio session to allow mixing with others again
        try audioSession.setCategory(.playAndRecord, mode: .default, options: [.defaultToSpeaker, .allowBluetooth, .mixWithOthers])
        try audioSession.setActive(true)
    } catch {
        audioErrorMessage = "Failed to stop recording: \(error)"
    }
}

So, by setting the mixWithOthers option initially, you let other app audio play while your app is running. When you start recording, you switch to a stricter catagory that pauses other audio. After you're done recording, you go back to the original setup.

Make sure to handle any errors that might pop up and manage these state transitions smoothly to give users a good experiance. And definetly test this on actual devices to make sure the audio behavior is what you expect.

Hope this helps! Karl