How to vanish player controls from lockscreen after the playback has finished in iOS10+?

While playing audio in background mode player controls appears on the lockscreen. How to remove it when the audio has stopped? If try to set:

MPNowPlayingInfoCenter.default().nowPlayingInfo = nil

player is still on the lockscreen, but the fields artist/song are empty

UPD (my code for audiosession):

in AppDelegate:

func setupAudioSession() {

    let audioSession = AVAudioSession.sharedInstance()

    do {
        try audioSession.setCategory(AVAudioSessionCategoryPlayback)
        try audioSession.setActive(true)

    } catch {
        print("Setting category to AVAudioSessionCategoryPlayback failed.")
    }
}

in Player class:

private func clearRemotePlayerInfo() { // call after stop button pressed
    try? AVAudioSession.sharedInstance().setActive(false)
    MPNowPlayingInfoCenter.default().nowPlayingInfo = [:]
}

TL;DR

Example on Github: https://github.com/JakubMazur/SO52243428

---

You shouldn't assign nil to this nowPlayingInfo.

What you should do to achieve this is:

1. Stop your playback (not necessary but it's good to clean up what you've created)
2. Set your session inactive
3. Clear nowPlayingInfo

So the code will look like:

self.player?.stop() // Where self.player is AVAudioPlayer
try? self.session?.setActive(false, with: .notifyOthersOnDeactivation) // Where self.session is AVAudioSession. You should do it with do-catch to good error catching.
MPNowPlayingInfoCenter.default().nowPlayingInfo = [:]

And it will behave like this:

EDIT:

I've wrote as simple as possible example for try it out. It's available on Github https://github.com/JakubMazur/SO52243428. Feel free to check it out and it it matches your case.