How to asynchronous update image in custom CNContactViewController

My app has a local database of contacts not stored in the internal contact store (users can however choose to add the contacts to the internal contact store)

I'm using the CNContactViewController to show the details of the contacts - but the image is not (always) stored in the database and has to be loaded (asynchronous) on each request.

A minimalized version of the ContactModel class:

class ContactModel {

    var id: String
    var givenName: String?
    var thumbnail: Data?

    init?(_ identifier: String?, givenName: String? = nil)
    {
        if ((identifier ?? "").isEmpty) {
            return nil
        }
        self.id = identifier
    }

    func toMutableContact() -> CNMutableContact
    {
        let contact = CNMutableContact()
        contact.contactType = .person
        contact.givenName = givenName

        if let thumbnail = thumbnail {
            contact.imageData = thumbnail
        }

        return contact
    }
}

The following code shows the contact details and gets the thumbnail from an online service

func showContact(_ model : ContactModel)
{
    // Create a 'CNMutableContact' from the 'ContactModel' object
    let contact : CNMutableContact = model.toMutableContact()

    let store = CNContactStore()
    let cvc = CNContactViewController(forUnknownContact: contact)
    cvc.delegate = self
    cvc.contactStore = CNContactStore()
    cvc.allowsEditing = false
    self.navigationController?.pushViewController(cvc, animated: true)

    var hasThumbnail: Bool = false
    if let _ = model.thumbnail {
        hasThumbnail = true
    }

    if !hasThumbnail {
        // Get the thumbnail from the online service
        getThumbnail(id: model.id) { (data, error) in

            // Everything above works as expected!

            // This does not work either
            contact.givenName = "XXX"

            if let data = data {
                // How can I update the image ?
                print("imageDataAvailable (before): \(contact.imageDataAvailable)") // returns: false
                contact.imageData = data
                print("imageDataAvailable (after): \(contact.imageDataAvailable)")  // returns: true
            }
        }
    }
}

I'm not going to save the contact to the internal contact store - just using the CNContactViewController to view the details.

If the image is set in the ContactModel then the image is shown perfectly - but if any values are changed afterwards then the view is not updated.

(The user can choose not to save the thumbnails to the local database due to space consumption)

You can't do that. The contact displayed by the CNContactViewController is an immutable contact with no connection to the CNMutableContact you created at the outset (it's a copy).