ioscore-dataicloudmirroringnspersistentcloudkitcontainer

CoreData + CloudKit Data Inconsistency after Network Failure


My app uses CoreData + CloudKit mirroring to synchronize data e.g. on an iPhone and a watch.
If data is modified on one device, the modification is uploaded to iCloud and later synchronized with other devices.
This works normally fine. However very rarely the following happens:

Data is modified on a device, and the app is terminated.
When the app is re-launched next time, not the modified data is displayed but the unmodified version.
I assume (I don't know how CoreData + CloudKit mirroring works internally) the following problem.

The problem:

Consider the following setup: One has a CoreData entity Item with some attributes, among them updatedAt: Date?. Each time an attribute is changed, updatedAt is updated, and the Item is saved to the persistent store that is mirrored to iCloud. After saving, the updated Item is exported to iCloud.
When the app is terminated and later re-launched, the iCloud version is imported, which does not have any effect since it is the modified version. However:
If the app is terminated before the modified version could be uploaded, e.g. because there is no network connection, iCloud has still the unmodified version.
After re-launch of the app, the unmodified version with an older updatedAt value is imported and overwrites the modified version with a newer updatedAt value. So the modification is lost.

Possible solution?:

My first idea is to use two persistent stores, a localStore that is not mirrored, and a mirrorStore that is mirrored. The entity Item is assigned to both stores. When an Item is saved, it is saved to both stores.
Normally, i.e. without the problem described above, both stores have an identical copy of the Item. When an Item is fetched, it is fetched only from the localStore by setting the affectedStores property of the fetch request accordingly.

However, when the problem arises, the Item in the mirrorStore is overwritten by an older version. This can be handled by listening to a .NSPersistentStoreRemoteChange notification of the mirrorStore. When notified, one could fetch the Item from the localStore and the mirrorStore, and select the version with the newer updatedAt value. In the described scenario, this would always be the Item in the localStore, but if the Item has been modified later on another device, the version in the mirrorStore could also be newer. In any case, the older version has to be overwritten with the newer version. This can be done by deleting the older version, and saving the newer version again to both stores. Then data is again consistent.

My questions:

Edit:

I realized by now one reason for unexpected termination of the app.
A background CoreData+CloudKit export may take too long on the Watch, see the following log:

2022-03-31 11:18:12.910276+0200 Watch Extension[2388:703470] [BackgroundTask]  
Background Task 122 ("CoreData: CloudKit Export"), was created over 30 seconds  
ago. In applications running in the background, this creates a risk of termination.  
Remember to call UIApplication.endBackgroundTask(_:) for your task in a timely  
manner to avoid this.  
…  
2022-03-31 11:19:00.036156+0200 Watch Extension[2388:703470] [BackgroundTask]  
Background task still not ended after expiration handlers were called:  
<_UIBackgroundTaskInfo: 0x16514b00>: taskID = 122, taskName = CoreData:  
CloudKit Export, creationTime = 61315 (elapsed = 82).  
This app will likely be terminated by the system.  
Call UIApplication.endBackgroundTask(_:) to avoid this.

Solution

  • The problem:

    The problem described does actually exist.
    The basic reason is that CoreData & CloudKit is not able to decide whether a CoreData record (a CoreData managed object) or its mirrored iCloud record (a CKRecord) is the "source of truth", i.e. is the valid record in case they are different.
    In all applications that I can imagine, the record modified last is the valid record (except from errors).
    Now, CKRecords have a system property modificationDate that is updated automatically when the CKRecord is changed. However, CoreData entiries don't have a system attribute like modificationDate. CoreData & CloudKit is thus not able to select the later modified record as the source of truth.
    If an app is launched or goes into foreground, CoreData & CloudKit triggers first an import from iCloud and updates the persistent store. This means that local updates that have not been exported yet to iCloud, e.g. because the app had been terminated before an export could be done due to a network problem, will be overwritten and are lost.

    My solution:

    All my CoreData entities have an attribute modificationDate. Such an attribute can also be set automatically, see here. My CoreData records are stored in a local persistent store, while the iCloud private database is mirrored by CoreData & CloudKit to another private persistent store. When this private persistent store is updated by mirroring, it sends an .NSPersistentStoreRemoteChange notification. The function that handles the notification compares the modificationDate fields of the CoreData and iCloud records, and selects the newer one, i.e. updates either the record in the local persistent store or the private persistent store.
    Of course there are conflicts possible between a managed context and a persistent store. Such a conflict has also to be resolved by selecting the newer version of the CoreData record. This can, however, be handled by using a custom merging policy as described here.