[SOLVED] Delta Queries not working on some tenants

Delta Queries not working on some tenants

I have wasted a month already being chased from pillar to post by MPN Support. We are busy developing a service to sync user profile images from Microsoft Graph to our system for clients using our SaaS platform. The list of clients includes 200 clients and 4’000’000 Microsoft users about are affected by this.

Doing full sync daily from Microsoft Graph is suicide on our end and it will be murder on the Microsoft Graph infrastructure as we run these syncs globally across the globe for all our clients. To get around us having to hammer the Graph service, we opted to run full sync once per client, store the delta token and then from that point on wards we only sync updates from the delta every 30 minutes.

During development, we tested this against one of our test tenants and it all went well. The full sync completes and from that point on wards we would see delta results if a user updated their profile image. We then started testing against our primary tenant and some client tenants and very quickly found that the delta query was not returning results on updates on some tenants. This brought the entire project plan to a halt and as a result, we cannot sync profile images, forcing to stay on the current plan of over querying Graph with live queries.

I have tried this across 25 tenants and 5 are failing.

var usersDeltaLink = SQLService.GetDeltaToken(syncConfig);

List<string> memberIds = new List<string>();

var usersDeltaRequest = _graphClient
    .Users
    .Delta()
    .Request(usersDeltaLink == null ? new Option[0] : new[]
    {
        new QueryOption("$deltatoken", usersDeltaLink.Token)
    });

var deltaUsers = await usersDeltaRequest.GetAsync();
int totalCount = 0;

List<Task> itemTasks = new List<Task>();
do
{
    totalCount += deltaUsers.Count;
    Log.Information($"Fetched out {deltaUsers.Count} users");
    itemTasks.AddRange(deltaUsers.CurrentPage.Select(x => x.Id).ToList().Select(async memAccId =>
    {
        await UpdateUserPhoto(memAccId);
    }));
}
while (deltaUsers.NextPageRequest != null && (deltaUsers = await deltaUsers.NextPageRequest.GetAsync()).Count > 0);

await Task.WhenAll(itemTasks);

Uri deltaLink = new Uri(deltaUsers.AdditionalData["@odata.deltaLink"].ToString());
var deltaToken = HttpUtility.ParseQueryString(deltaLink.Query).Get("$deltatoken");

I expect Microsoft Graph Delta queries to work properly and not have 20% failure rate on tenants.

Solution