EF Core 8 does not merge entities

When switching from EF Core 5 to version 8, the following problem was revealed.

var task = await _context.Tasks
    .Include(t => t.Document)
    .ThenInclude(t => t.RouteStepValues)
    .FirstOrDefaultAsync(t => t.Id == taskId);

var documentId = task.Document.ParentDocumentId.HasValue ? 
    task.Document.ParentDocumentId.Value : task.Document.Id;

var document = await _context.Documents
    .Include(d => d.RouteStepValues)
    .Include(t => t.DocumentType)
    .ThenInclude(t => t.Components)
    .ThenInclude(t => t.ComponentPropertyDependencies)
    .ThenInclude(t => t.ElementProperty)
    .FirstOrDefaultAsync(d => d.Id == documentId);

var currentRouteStep = document.RouteStepValues.FirstOrDefault(t => t.TaskId == taskId);

var currentRouteGroup = currentRouteStep.Document.DocumentType.Components.FirstOrDefault(t => t.Id == componentId);

The Components do not contain anything for me when running this code on EF Core 8. In EF Core 5, the necessary values were pulled up there. Have I already downloaded the required document data, should they have been locally when requested from the currentRouteStep?

When I put it away .Include(t => t.Document).ThenInclude(t => t.RouteStepValues) from the task request everything worked correctly.

I've done additional tests. The data in the collection is either there or missing, it's generally unclear why it depends. Sometimes they pull up normally, sometimes the collection is empty.

I also tried to get it directly from the document, the situation is similar.

var currentRouteGroup = document.DocumentType.Components.FirstOrDefault(t => t.Id == componentId);

Solution

It seems strange. On the surface it looks like you might be encountering an issue where in the first case where you eager-load Document as part of the Task without also eager loading the document type and components then perhaps EF's change tracker might have returned that tracked Document and ignored the loading of the DocumentType and Components as requested in the second query...

However, I build a test to check this particular scenario and EF still eager-loaded the requested data in the second query.

In my test case I had a Parent->Children->Comment relationship where I first queried a particular parent and their children without eager loading the comments, then I loaded a Child with their Comments eager loaded. I was suspecting that EF might have provided the tracked Child as it was loaded from the first query without processing the Include further, however it did reload the child and it's associated comment.

var test = context.Parents
    .Include(x => x.Children)
    .Single(x => x.ParentId == 1); // No comment loaded.
var test2 = context.Children
    .Include(x => x.Comment)
    .First(x => x.ChildId == 1); // a child from parent #1

If all you want is the DocumentId to use, it would be better to just project that and avoid loading any entities in the tracking cache in the first place:

var documentId = await _context.Tasks
    .where(t => t.Id == taskId)
    .Select(t => t.Document.ParentDocumentId.HasValue
         ? t.Document.ParentDocumentId.Value 
         : t.Document.Id)
    .SingleAsync();

That sounds like it would solve your problem, but it doesn't explain why your second query doesn't appear to be loading the related data if that Document happens to be eager loaded.

If it is a read-only operation where you aren't making changes to the document then you could try adding an .AsNoTracking() to the second query with your original first query and see if that changes anything. If the second query works with AsNoTracking then your particular scenario seems to be getting tripped up by EF's change tracker and it might be worth looking at your specific entity declaration in case you are doing anything "curious" that might explain it.. If it still fails with AsNoTracking then I highly suspect there is something odd in your entity configuration/implementation.

One culprit to check is how your navigation properties are declared to make sure they are kept simple and straight-forward, no "clever" logic, especially with collections. For instance:

// Document.DocumentType:

public virtual DocumentType DocumentType { get; protected set; }  

// DocumentType.Components:

public virtual ICollection<Component> Components { get; protected set; } = []; // or new List<Component>();

Setters on navigation properties should ideally be protected unless you want to allow values to be changed. Setters on collection navigation properties should always be protected as code should never re-initialize them once EF sets them up. If you have code that uses private members and conditional logic in getters/setters for collection navigation properties this could be screwing up EF's use of these properties.