entity-framework-core-3.1

The required column 'xxxID' was not present in the results of a 'FromSql' operation - .Net Core EF


I am trying to find better and performance efficient approach for bulk delete in .NET Core EF (3.1.9). (Approx 500K to 1 Mil records to be deleted in one shot.)

Model:

public class Employee
{
    [Key, DatabaseGenerated(DatabaseGeneratedOption.Identity)]
    public int EmpID { get; set; }
    public string EmpName { get; set; }
}

And database table as:

CREATE TABLE [dbo].[Employee]
(
    [EmpID] [int] IDENTITY(1,1) NOT NULL,
    [EmpName] [nchar] (20) NULL,

    CONSTRAINT [PK_dbo].[Employee] 
        PRIMARY KEY CLUSTERED ([EmpID] ASC)
                    WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, 
                          IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, 
                          ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]

Trying to delete records with generic method-1.

public int Delete<TEntity>(Func<TEntity, bool> predicate) where TEntity: class
{
    return DbContext.Set<TEntity>
                    .FromSqlRaw($"Delete from dbo.Employee")
                    .Where(predicate).Count();
}

And calling this method as

Func<Employee, bool> myPredicate = x => x.EmpID > 10;
int deletedCount = myclass.Delete(myPredicate);

Exception thrown:

InvalidOperationException: The required column 'EmpID' was not present in the results of a 'FromSql' operation.

ASP.NET Core EF generates query:

DELETE FROM [dbo].[Employee]

fail: Microsoft.EntityFramework.Query[10100] .... stack trace with above error.

Already looked into this:

.NET Core Entity Framework InvalidOperationException

The required column 'CustomerId' was not present in the results of a 'FromSql' operation

The required column 'id' was not present in the results of a `FromSql` operation in EFcore

So not sure why it is throwing above error. As database has the correct PK and model also has it. Tried with some other entities too, it always throws the same error with column name 'xxxEntityColID'.

However, if I use following code then it works:

public int Delete<TEntity>(string whereCondition, params object[] parameters) where TEntity: class
{
    return DbContext.Database.ExecuteSqlRaw($"Delete from dbo.Employee WHERE {whereCondition}", parameters);
}

// and calling like 
string myCondition = "EmpID > 10";
int deletedCount = myclass.Delete<Employee>(myCondition, new object[0]);

.NET Core EF generates following SQL and work without any error.

DELETE FROM [dbo].[Employee] WHERE EmpID > 10;

Questions


Solution

    1. Looks like FromSqlRaw() is worked with "SELECT.." queries.
    2. For Bulk delete I evaluated following approach and it is better than dbContext.RemoveRange(entities); Because RemoveRange() generated separate query for each entity. Then every single query takes some time, so not suitable for Bulk delete. In contrast ExecuteSqlRaw(with where condition), just generates single query which hardly takes 250ms for 200K records. On high end servers it will take lesser for sure.
    public int Delete<TEntity>(string whereCondition, params object[] parameters) where TEntity: class
    {
        return DbContext.Database.ExecuteSqlRaw($"Delete from dbo.Employee WHERE {whereCondition}", parameters);
    }