sql-servernotindynamics-business-central

NOT IN statement is slowing down my query


I have a problem with my query. I have a simple example here that illustrates the code I have.

SELECT distinct ID 
FROM Table  
WHERE IteamNumber in (132,434,675) AND Year(DateCreated) = 2019
      AND ID NOT IN (
                     SELECT Distinct ID FROM Table  
                     WHERE IteamNumber in (132,434,675) AND DateCreated < '2019-01-01')

As you can see, I'm retrieving unique data id's that has been created in 2019 and not earlier.

The select statements works fine, but once I use the NOT IN statement, the query could easily go 1 minute plus.

My other question could this be related to the computer/server performance that is running the SQL Server for Microsoft Business Central? Because the same query worked perfectly after all even with the (NOT IN) statement, but that was in Microsoft dynamics C5 SQL Server.

So my question is there something wrong with my query or is it mainly a server issue?

UPDATE: here is a real example: this takes 25 seconds to retrieve 500 rows

Select count(distinct b.No_),'2014'
from [Line] c    
inner join [Header] a
on a.CollectionNo = c.CollectionNo
Inner join [Customer] b
on b.No_ = a.CustomerNo

where  c.No_ in('2101','2102','2103','2104','2105')
and year(Enrollmentdate)= 2014 
and(a.Resignationdate < '1754-01-01 00:00:00.000' OR a.Resignationdate >= '2014-12-31')


and NOT EXISTS(Select distinct x.No_
                 from [Line] c    
                 inner join [Header] a
                 on a.CollectionNo = c.CollectionNo
                 Inner join [Customer] x
                 on x.No_ = a.CustomerNo
                 where x.No_ = b.No_ and 
                       c.No_ in('2101','2102','2103','2104','2105')
                       and Enrollmentdate < '2014-01-01'
                       and(a.Resignationdate < '1754-01-01 00:00:00.000' OR a.Resignationdate > '2014-12-31'))

Solution

  • I usually prefer JOINs than INs, you can get the same result but the engine tends be able to optimize it better.

    You join your main query (T1) with what was the IN subquery (T2), and you filter that T2.ID is null, ensuring that you haven't found any record matching those conditions.

    SELECT distinct T1.ID 
    FROM Table T1 
         LEFT JOIN Table T2 on T2.ID = T1.ID AND 
                         T2.IteamNumber in (132,434,675) AND T2.DateCreated < '2019-01-01'
    WHERE T1.IteamNumber in (132,434,675) AND Year(T1.DateCreated) = 2019 AND
          T2.ID is null
    

    UPDATE: Here is the proposal updated with your real query. Since your subquery has inner joins, I have created a CTE so you can left join that subquery. The functioning is the same, you left join your main query with the subquery and you return only the rows with no matching records found on the subquery.

    with previous as (
      Select x.No_
      from [Line] c    
           inner join [Header] a on a.CollectionNo = c.CollectionNo
           inner join [Customer] x on x.No_ = a.CustomerNo
      where     c.No_ in ('2101','2102','2103','2104','2105')
            and Enrollmentdate < '2014-01-01'
            and (a.Resignationdate < '1754-01-01 00:00:00.000' OR a.Resignationdate > '2014-12-31'))
    )
    Select count(distinct b.No_),'2014'
    from [Line] c    
         inner join [Header] a on a.CollectionNo = c.CollectionNo
         inner join [Customer] b on b.No_ = a.CustomerNo
         left join previous p on p.No_ = b.No_
    where    c.No_ in ('2101','2102','2103','2104','2105')
         and year(Enrollmentdate)= 2014 
         and (a.Resignationdate < '1754-01-01 00:00:00.000' OR a.Resignationdate >= '2014-12-31')
         and p.No_ is null