sql-serverssisetlsql-server-2016data-profiling

Data profiling Task - custom Profile Request


Is there any option to create a custom Profile Request for SSIS Data Profiling Task?

At the moment there are 5 standard profile requests under SSIS Data Profiling task:

  1. Column Null Ratio Profile Request
  2. Column Statistics Profile Request
  3. Column Length Distribution Profile Request
  4. Column Value Distribution Profile Request
  5. Candidate Key Profile Request

I need to add another one (Custom one) to get summary of all numeric values.

Thanks in advance for your helps.


Solution

  • Based on this Microsoft Documentation, SSIS Data profiling Task has only 5 main profiles (listed on your question) and there is no option to add a custom profile.

    For a similar reason, i will create an Execute SQL Task to achieve that, you can use the aggregate functions you need and ISNUMERIC function in the where clause :

    SELECT MAX(CAST([Column] AS BIGINT)) -- Maximum value
           ,MIN(CAST([Column] AS BIGINT)) -- Minimum value
           ,COUNT(Column) -- Count values
           ,COUNT(DISTINCT [Column]) -- Count distinct values
           ,AVG(CAST([Column] AS BIGINT)) -- Average
           ,SUM(CAST([Column] AS BIGINT)) -- Sum
    FROM TABLE
    WHERE ISNUMERIC([Column]) = 1