talendtalend-mdm

Loading data based on condition of target table in talend


I have a table(target) which has 5 rows, one of the columns name:slno have values (12,13,14,34,56), I need to load data from my source table to target based on the max value of target.

Example :

If in the source table for slno column values are (12,13,14,34,56,88,89,90,99) then only (88,89,90,99) values should go to target (along with all row values), basically I need to find max from target and based on that I need to load rows after that value.

I tried using tJavaRow, tSetGlobalVar, tAggregateRow, but not able to figure out how to map.


Solution

  • There are many ways you can do this.

    If your source and target tables are on the same database, you can filter your source query like this:

    select *
    from source
    where slno > (select max(slno) from target)
    

    And then load the rows in your target table.

    But if they are not, you can do it in Talend :

    enter image description here

    The lookup on target gets the max value of slno :

    SELECT max(slno)
    FROM target
    

    Its schema contains only one column (max_slno):

    enter image description here

    And inside the tMap, only send the rows where the source's slno is greater than the maximum value of target's slno :

    enter image description here