datastage

Separate Numeric Range value from single row to individual rows per value in range


I have a CSV with one field that has numerical values as strings. A few of these values are a numeric range [e.g. 30-34]. For rows that have a range value in this field I need to split the range values into separate individual rows with the unique value from the range. I do not know how to do this, nor how to word this to search for the appropriate answer :-(

I have done some basic research which led to the pivot stage but the instructional videos were more for reducing horizontal columns to rows. I originally thought that the Transform stage loop might work but I was not able to determine the logic required.


Solution

  • Using a Transformer stage and loop condition was the correct answer.

    My file contains a field we'll call Code. In that field are mostly numerical values, but sometimes there are numerical ranges like '33-35' to express the same other field values for all of those codes in the range. This is how the file is created by the vendor. For our use, we need all the unique codes represented with their own records.

    Step 1: Create a Stage Variable

    First, I created a stage variable named NumOfCodes and used the following derivation to calculate the number of individual values represented:

    IF Index(DSLink15.Code, '-', 1) > 0 THEN 
        (Field(DSLink15.Code, '-', 2) - Field(DSLink15.Code, '-', 1) + 1) 
    ELSE 
        1
    

    Step 2: Configure the Loop Condition

    In the Loop condition, I set the while derivation to:

    @ITERATION <= NumOfCodes
    

    This ensures the loop will iterate for each code value.

    Step 3: Create a Loop Variable

    I created a loop variable called DerivedCode and used the following derivation to calculate the correct code value for the given iteration:

    IF Index(DSLink15.Code, '-', 1) < 0 THEN 
        DSLink15.Code 
    ELSE 
        IF @ITERATION = 1 THEN 
            Field(DSLink15.Code, '-', 1) 
        ELSE 
            (Field(DSLink15.Code, '-', 1) + (@ITERATION - 1))
    

    Step 4: Populate the Output Field

    Finally, I used the loop variable DerivedCode to populate my output Code field. strong text