azure-data-lake-gen2azure-data-factoryexpressionbuilder

Data Flow Partition by Column Value Not Writing Unique Column Values to Each Folder

I am reading an SQL DB as source and it outputs the following table.

enter image description here

My intention is to use data flow to save each unique type into a data lake folder partition probably named as specific type.

I somehow manage to create individual folders but my data flow saves the entire table with all types into each of the folders.

my data flow

enter image description here

Source

enter image description here

Window

enter image description here

enter image description here

Sink enter image description here

enter image description here

Any ideas?

Solution

  • I create a same csv source and it works well, please ref my example.

    Windows settings: enter image description here enter image description here

    Sink settings: choose the file name option like this enter image description here

    Note, please don't set optmize again in sink side.

    The output folder schema we can get:

    enter image description here

    Just for now, Data Factory Data Flow doesn't support custom the output file name.

    HTH.