powerbiazure-data-factoryazure-data-lakeazure-synapsewm-copydata

Synapse CopyData Tool Changing datetime to int96


So I'm pulling items from a database via a query to put that data into a datalake. All that works great, the preview comes out wonderfully on the source tab. On the Mapping tab however, Whenever I hit "Import Data" or even input the "StartDate" column manually to a blank mapping (which is a datetime in the source DB) -- it changes the startdate to int96. Then, once I pull this data into PBI obviously I have to do a BUNCH of weird massaging to get the int96 back to a datetime. It's rediculous.

Here is a pic of what's happening.

Does anybody know why this is happening or what I can do to map the sink column as a datetime? I can't seem to change the type anywhere.

MappingTabe and Datetime Change


Solution

  • Parquet internally stores dates as integers, but the clients, including Power BI should automatically convert them back to dates. EG this works fine for me, with a parquet file created as you indicate.

    let
        Source = AzureStorage.DataLake("https://xxxx.dfs.core.windows.net/datalake/stage/xxx.parquet"),
        f = Source{[#"Folder Path"="https://xxxx.dfs.core.windows.net/datalake/stage/",Name="xxx.parquet"]}[Content],
        #"Imported Parquet" = Parquet.Document(f)
    in
        #"Imported Parquet"