azuresnowflake-cloud-data-platformazure-data-factoryazure-cosmosdbazure-cosmosdb-mongoapi

How to copy JSON array columns from Azure Cosmos DB enabled Mongo DB document to Snowflake column (String/Variant) using Azure Data Factory V2?


I am trying to copy Azure Cosmos-DB enabled Mongo-DB collection Documents to the Snowflake table using Azure Data Factory V2. Some of the columns present in the Document are array (multiple values). I thought that: If I use String or Variant column at the destination "Snowflake", Azure Data Factory V2 will be able to copy it successfully but it is failing with error: (ErrorCode=JsonUnsupportedHierarchicalComplexValue,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The retrieved type of data JArray is not supported yet. Please either remove the targeted column 'REQUESTDATE' or enable skip incompatible row to skip the issue rows.,Source=Microsoft.DataTransfer.Common,'):

Mapping:

enter image description here

Error:

enter image description here

I have tried everything but was unsuccessful. I also noticed that even Data Flow does not support Mongo DB as a source. Any assistance will be appreciated.

Thanks, Bilal


Solution

  • I tried the same and also failed for the same JArray data type may be it is taking it as Array directly To overcome this issue You can use this following workaround: