We have 250 powerbi reports build on top of Azure Synapse, now we are migrating from Azure Synapse to Databricks (DB SQL). How to plan for cutover and strategy for PowerBI
I just seeking high level points we have to take care for planning. Any techie who did this in past, kindly share their design approach.
I got some high level points like :
But these are very generic.
Please help
As you mentioned you atre migrating from Azure synapse to azure databricks below are some strategy
You can migrate data from Azure Synapse Analytics to Databricks in a few different ways:
Using Azure Data Factory (ADF): Set up a pipeline to copy data from Synapse SQL to Azure Data Lake Storage (ADLS), then load it into Databricks. Using Direct Query: Databricks has built-in connectors that let you directly query and transfer data from Synapse, which works well for real-time processing. Using Delta Lake: Convert your Synapse data into Delta format in Databricks to take advantage of ACID transactions and faster query performance.
Note: For handling large amounts of data and frequent transfers, it's best to use Azure Data Factory along with Delta Lake in Databricks. This setup is both scalable and efficient.
Also You can use Azure Databricks to pull data from Azure Synapse using Apache Spark. With its built-in connectors, Databricks can read directly from Synapse and then write the data to Azure Data Lake Storage (ADLS).