How to BULK Read from Azure SQL server and BULK INSERT to Cassandra in Camel

I want to read 5+ million events from an Azure SQL DB table and perform a BULK INSERT to Cassandra. The table has 2 columns. I see the SQL component available for reading from Azure SQL DB. https://camel.apache.org/components/3.7.x/sql-component.html

Question: Consuming from Azure SQL DB

1. Is there a better way to read all the rows and store in a map considering 5M records ?
2. Is there a possibility to read messages in batches ?

There is a cql component available for Cassandra https://camel.apache.org/components/3.7.x/cql-component.html

Question: Producing to Cassandra

1. Can we INSERT in batches ?

Can I use camel for this use case ?

For each table in your Azure database, we recommend that you export the data into a CSV file. There are lots of tools and methods available that would allow you to do that. For example, have a look at Import and export data from Azure SQL Database.

Once you have exported your data to CSV, you can use the DataStax Bulk Loader tool (DSBulk) to bulk load it to a Cassandra table.

Here are some references with examples to help you get started quickly:

• Blog - DSBulk Intro + Loading data
• Blog - More DSBulk Loading examples
• Blog - Counting records with DSBulk
• Docs - Loading data examples
• Answered questions - DS Community

DSBulk is open-source so it's free to use. Cheers!