csvapache-camelcamel-sqlcamel-spring-dsl

Load CSV to database using Apache Camel Spring XML (CSV to SQL components)


I want to do a simple file polling that takes in a CSV file, unmarshals it and loads specific fields into database. I guess that should be a pretty common scenario but I need to use Spring XML instead of creating a java processor. To my surprise I found hard to find any examples on that looking all around the internet. Probably I just didn't look in the right places but for what's worth, I'm sharing my question and my own answer in case someone else finds it useful.

Here's what I'm trying to achieve:

  1. Automatically pick a CSV file from a folder.
  2. Load specific fields from CSV into table

CSV looks like this

ID,KEY,FULLNAME,DOCID
1,1,PERSON1,THY
2,1,PERSON2,XCV
3,1,PERSON3,OIU
4,1,PERSON4,KJM

Which needs to be saved into table (only ID and FULLNAME fields):

PERSON

ID NAME
1 PERSON1
2 PERSON2
3 PERSON3
4 PERSON4

I'm using Camel 3.14.0.


Solution

  • Spring XML DSL route that worked for me:

      <route>
         <from uri="file://D:\path?include=(?i).*.csv&amp;moveFailed=ErrorFiles&amp;delay=5000"/>
         <unmarshal>
            <csv captureHeaderRecord="true" useMaps="true"/>  
         </unmarshal>
         <!-- Clear contents from destination table.  -->
         <to uri="sql:DELETE FROM PERSON?datasource=#customerDS&amp;noop=true"/> 
         <split> <!--Split unmarshalled body in individual maps that will be sent one by one to sql component -->
            <simple>${body}</simple> <!--As unmarshal is a List <Map> then this automatically gets list split in individual maps I guess -->
            <log message="Record being processed:  ${body}" loggingLevel="INFO"/>
            <to uri="sql:INSERT INTO PERSON(ID,NAME) VALUES(:#ID,:#FULLNAME)?dataSource=#customerDS" /> 
         </split>
      </route>