apacheapache-nifihortonworks-data-platformhortonworks-dataflowapache-minifi

How to automate DBCPConnectionPool Controller service in Apache NiFi?


I am new to Apache NiFi and my basic task to ingest data from multiple database sources into HDFS using Apache Nifi. I use 'DBCPConnectionPool' Controller service for database connection pooling service, where I need to provide information such as 'Database Connection URL', 'Database Driver Class Name', 'Database Driver Location', 'Database User', 'Passoword' etc. Now, since I am ingesting from multiple data sources, so everytime the source database changes I manually need to change all the above mentioned information. I am looking for a way so that we can automate this task. For example, The 'DBCPConnectionPool' controller service should be able to dynamically get the 'Database Connection URL','Database Driver Class Name' and other informations depending upon the source database.


Solution

  • @Rishab Prasad,

    NiFi supports RestAPI for dynamically change the value 'DBCPConnectionPool' properties.

    Check below reference but for this you need to create empty cotroller service in an required processors.

    https://nifi.apache.org/docs/nifi-docs/rest-api/index.html

    If you created empty controller service then that having seperate id. with help of above rest api for update the controller services you can dynamically change its values.

    This is only way to dynamically change the DBCP Connection pool.

    Otherwise you need to change the nifi source to achieve your requirment.