elasticsearchapache-nifi

How to Delete Data from Elasticsearch Using Apache NiFi's PutElasticsearchHttp Processor?


I'm using the putelasticsearchhttp processor in Apache NiFi to insert data into Elasticsearch by providing the Elasticsearch URL. Now, I need to delete and update data in Elasticsearch using the same processor.

When I set the index operation parameter to delete in the PutElasticsearchHttp processor (with below configuraion), I encounter the following error:

error I'm facing

Questions:

  1. Is it possible to delete data from Elasticsearch using the PutElasticsearchHttp processor in Apache NiFi? If yes, how can I correctly configure it?
  2. If it is not possible with the PutElasticsearchHttp processor, which NiFi processor should I use to delete data from Elasticsearch, and how should I configure it?

Note:

Here I'm using elasticsearch url to access elastic search and it is not running in my local.

Any guidance on how to delete or update data in Elasticsearch using Apache NiFi, with a focus on using the Elasticsearch URL, would be greatly appreciated.


Solution

  • For delete operations, the processor needs to know the identifier of the document in order to delete it. It uses the Identifier Attribute property to determine the identifier of the document to delete. Ensure your FlowFile has an attribute (let's call it es.doc.id) and it contains the identifier of the document to delete. Then set the value of the Identifier Attribute property in PutElasticsearchHttp to es.doc.id.

    Please note that in newer versions of NiFi, PutElasticsearchHttp has been deprecated in favor of PutElasticsearchJson, and thus it may be removed in a future (major) release.