queueapache-nifi

NiFi how to clear all queues


I'm currently using NiFi 1.5.0 (but it's the same with the previous versions) and I wonder if there is a way to clear all queues in the same time ?

When the number of processors increase, it can be really long to reset everything.

(I already know how to clear a single queue : How to clear NiFi queues? )

I'm looking for a solution using either the UI or the API

Thanks in advance !


Solution

  • I haven't had time to test this thoroughly, but it should work:

    # In your linux shell - NiPyAPI is a Python2/3 SDK for the NiFi API
    pip install nipyapi
    python
    # In Python
    from nipyapi import config, canvas, nifi
    # Get a flat list of all process groups
    pgs = canvas.list_all_process_groups()
    # get a flat list of all connections in all process groups
    cons = []
    for pg in pgs: cons += nifi.ProcessgroupsApi().get_connections(pg.id).connections
    # Issue a drop order for every connection in every process group
    for con in cons: nifi.FlowfilequeuesApi().create_drop_request(con.id)
    

    Edit: I went ahead and implemented this as it seems useful: https://github.com/Chaffelson/nipyapi/issues/45

    import nipyapi
    pg = nipyapi.canvas.get_process_group('MyProcessGroup')
    nipyapi.canvas.purge_process_group(pg, stop=True)
    

    The stop option will deschedule the Process Group before purging it, just to be extra handy