data processing pauses for the consumer

We have a system that pushes events onto an Azure event hub, and then another component in the system processes these events from the hub. The application is written in node.js and uses the standard Azure EventHub libraries.

On occasions, we see an issue where the producer is successfully pushing events onto the event hub, but they aren't appearing to the consumer immediately. No events appear for a period (normally somewhere up to a couple of minutes) then all the events become available, and the consumer has to process the backlog. We have set the maxWaitTimeInSeconds to 1, but when this issue occurs, the processEvents doesn't get called for a considerably longer period.

The graph below from Azure shows the "pause" that is appearing where the events aren't being processed and so build up. Then the consumers catching up.

How can we prevent this from occurring? We have a real-time system which is displaying this data, and so we can't have pauses like this in the data.

This was resolved by upgrade to 5.11.3 of the azure-event-hubs package.