ProducerBlockedQuotaExceededError: Cannot create producer on topic with backlog quota exceeded

I have a Lucidworks Fusion 5 kubernetes installation setup on AWS EKS and currently one of the services, Connector Classic REST service, is experiencing an outage. After digging into the logs I found:

ERROR in com.lucidworks.logging.logback.PulsarAppender[PULSAR] - Failed to connect to Pulsar topic persistent://uat/_logs/system_logs at: pulsar://uat-pulsar-broker:6650 org.apache.pulsar.client.api.PulsarClientException$ProducerBlockedQuotaExceededError: Cannot create producer on topic with backlog quota exceeded

From here I took a look at the pulsar-broker configmap and found:

backlogQuotaDefaultLimitGB: "10"

I am not well versed in Apache Pulsar but it seems to me that if I try raising this backlog quota limit it should allow the Connector Classic REST service to connect to the broker again, which should bring the Connector Classic REST service back up. Is that a fair assessment ?

• Without changing any configuration is there a way to resolve this issue (e.g. restarting a particular service, ...) ?
• What is the best way to go about this ?

Thanks in advance.

In order to resolve this issue I followed these steps:

1. Shell into the pulsar-broker pod

2. Change directories into the /pulsar/bin directory

3. Use pulsar-admin CLI to find the subscription that needs to be cleared

   ./pulsar-admin topics subscriptions <topic>

4. Clear the backlog with the following command

   ./pulsar-admin topics clear-backlog <topic> -s <subscription>

5. Shell out and delete the Connector Classic REST pod

6. After a few minutes the service comes back up