javascriptnode.jsrabbitmqamqp

RabbitMQ: Delete a queue if consumer is gone forever?


Imagine each user has it's own queue. Messages are put in to the queue when the user is offline and when they reconnect, they receive all those messages.

However, what if the client leaves and never comes back (stops using the service)? Does RabbitMQ have a way to delete a queue if messages haven't been consumed in a while? Messages might be coming in, but they aren't going out because the client is no longer active. How could I handle this with the amqp.node plugin?


Solution

  • Imagine each user has it's own queue. Messages are put in to the queue when the user is offline and when they reconnect, they receive all those messages.

    this is a common idea, but one with unfortunate consequences, that you should avoid.

    as i wrote in that linked article:

    You could potentially work around this by having a queue per user. You could set up your code so that the web server only subscribes to a queue for the user when that user is logged in and available.

    This doesn’t solve the problem.

    Now you have 30,000 messages distributed across 1,000 queues – a bad situation in itself. This requires RabbitMQ to persist the queues and the messages within the queues. And what happens when all 1,000 users log in to your system at the same time? Now your web server has 1,000 queue subscriptions to deal with.

    sure, you could argue that you won't have that many users. but it gets worse. in your situation, you won't be able to delete the queue the way you want, automatically.

    you can set a queue to automatically delete once all consumers are disconnected. but keeping the queue around would require a consumer per user queue, where the consumer is connected for as long as that user exists. if the application crashes, the queue and all of it's messages would disappear.

    TTL will only do part of what you want. it will put a life on the message in the queue and the message will be removed after that timeout... but it won't delete the queue.

    ultimately, you don't want to do this with RabbitMQ.

    again, from my article on the subject:

    Getting back to the original scenario, what should you do when the user is not online and you can’t send the message to them immediately?

    Store the message in a database.

    Add a field to the database record that says who this message belongs to. When the user reconnects later, query the database for any messages that this user needs to see and send them along at that time.

    The full process started above, then becomes this:

    • User’s browser connects to SignalR/Socket.io/Pusher/websockets on web server
    • Web server checks a queue for updates that happen during a long running process
    • When a message for a logged in user comes in
      • If the user is logged in, broadcast the message through the websocket to the user
      • If the user is not logged in, store the message in a database
    • When the user logs in again, query the database and send all waiting messages

    It’s what you would have done before the idea of a message queue came in to play, right? It should be what you would do now that you have a message queue, as well.