rabbitmqmasstransittopshelf

MassTransit/RabbitMQ: one service or multiple services


I have around 300 different consumers / 300 message types / 300 queues with the most wildest functionality behind it.

From the extreme side:

? update: from 1 to 300 queues


Solution

  • RabbitMQ can support hundreds of queues simultaneously, and each queue should be responsible for one specific type of message e.g. a response status or an online order information or a stack trace information for further processing by some other unit of work, these three are not same and if you are keeping them all in one then please segregate them into different queues.

    If you will keep all the data in one queue it will also effect your application performance as each queue works in a sequential order and since you have 300 consumers that wait for 300 types of messages, almost all of them could be in waiting state and it is also a reason for complex decision making algorithm, if you are using one to figure out the correct consumer.

    What could also go wrong with a single queue is that it is now a bottleneck which could obstruct the functioning of the whole application, if that queue fails, because every consumer listens to it. By having different queues the rest of the system can still process if one particular queue faces an issue.

    Instead of going for 1 consumer per service you can check if there's anything common and if the services can take up more consumers than one after increasing the number of queues from 1 to many.