redispublish-subscribe

Many channels with few publications or few channels with many publications in Redis PubSub?


I am using Redis PubSub for a notification system in an application. I usually have around 1000 concurrent users. For each user I have a personal channel with the user's ID through which several personal notifications are sent. But also, each user is related to several "personal tasks" (they are personal for each user, approximately 15 on average per user) so I have two options:


Solution

  • As long as you using a single call to SUBSCRIBE, I don't think it really matters.

    If you using a single channel, you'll have to put some sort of marker in the message for the type (user, task, etc.) and then write code to process that and route the message to the right place.

    If you have multiple channels, that will be expressed in the channel name but you'll still need to write code to read that and route the message.

    The thing you should not do is use multiple channels—each with their own connection to Redis that subscribes to a single channel. That would result in ~15,000 simultaneous connections to Redis, which is a lot.

    Here's some info from the redis.conf file about maxclients that is relevant to this:

    # Set the max number of connected clients at the same time. By default
    # this limit is set to 10000 clients, however if the Redis server is not
    # able to configure the process file limit to allow for the specified limit
    # the max number of allowed clients is set to the current file limit
    # minus 32 (as Redis reserves a few file descriptors for internal uses).
    #
    # Once the limit is reached Redis will close all the new connections sending
    # an error 'max number of clients reached'.
    #
    # IMPORTANT: When Redis Cluster is used, the max number of connections is also
    # shared with the cluster bus: every node in the cluster will use two
    # connections, one incoming and another outgoing. It is important to size the
    # limit accordingly in case of very large clusters.
    #
    # maxclients 10000
    

    Hope this helps and best of luck!