node.jsmongodbsocket.iochangestream

Why is socket.io-mongo-adapter storing every heartbeat event in my socket.io-adapter-events collection?

I have created an application using nodejs and socket.io. As my backend is distributed across multiple server nodes, I have implemented the socket.io-mongo-adapter package by following the example provided by socket.io's documentation found here. I am using a capped collection to deal with the documents created by the adapter.

The application is working in its entirety, however, the socket.io-adapter-events collection seems to be storing every single heartbeat event from each of the connected clients. The collection is not growing to an exorbitant size because of the capped collection logic, however, the sheer number of events being constantly written is causing a significant load on the database. See below for an example of one of the Heartbeat events:

{ //one of MANY Heartbeat events
        "_id" : ObjectId("635b8e6788b91f23c78cafc5"),
        "type" : 2,
        "uid" : "24b54dc3747173cb",
        "nsp" : "/"
}

I went into the socket.io-mongo-adapter package, and I removed the two places where the package publishes EventType.HEARTBEAT Lines 248 and 421 in the github repository file. When I test this change locally, there are no issues, however I do not have a robust system for testing, or even understanding what this change does. Could someone please help inform if this change will cause unintended results? Is it important for these heartbeat events to be shared across separate server nodes? The documentation specifically states that

Every packet that is sent to multiple clients (e.g. io.to("room1").emit() or socket.broadcast.emit()) is:

A.) sent to all matching clients connected to the current server

B.) inserted in a MongoDB capped collection, and received by the other Socket.IO servers of the cluster

By my initial understanding, the heartbeat events were unimportant to other servers. It has been difficult to find clarity surrounding the importance/necessity of these events.

Thanks for your help in advance!

Solution

  • Upon further research, and some help from the maintainer of socket.io, I have learned the following:

    1. The heartbeats are generated per instance of the socket.io-adapter. The heartbeat notifications are used to keep each socket.io-mongo-adapter instance aware of the other socket.io-mongo-adapter instances' existence. Additionally, heartbeats should only trigger if the socket instance has not sent any other events within the past heartbeatInterval (default, 5 seconds).

    2. Certain functions in the socket.io-mongo-adapter need to know the list of active nodes. For example, fetchSockets() and broadcastWithAck(). If the heartbeat logic is not operating with its default behavior, it is possible that these methods will return stale, or incomplete data, or fail to send data to the complete list of intended recipients.

    3. For my specific use-case, the heartbeat functionality is unimportant and cumbersome, so to resolve this, I have increased the heartbeatInterval and heartbeatTimeout to exorbitantly large values for my application (30 and 60 days, respectively). This effectively prevents the heartbeat events from triggering. However, this may not be right for all use-cases. Carefully check if you are using any functions which rely upon each node being aware of the others.