mongodbperformancequery-optimizationreplicasetin-memory

Mongodb in memory


The goal is to optimize the read of near-static data. My protocol database has a few collections each of which with couple of thousands of records. The data in the protocol database rarely changes (say once a day via a web interface)

The problem: My messaging applications need to read relevant protocol data structures to process each message. In a sample flow, a message with message type "abc" comes in, the service runs circa 10 queries to find data relevant to message type "abc" from the protocol database and then it carries out the processing of the message. The search queries altogether take upto 800ms even after indexing and query optimzation. I need it to be much faster.

My perceived solution: What I am thinking is to create a replica set with a primary and secondary MongoDBs. The primary is in disk and the secondary is in memory (--storageEngine inMemory). And I set the applications to only read from the secondary. My assumption is that the secondary will be much faster due to being hold in memory. And being synced with the primary gives me assurance that the protocol data in the in-memory database stays uptodate.

What do you think?


Solution

  • Here is the solution:

    In this scenario, possible with MongoDB Enterprise Advanced, the primary node of the replica set uses an in-memory storage engine. It has two other nodes, one of which uses an in-memory storage, the other one using the WiredTiger engine. The secondary node using the disk storage is configured as a hidden member.

    In case of a failure, the secondary in-memory server would become the primary and still provide quick access to the data. Once the failing server comes back up, it would sync with the server using the WiredTiger engine, and no data would be lost.

    https://www.mongodb.com/databases/in-memory-database

    enter image description here