javaamazon-web-servicesamazon-dynamodb

Where is routing implemented (Request routing to the different replicas) when making request to the DynamoDB server?


DynamoDB has multiple nodes and all nodes are equal: there is no single leader. I am assuming we have a DynamoDB instance and all the data can be stored on a single node. But we have several replicas for availability and redundancy.

So writing and reading can happen on any replica node. I am just curious, how DynamoDB client decides to which node the request should be sent? Or the application developer has to take care of this routing themselves and the DyanmoDB client simply forwards these requests to the DB node.

If someone could explain to me where this routing happens and could point me to the code where this routing is implemented, that would be a great help?

I am new to the DynamoDB and trying to understand how things work.


Solution

  • Fundamentally this is an implementation detail that AWS handles - you won't find it in your Java code.

    DynamoDB has multiple nodes and all nodes are equal: there is no single leader.

    This is a misconception, the table is split up into partitions and each partition consists of three storage nodes. For each partition a storage node is selected that is the primary contact point for writes.

    A few years ago there was an interesting talk by Jason Sorensen about the internals of DynamoDB at re:invent, you can see it on YouTube.

    To summarise it, there is a fleet of request router instances that are aware of the storage nodes and how the hashing works. The storage router selects the leader node of the partition and writes data to it. As soon as two of the storage nodes acknowledge the write, the data is considered to be persisted.