I need to store a large number of Keys in an shared memory Aerospike Primary Index. My specs are as follows: Namespaces : 2, Number of Records : 500 M, TTL : 1D, Replication: 2
According to Aerospike Capacity planning: By String Key should need 64 bytes × (replication factor) × (number of records)
= 64 * 2 * 0.5 * 10^9 = 64 GB of space per namespace.
Question is the the index separate for every set? For EG, if I have 20 sets per namespace, do I need 20 * 64 gigs in memory for that namespace?
The primary index is not separate for every set.
According to https://aerospike.com/developer/blog/of-queries-and-indexes, "In Aerospike, there is only one system-defined primary index for each namespace" which makes sense because you can change the way you store your primary index - shared memory (shmem), persistent memory (PMem) and flash (NVMe SSDs) at the namespace level.
Set is an optional unit to group records, its the equivalent of a table but without an explicit schema, for sets you have a dedicated optional Set Indexes which allows you to index the membership of records to their set.