databaseshardinginfinispancapedwarf

Infinispan data persistence sharding?


Capedwarf uses Infinispan to store data, in what way can Infinispan be configured to persist data on a node machine with maximum disk space, for example, each server hosting Capedwarf only have 1TB of mounted block storage, how do you configure Infispan such that if the "overall" data exceeds 1TB it would be "sharded" across different server?

Running Capedwaft it stores infinispan data in: $\CapeDwarf_WildFly_2.0.0.Final\standalone\data\infinispan\capedwarf


Solution

  • When using local storage (single file store, soft index store or rocksdb store) in combination with a distributed cache, the data is already "sharded" based on ownership: each node will approximately store TOTAL DATA / NUM_NODES * NUM_OWNERS. For example, store 1GB of data on a 5-node cluster in a distributed cache with 2 owners (the default), each node would require approximately 400MB. As the data is not perfectly balanced, allow for a certain margin of difference (typically 10-15%). Alternatively you can use a shared store (jdbc, cloud) which would store data externally.