aerospikeaerospike-ce

Aerospike BatchRead Api find source/originating Set for records in response


Aerospike Batch read api takes an array of sets as input like this:

        final Key[] keys1 = new Key[3];
        keys1[0] = new Key(config.getNamespace(), config.getSetName(1)), key);
        keys1[1] = new Key(config.getNamespace(), config.getSetName(2)), key);
        keys1[2] = new Key(config.getNamespace(), config.getSetName(3)), key);

And returns response as a array of records (Record[3]), with elements from the same set in the recordList in correct order if all 3 sets contain the key.

If one of the sets doesn't have the key present, then I get a response as Record[2] and no way to tie the elements to the originating set.

Is there a way around this? My sets are grouped hourly/daily. I want to make only one call to aerospike, and be able to distinguish data between sets.

I don't want to make composite keys(huge no. of keys: 10^9) and also cannot make multiple namespaces due to the query presumably becoming expensive.


Solution

  • I noted you requested me to comment. I modeled your problem in Jupyter Notebook. Let me post the relevant code segments. As @kporter said, for records not present, it returns null.

    Key key1 = new Key("test", "testSet1", 1); 
    Key key2 = new Key("test", "testSet2", 2);  // We will not create this record 
    Key key3 = new Key("test", "testSet3", 3); 
    Key key4 = new Key("test", "testSet4", 4);
    Bin b1 = new Bin("bin1", 100);
    Bin b2 = new Bin("bin1", 200);
    Bin b3 = new Bin("bin1", 300);
    Bin b4 = new Bin("bin1", 400);
    client.put(null, key1, b1);
    //client.put(null, key2, b2); //Skipping so its a null record
    client.put(null, key3, b3);
    client.put(null, key4, b4);
    

    This creates 3 records at keys key1, key3 and key4. Now lets do a batch read:

    BatchPolicy batchPolicy = new BatchPolicy();    
    Record [] bReads = client.get(batchPolicy, new Key[] {key1, key2, key3, key4});    
    // process the batch reads
    for (int i = 0; i < bReads.length; i++) {
        Record bRead = bReads[i];    
        if (bRead != null) {          // check individual record
            long bin1Val = bRead.getLong("bin1");         
            System.out.format("Result[%d]: bin1: %d\n",i, bin1Val);
        }
        else {   // error in individual key's operations
            System.out.format("Result[%d]: not found: \n", i);
        }
    }
    

    Output is:

    Result[0]: bin1: 100
    Result[1]: not found: 
    Result[2]: bin1: 300
    Result[3]: bin1: 400