[SOLVED] Aerospike List operation : how to delete one item from all list bins in a set

Aerospike List operation : how to delete one item from all list bins in a set

I have a Set that have two Bins(pId is long , values is list) and list index is applied on values. Now i want to remove specify value from values list of all records.

I used ListOperation.removeByValue and it's working but it's not fast when records size is more than 100000 records.it's take about 11 seconds to do loop operation.

WritePolicy clientWritePolicy = new WritePolicy();
clientWritePolicy.commitLevel = CommitLevel.COMMIT_MASTER;
clientWritePolicy.recordExistsAction = RecordExistsAction.UPDATE;
clientWritePolicy.expiration = 300;
              
Statement statement = new Statement();
statement.setIndexName("values_index");
statement.setNamespace("test");
statement.setSetName("setTest");
               
statement.setFilter(Filter.contains("values", IndexCollectionType.LIST, value));
QueryPolicy queryPolicy = new QueryPolicy();
RecordSet recSet = client.query(queryPolicy, statement);
    
recSet.forEach(rec->{
   client.operate(clientWritePolicy, rec.key,ListOperation.removeByValue("values",Value.get(value), ListReturnType.NONE));
   });

How can i optimize this code?is it any other solution like use UDF?

update 1: I wrote below function in lua file:

function removeListByValue(rec,v)
    local l = list()
    for value in list.iterator(rec["values"]) do
        if(value ~= v) then
             list.append(l, value)
        end
     end
     rec["values"] = l;
     aerospike:update( rec );
end

and run it from java client :

WritePolicy clientWritePolicy = new WritePolicy();
clientWritePolicy.commitLevel = CommitLevel.COMMIT_MASTER;
clientWritePolicy.recordExistsAction = RecordExistsAction.UPDATE;
clientWritePolicy.expiration = 300;

Statement statement = new Statement();
statement.setIndexName("values_index");
statement.setNamespace("test");
statement.setSetName("setTest");

statement.setFilter(Filter.contains("values", IndexCollectionType.LIST, value));
client.execute(clientWritePolicy, statement, "luaPackage", "removeListByValue",Value.get(value));

it seems that UDF takes more time to update or remove records.it's about 29 seconds for 200 records.

Solution

You can apply operations (your ListOperation) in execute instead of using a UDF. You can specify operations in the Statement object using setOperations() and then just use execute(wPolicy, statement).

e.g.
Operation[] operations = new Operation[1];
operations[0]= ListOperation.removeByValue(.... 
stmt.setOperations(operations);
WritePolicy wPolicyScan = new WritePolicy();
ExecuteTask et = client.execute(wPolicyScan, stmt)

Java API:

execute(WritePolicy policy, Statement statement, Operation... operations) Apply operations on records that match the background query statement filter