luauser-defined-functionsaerospikeaerospike-ce

UDF Write Generation Check within UDF context


I Have Use case for Aerospike where I have multiple Records like:

+--------+---------+
| PK     | signal  |
+--------+---------+
| 123451 | 1       |
| 102221 | 1.0816  |
+--------+---------+

I have Service A running on VM fleet of ~1000 Vms, which read this signal value every 3 mins. Now this signal value also needs to be updated every 3 mins based on other real world values (x...y) in the scope of Service A. It can either go up, down every 3 mins, and also reset every 24 hrs. Also all 1000 vms need to see the same updated value of signals at the same time.

Now the ideal way might be to have a separate service (Service B) operating that updates this signal every 3 mins. But the dynamic values are difficult to recreate in any other service, and are not easily exported.

So to do it in the Service A, I can follow a CAS(Check and Set) pattern, at every VM Where I have a scheduler doing this every 3 mins interval synced to epoch:

  1. Read the Signals
  2. Compute updated signals
  3. Write back new compute Signals

Now this is inefficient because best case all 1000 Service A vms will be writing back the same value to aerospike, ranging to worst case where I may have

  1. Write contention(Key busy) due to 1000 concurrent writes
  2. Different signals being written back due to edge cases of different dynamic values of X and Y in different contexts.

Thought about using Write policy generation to achieve this where:

  1. I read all keys, preserve generation of all keys
  2. Compute all new keys.
  3. Write all new keys back, with match generation check on write policy
  4. Raad back final updated key (Is this needed?)

Looks clumsy, so tried an UDF like this:

function updateSignal(rec, binName, someParamX, someParamY, hyperParameter, resetFlag)
    if aerospike:exists(rec) then
      local currentSignal = rec[binName]
      local geneneration = record.gen(rec)
      if (someParamX > someParamY) then
        local multiplier = someFunction()
        local updatedSignal = currentSignal * multiplier
        rec[binName] = updatedSignal
      else
        rec[binName] = 1
      end  
      aerospike:update(rec)
    else
      aerospike:create(rec)
      rec[binName] = 1
      aerospike:update(rec)
    end
    return rec[binName]  -- Return the updated value
end

Problem is that in the UDF context I don't really have any write gen checking ability, if I don't pass the write gen initially to the UDF.

Also if I pass the write gen as part of the UDF call, I also need to return back the current value of the signal if the write fails due to gen check(collisions). Since I wont have the updated signal once inside the udf context if the record was updated from elsewhere. (Can this even happen? Read somewhere udfs lock the record? is it a Read lock/write lock?)

Any way around this? Am I missing something?


Solution

  • The record UDF will be running entirely while holding a lock on the record. You therefore cannot have a record changed by another VM while inside a UDF that would read and then update the record. Hope this helps (I may have missed some subtleties of your specific use case).