erlangmnesiamochiwebyawsnitrogen

how do we efficiently handle time related constraints on mnesia records?

i am writing records into mnesia which should be kept there only for an allowed time (24 hours). after 24 hours, before a user modifies part of them, the system should remove them automatically. forexample, a user is given free airtime (for voice calls) which they should use in a given time. if they do not use it, after 24 hours, the system should remove these resource reservation from the users record.

Now, this has brought in timers. an example of a record structure is:

 -record(free_airtime,
            {
                reference_no,
                timer_object,   %% value returned by timer:apply_after/4
                amount
            }).

The timer object in the record is important because in case the user finally puts to use the resources reserved before they are timed out (or if they time out),the system can call timer:cancel/1 so as to relieve the timer server from this object. Now the problem, i have two ways of handling timers on these records:

Option 1: timers handled within the transaction

 reserve_resources(Reference_no,Amnt)->
    F = fun(Ref_no,Amount) -> 
            case mnesia:read({free_airtime,Ref_no}) of
                [] -> 
                    case mnesia:write(#free_airtime{reference_no = Ref_no,amount = Amount}) == ok of
                        true -> 
                            case timer:apply_after(timer:hours(24),?MODULE,reference_no_timed_out,[Ref_no]) of
                                {ok,Timer_obj} -> 
                                    [Obj] = mnesia:read({free_airtime,Ref_no}),
                                    mnesia:write(Obj#free_airtime{timer_object = Timer_obj});
                                _ -> mnesia:abort({error,failed_to_time_object})
                            end;
                        false -> mnesia:abort({error,write_failed})
                    end;
                [_] -> mnesia:abort({error,exists,Ref_no})
            end
        end,
    mnesia:activity(transaction,F,[Reference_no,Amnt],mnesia_frag).

About the above option.

Mnesia docs say that transactions maybe repeated by the tm manager (due to some reason) until they are successful, and so when you put code which is io:format/2 or any other which has nothing to do with writes or reads, it may get executed several times. This statement made me pause at this point and think of a way of handling timers out of the transaction it self, so i modified the code as follows:

Option 2: timers handled outside the transaction


reserve_resources(Reference_no,Amnt)->
    F = fun(Ref_no,Amount) -> 
            case mnesia:read({free_airtime,Ref_no}) of
                [] -> 
                    P = #free_airtime{reference_no = Ref_no,amount = Amount},
                    ok = mnesia:write(P),
                    P;
                [_] -> mnesia:abort({error,exists,Ref_no})
            end
        end,    
    Result  =   try mnesia:activity(transaction,F,[Reference_no,Amnt],mnesia_frag) of
                    Any -> Any
                catch
                    exit:{aborted,{error,exists,XX}} -> {exists,XX}
                    E1:E2 -> {error,{E1,E2}}
                end,
    on_reservation(Result).

on_reservation(#free_airtime{reference_no = Some_Ref})-> 
    case timer:apply_after(timer:hours(24),?MODULE,reference_no_timed_out,[Some_Ref]) of
        {ok,Timer_obj} -> 
                [Obj] = mnesia:activity(transaction,fun(XX) -> mnesia:read({free_airtime,XX}) end,[Some_Ref],mnesia_frag),
                ok = mnesia:activity(transaction,fun(XX) -> mnesia:write(XX) end,[Obj#free_airtime{timer_object = Timer_obj}],mnesia_frag);
        _ -> 
            ok = mnesia:activity(transaction,fun(XX) -> mnesia:delete({free_airtime,XX}) end,[Some_Ref],mnesia_frag),
            {error,failed_to_time_object}
    end;
on_reservation(Any)-> Any.

The code to handle time out of the reservation:

reference_no_timed_out(Ref_no)->
    do_somethings_here.....
    then later remove this reservation from the database....below..
    ok = mnesia:activity(transaction,fun(XX) -> mnesia:delete({free_airtime,XX}) end,[Ref_no],mnesia_frag).

Now i thought that in option 2, i am safer by keeping the timer processing code out, even when mnesia_tm re-executes the transaction due to its reasons , this piece of code is not run twice (i avoid having several timer objects against the same record).

Question 1: Which of these two implementations is right? and/or wrong? Tell me (also) wether both of them are wrong

Question 2: The module timer, is it well suited for handling large numbers of timer jobs in production?

Question 3: As compared to Sean Hinde's timer_mn-1.1, which runs on top of mnesia, is the timer module (possibly running on top of Ets tables) less capable (for real) in production? (am asking this because using Sean Hinde's timer_mn on a system which itself is using mnesia appears to be a problem in terms schema changes, node problems e.t.c)

If any one has another way of handling timer related problems with mnesia, update me thanx guys...

Solution

  • Question 1:

    Handle the timer outside the transaction. When transactions collide in Mnesia, they are simply repeated. That would give you more than one timer reference and two triggers of the timer. It is not a problem per se, but if you wait until the success of the transaction before installing the timer, you can avoid the problem.

    The second solution is what I would do. If the TX is okay, you can install a timer on it. If the timer triggers and there is no reference to the object, it doesn't matter. You are only to worry about if this situation happens a lot since you would then have a large number of stray timers.

    Question 2:

    The timer module is neat, but the performance guide recommends you use the erlang:start_timer BIFs instead, see

    http://www.erlang.org/doc/efficiency_guide/commoncaveats.html#id58959

    I would introduce a separate process as a gen_server which handles the timing stuff. You send it a remove(timer:hours(24), RefNo) message and then it starts up a timer, gets a TRef and installs a mapping {TRef, RefNo, AuxData} in either Mnesia or ETS. When the timer trigger, the process can spawn a helper removing the RefNo entry from the main table.

    At this point, you must wonder about crashes. The removal gen_server may crash. Also, the whole node may crash. How you want to reinstall timers in the case this happens is up to you, but you ought to ponder on it happening so you can solve it. Suppose we come up again and the timer information is loaded in from disk. How do you plan on reinstalling the timers?

    One way is to have AuxData contain information about the timeout point. Every hour or 15 minutes, you scan all of the table, removing guys that shouldn't be there. In fact, you could opt for this being the main way to remove timer structures. Yes, you will give people 15 minutes of extra time in the worst case, but it may be easier to handle code-wise. At least it better handles the case where the node (and thus the timers) die.

    Another option again is to cheat and only store timings rougly in a data structure which makes it very cheap to find all expired RefNo's in the last 5 minutes and then run that every 5 minutes. Doing stuff in bulk is probably going to be more effective. This kind of bulk-handling is used a lot by operating system kernels for instance.

    Question 3

    I know nothing about timer-tm, sorry :)