erlangelixirdata-cleaningetscache-expiration

Remove all the records from ets table having datestamp older than 10 seconds


I have an ets set table in an elixir app. I need to clean-up records which have their updated_at field older than 10 seconds. Is there a way I can set expiry or do it manually without iterating over all the records? I match records based on the timestamps greater than given time.

example record:

key: key_1
record: %{id: key_1, updated_at: ~N[2018-12-19 10:08:47.803075]}

so far I have this code

def clean_stale(previous_key) do
  if previous_key == :"$end_of_table" do
    :ok
  else
    device = get(previous_key)
    next_key = :ets.next(__MODULE__, previous_key)
    if NaiveDateTime.diff(NaiveDateTime.utc_now, device.last_recorded_at) > 10 do
      remove(device.id)
    end
    clean_stale(next_key)
  end
end

Solution

  • If you store the "updated at" time as an integer instead of as a NaiveDateTime struct, you can use a match spec.

    For example, to get the current time as the number of seconds since the Unix epoch:

    > DateTime.to_unix(DateTime.utc_now())
    1545215338
    

    You can do something like this:

    iex(3)> :ets.new(:foo, [:public, :named_table])
    :foo
    iex(4)> :ets.insert(:foo, {:key1, DateTime.to_unix(DateTime.utc_now())})
    true
    iex(5)> :ets.insert(:foo, {:key2, DateTime.to_unix(DateTime.utc_now())})
    true
    iex(6)> :ets.tab2list(:foo)
    [key2: 1545215144, key1: 1545215140]
    iex(7)> :ets.select_delete(:foo, [{{:_, :"$1"}, [{:<, :"$1", 1545215144}], [true]}])
    1
    iex(8)> :ets.tab2list(:foo)
    [key2: 1545215144]
    

    In the call to ets:select_delete/2, I pass a match specification. It consists of three parts:

    So after calling select_delete, only the second record remains in the table.


    If the timestamp is inside a map, you can use map_get to access it and compare it:

    :ets.select_delete(:foo, [{{:_, :"$1"},
                               [{:<, {:map_get, :updated_at, :"$1"}, 1545215339}],
                               [true]}])
    

    Or (in Erlang/OTP 18.0 and later) match out the map value:

    :ets.select_delete(:foo, [{{:_, #{updated_at: :"$1"}},
                               [{:<, :"$1", 1545215339}],
                               [true]}])