searchkick

Searchkick memory leak


Running rake searchkick:reindex CLASS=Product for an application causes the Rake process to leak memory; after about 15-20 minutes it's bad enough to freeze a Debian system with 16GB of RAM. There are ~3800 "Product" records.

I managed to work around this problem with the following code in a Rake task:

connection = ActiveRecord::Base.connection
res = connection.execute('select max(id) from products')
id = res.getvalue(0,0)
1.upto(id) do |i|
  p = Product.find_by_id(i)
  next unless p

  p.reindex
end

This is also a little quicker.

Can anyone suggest a means to investigate this memory leak? It would be useful to do so in more detail before considering opening a ticket.


Solution

  • This causes a problem with generating indexes: Text fields are not optimised for operations that require per-document field data

    That problem can be fixed by adding the following to the code above:

    Product.reindex(import: false)
    # Rest of code goes here...