rubyruby-on-rails-3intercom

Ruby Batch Processing an Array for Intercom

I have an array of objects which need processing. I need to send them to a 3rd party system via their API, which only allows me to submit 100 objects at a time.

So let's say I have an array of objects like this

myUserArray = [{first_name: 'Jon', last_name: 'Snow'}, {first_name: 'Sansa', last_name: 'Stark'}...]

I end up sending this to their API like this

intercom.users.submit_bulk_job(create_items: myUserArray)

This works fine when the number of objects less than 100 but throws an error when greater than 100 due to their rate limiting, which is fair enough. I have 5000 objects to process, so I need a way of batching the myUserArray into chunks of 100 until they are all done. Would appreciate any advice !

Solution

  • Enumerable#each_slice comes to the rescue:

    myUserArray.each_slice(100) do |slice|
  intercom.users.submit_bulk_job(create_items: slice)
end