[SOLVED] ruby eventmachine http-request deferrable

ruby eventmachine http-request deferrable

That's my first time with EM so I really need some help here

so here's the code:

EM.run do
  queue = EM::Queue.new
  EM.start_server('0.0.0.0', '9000', RequestHandler, queue)

  puts 'Server started on localhost:9000' # Any interface, actually

  process_queue = proc do |url|
    request = EM::HttpRequest.new(url, connect_timeout: 1).get # No time to wait, sorry

    request.callback do |http| # deferrable
      puts http.response_header.status
    end

    queue.pop(&process_queue)
  end

  EM.next_tick { queue.pop(&process_queue) }
end

I've read a couple of articles about EM, now my understanding of above code is the following:

EM::HttpRequest is deferrable, which means it won't block a reactor. But when I try running 50 concurrent connections with ab, it only serves ~20 concurrently ( according to ab report ).

But if I place the process_queue execution inside EM.defer( which means it will run in a separate thread? ) it performs just fine.

Why is it so? process_queue just inits a deferrable object and assigns a callback, how does running it inside EM.defer makes a difference?

Solution

One thing you may want to do is put the queue.pop(&process_queue) in the process_queue callback inside an EM.next_tick. Currently you're going to process all of the queued connections before you allow anything new to connect. If you put the queue.pop into a next_tick call you'll let the reactor do some work before you process the next item.