rubyamazon-ec2resquechef-infrabluepill

bluepill not detecting that processes have, in fact, started successfully, and so creates new ones


I have one (EC2) Ubuntu server where bluepill is working just fine to start and monitoring resque processes (and it has done so on other nodes in the past).

I'm setting up a new node, and for some reason on this node bluepill does not recognize that the processes have started and are running, and so keeps creating new ones. I'm a little baffled by what's causing this. The 2 nodes are almost identical; they're both EC2 servers provisioned by the same chef scripts. It is true that the one not working is 'production' and the other 'staging', but there's almost no difference due to that.

Any thoughts or suggestions before I fork the github project and start inserting more monitoring, to try and figure out what's going on? There's been discussion on this list in the past about troubles w/ bluepill and resque, but as I said this is working fine on my staging server, and has worked fine on earlier production servers (although I will note that this new production server is ruby 1.9.3 (vs 1.9.2) and rails 3.2 (vs. 3.1)).

Here's my .pill file (or more specifically, my chef cookbook's template file):

ENV["RAILS_ENV"] = "<%= node.chef_environment %>"
ENV["QUEUE"] = "*"

Bluepill.application("zmx_app") do |app|
  app.working_dir = "/srv/zmx/current"
  app.uid = "root"
  app.gid = "root"
  2.times do |i|
    app.process("resque-#{i}") do |process|
      process.group = "resque"
      process.start_command = "rake resque:work"
      process.pid_file = "/srv/zmx/current/tmp/pids/resque_workers-#{i}.pid"
      process.stop_command = "kill -QUIT {{PID}}"
      process.daemonize = true
    end
  end
end

Solution

  • This turned out to be a bug in bluepill, which I have forked, fixed, and submitted a pull request.

    And I'm not sure why I didn't realize that there was, in fact, a difference between my two environments: staging/old prod was on bluepill 0.0.55, my new production environment on 0.0.58.