I'm getting a lot of "IOError: Socket closed" exceptions from amqplib.client_0_8.method_framing.read_method when running my celery workers with the --pool=eventlet option. I'm also seeing a lot of timeout exceptions from eventlet.hubs.hub.switch.
I'm using an async_manage.py script similar to the one at https://gist.github.com/821848, running the works like:
./async_manage.py celeryd_detach -E --pool=eventlet --concurrency=120 --logfile=<path>
Is this a known issue, or is there something wrong with my configuration or setup?
I'm running djcelery 2.2.4, Django 1.3, and eventlet 0.9.15.
The problem was a side effect of some code that was blocking. I managed to detect the blocking code using the eventlet option described in this article.
There were 2 places where blocking was occuring: DNS lookups, and MySQL database access. I managed to resolve the first by installing the dnspython package, and the second my using the undocumented MySQLdb option in eventlet:
import eventlet
eventlet.monkey_patch()
eventlet.monkey_patch(MySQLdb=True)