djangocelerydjango-celeryeventlet

Lots of socket errors with celery eventlet tasks


I'm getting a lot of "IOError: Socket closed" exceptions from amqplib.client_0_8.method_framing.read_method when running my celery workers with the --pool=eventlet option. I'm also seeing a lot of timeout exceptions from eventlet.hubs.hub.switch.

I'm using an async_manage.py script similar to the one at https://gist.github.com/821848, running the works like:

./async_manage.py celeryd_detach -E --pool=eventlet --concurrency=120 --logfile=<path>

Is this a known issue, or is there something wrong with my configuration or setup?

I'm running djcelery 2.2.4, Django 1.3, and eventlet 0.9.15.


Solution

  • The problem was a side effect of some code that was blocking. I managed to detect the blocking code using the eventlet option described in this article.

    There were 2 places where blocking was occuring: DNS lookups, and MySQL database access. I managed to resolve the first by installing the dnspython package, and the second my using the undocumented MySQLdb option in eventlet:

    import eventlet
    eventlet.monkey_patch()
    eventlet.monkey_patch(MySQLdb=True)