pythondjangogeventgrequests

Mysterious error when using django server with grequests


Currently, I am running a vagrant server on Ubuntu 14.04 and I test all my django modules by using the simple python manage.py runserver 0.0.0.0:8000

Since I am connecting to the django webserver using chrome through http://localhost:8000 and the server is running on a VM, I am port forwarding through the usage of the following setting in Vagrantfile

config.vm.network "forwarded_port", guest: 8000, host: 8000

Everything runs normally (all modules/views/tests function as expected), however, ever since I started using grequests I get this weird error

Exception happened during processing of request from ('10.0.2.2', 63520)
Traceback (most recent call last):
  File "/home/vagrant/anaconda3/lib/python3.6/socketserver.py", line 639, in process_request_thread
    self.finish_request(request, client_address)
  File "/home/vagrant/anaconda3/lib/python3.6/socketserver.py", line 361, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/home/vagrant/anaconda3/lib/python3.6/socketserver.py", line 696, in __init__
    self.handle()
  File "/home/vagrant/anaconda3/lib/python3.6/site-packages/django/core/servers/basehttp.py", line 159, in handle
    self.raw_requestline = self.rfile.readline(65537)
  File "/home/vagrant/anaconda3/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
  File "/home/vagrant/anaconda3/lib/python3.6/site-packages/gevent/_socket3.py", line 385, in recv_into
    self._wait(self._read_event)
  File "/home/vagrant/anaconda3/lib/python3.6/site-packages/gevent/_socket3.py", line 157, in _wait
    self.hub.wait(watcher)
  File "/home/vagrant/anaconda3/lib/python3.6/site-packages/gevent/hub.py", line 651, in wait
    result = waiter.get()
  File "/home/vagrant/anaconda3/lib/python3.6/site-packages/gevent/hub.py", line 899, in get
    return self.hub.switch()
  File "/home/vagrant/anaconda3/lib/python3.6/site-packages/gevent/hub.py", line 630, in switch
    return RawGreenlet.switch(self)
gevent.hub.LoopExit: ('This operation would block forever', <Hub at 0x7f3b777e8af8 epoll pending=0 ref=0 fileno=34>)

Note that I am not using grequests and that simply importing it seems to cause this error even when it is not being called or anything

Anyone have any ideas?


Solution

  • This is an issue with one of the underlying dependency - gevent, which overrides the default behaviour of python builtin such as time, etc.

    You would have to monkeypatch. Something like:

    from gevent import monkey
    monkey.patch_all()
    

    Here is the relevant gevent documentation.

    I recently ran into this exact issue - so stopped using grequests and implemented my own async requests logic