python-3.xtornadozeromqdistributed-computingpyzmq

Unable to run ZMQStream with Tornado Event loop using python 3.7


I've been trying to set up a server / client using zmq eventloop for REQ / REP messaging. Since python 3 doesn't support the eventloop provided by zmq, I'm trying to run it with tornado's eventloop.

I'm facing issues running zmqStream with tornado's event loop using python 3.

I created the server / client code using zmq's zmqStream and tornado's eventloop. The client is sending the correct messages, but the server doesn't seem to be responding to the message requests.

The server side code:

from tornado import ioloop
import zmq

def echo(stream, msg):
   stream.send_pyobj(msg)

ctx = zmq.Context()
socket = ctx.socket(zmq.REP)
socket.bind('tcp://127.0.0.1:5678')
stream = ZMQStream(socket)
stream.on_recv(echo)
ioloop.IOLoop.current().start()

The client side code:

import zmq

context = zmq.Context()
socket = context.socket(zmq.REQ)
socket.connect("tcp://127.0.0.1:5678")

for request in range (1,10):
   print("Sending request ", request,"...")
   socket.send_string("Hello")
   # Get the reply.
   message = socket.recv_pyobj()
   print("Received reply ", request, "[", message, "]")

I was expecting the server to return back the request messages being sent by the client. But it is just not responding to the requests being sent.


Solution

  • server doesn't seem to be responding

    Step 0:

    One server-side SLOC, stream = ZMQStream( socket ) calls a function, that is not MCVE-documented and must and does fail to execute to yield any result: "ZMQStream" in dir() confirms this with False

    Remedy: repair the MCVE and also print( zmq.zmq_version ) + "ZMQStream" in dir() confirmation


    Step 1:

    Always prevent infinite deadlocks, unless due reason exists not to do so, with setting prior to doing respective .bind() or .connect() <aSocket>.setsockopt( zmq.LINGER, 0 ). Hung forever applications and un-released (yes, you read it correctly, infinitely blocked) resources are not welcome in distributed computing systems.


    Step 2:

    Avoid a blind distributed-mutual-deadlock the REQ/REP is always prone to run into. It will happen, one just never knows when. You may read heaps of details about this on Stack Overflow.

    And remedy? May (and shall, where possible) avoid using the blocking-forms of .recv()-s (fair .poll()-s are way smarter-design-wise, resources-wise) may use additional sender-side signalisation before "throwing" either side into infinitely-blocking .recv()-s (yet a network delivery failure or other reason for a silent message drop may cause soft-signaling to flag sending, which did not result in receiving and mutual-deadlocking, where hard-wired behaviour moves both of the REQ/REP side into waiting one for the other, to send a message (which the counterparty will never send, as it is also waiting for .recv()-ing the still not received one from the (still listening) opposite side ))


    Last, but not least:

    The ZeroMQ Zen-of-Zero has also a Zero-Warranty - as messages are either fully delivered (error-free) or not delivered at all. The REQ/REP mutual deadlocks are best resolvable if one never falls into them (ref. LINGER and poll() above)