[SOLVED] Pyro communication extremely slow on the same host

Pyro communication extremely slow on the same host

I have implemented a very simple client/server application using Pyro. While the Pyro tips&tricks advises against it, I'm using the server to send images to the client (uncompressed, numpy arrays to be precise). It doesn't have to go through the network, everything stays on localhost, so I wouldn't see a problem. What happens is that the client is 1 order of magnitude slower than the server, any ideas why?

Here the code:

Server:

import numpy as np
import time
import Pyro.core

class DataProd(Pyro.core.ObjBase):
    def __init__(self, batch_size):
        print 'Loading data into memory'
        self.all_imgs = np.load('data.npy')
        Pyro.core.ObjBase.__init__(self)

    def get_batch(self, batch_size=32, flip=False):

        print 'getting 1 batch from PYRO'
        s = time.time()
        #process the images
        print time.time()-s
        return images


def main():
    Pyro.core.initServer()
    daemon=Pyro.core.Daemon()
    uri=daemon.connect(DataProd(32),'dataprod')
    print "The daemon runs on port:",daemon.port
    print "The object's uri is:",uri
    print "Starting request loop"
    daemon.requestLoop()

if __name__ == '__main__':
    main()

And client:

import Pyro.core
import time

data = Pyro.core.getProxyForURI("PYROLOC://localhost:7766/dataprod")
while True:
    s = time.time()
    im = data.get_batch(32)
    print time.time()-s

The server prints:

getting 1 batch from PYRO
0.526908874512
getting 1 batch from PYRO
0.51292014122
getting 1 batch from PYRO
0.523808956146
getting 1 batch from PYRO
0.536481142044
getting 1 batch from PYRO
0.518028974533

And the client:

4.93717813492
4.05996489525
3.40680289268
3.79327297211
3.99453115463

Why is there a 1 order of magnitude difference between the two? The client doesn't do anything else but to request images..

Thanks!

Solution

Have you profiled to see where exactly in the client the perceived slowdown occurs?

I suspect the deserialization of large numpy data is causing this. Read this https://pythonhosted.org/Pyro4/tipstricks.html#pyro-and-numpy