matrixsparse-matrixzeromqchapel

How to send sparse vectors and matrices over ZeroMQ?

I have a matrix A (how many stories start this way?) that is sparse. 

[
[0,   0, 0, 1.2, 0]
[0,   0, 0, 0,   0]
[3.5, 0, 0, 0,   0]
[0    7, 0, 0,   0]
]

I want to send variants of this back and forth between processes using ZeroMQ. Assume the client and servers are of different languages that have no common serialization format. Here are some tasks.

  1. Create A. This is complicated by needing to send the "frame" of the matrix, here (4,5).
  2. Update A[4,2] from 7 to 6.
  3. Take a sparse vector v=[0,0,3.1,0,0] and multiply it by A and get the result back.

I've been told that sending byte streams is probably the best solution, but I can't find any examples between different libraries and in a sparse format.

My default will be to have a Python, C++ or Chapel pairing if someone can speak to those.

Solution

  • Since the design is very simple, I decided to go with a string representation with preset delimiters. Without something like protobuf it was easy to just have an understood protocol. I used the format:

    msg = '<i_index>:<j_index>:<value>^<i_index>:<j_index>:<value>`

    In my case, the server already knew the frame of the matrix. So to send the vector [0 0 0.8 0 0 1.4 0 0] I would do

    msg = '0:2:0.8^0:5:1.4'

    It works so long as I don't have to send the massive matrix over.