pythonflutterdartsocketsbytestream

How to recieve and parse data coming from a Python socket server in dart?


I have a socket server running in Python that sends a byte stream through the socket in which, the first 8 bytes contain the size of the remaining message body. The code looks something like this:

    data_to_send = struct.pack("Q", len(pickle_dump))+pickle_dump
    
    client_socket.sendall(data_to_send)

The client endpoint of this, that is written in Python looks like this:

data = b''
PAYLOAD_SIZE = struct.calcsize('Q')

while len(data) < PAYLOAD_SIZE:
    packet = client_socket.recv(1024)
    if not packet:
        break
    data += packet

    print(len(data), PAYLOAD_SIZE)

packed_msg_size = struct.unpack('Q', data[:PAYLOAD_SIZE])[0]
data = data[PAYLOAD_SIZE:]

while len(data) < packed_msg_size:
    data += client_socket.recv(1024)

print(type(data))
frame_data = data[:packed_msg_size]
data = data[packed_msg_size:]

Now I am kind of new to dart and flutter. So I kind of tried recieving the packets through dart's sockets (which I am not sure how it is implemented) like this:

Socket.connect("10.0.2.2", 7878).then((socket) {
        SOCKET = socket;

        setState(() {
          print('Connected to: '
              '${socket.remoteAddress.address}:${socket.remotePort}');
          log = '${socket.remoteAddress.address}:${socket.remotePort}';
        });

        var FULL_CLUSTER = ByteData(0);


        // MAIN STUFF STARTS HERE
        socket.listen((data) {
          Uint8List size_bytesdata = data.sublist(0, 8);
          print(size_bytesdata.buffer.asByteData().getUint64(0)); // Prints -6552722064661282816
          print(size_bytesdata);
          socket.destroy();
        });
        //

      }).onError((error, stackTrace) {
        setState(() {
          log = '$error\n\n$stackTrace';
        });
      });

Now first of all, I figured out that Dart's sockets shows bytes in Uint8List format, so I tried doing what I did in the python client script, but the values do not match.

The len(pickle_dump) has the value 921765 but the dart code(as mentioned in the snippet) prints -6552722064661282816. Now I believe the type recognition by dart is what's causing the trouble as I have packed is as 'Q' (using struct) in the Server script, which stands for 'Unsigned long long' but, in dart, I parsed it as 'Unsigned int'. How do I solve this issue? Also, a brief discussion what's exactly going will help a lot.


Solution

  • As mentioned by pskink, the problem here is caused because the default Endian of Dart's getUint64() is different than that of Python's struct library (which uses the 'little' endianness to pack data). So just rewritting the line fromgetUint64(0) to getUint64(0, Endian.little) fixes the problem in this case.