I'm working on simulating onion routing on my PC. To do this, I need to split requests (e.g., HTTP requests) into TCP packets with a fixed size of 512 bytes and send them to the next node using sockets.
For example, I'd like to make the following HTTP request:
GET / HTTP/1.1\r\n
Host: 127.0.0.1\r\n
Accept: text/html\r\n
Accept-Encoding: gzip, deflate, br\r\n
Accept-Language: en-US,en;q=0.9\r\n
However, I'm encountering a problem: I can't set the TCP packet size directly. If I manually split the packets before sending, they aren't transmitted in the same TCP stream, so the server can't reconstruct the request in the correct order.
Does anyone know how to split TCP packets correctly in Python?
From the perspective of the application layer TCP is just a byte stream and there is no way to specify how exactly this byte stream should be packetized for transport.
In fact, TCP tries to optimize delivery by putting as much payload as useful into a packet in order to reduce transport overhead. Specifically this is done with Nagle's algorithm which can merge multiple application layer send together into the same packet. By disabling this algorithm one can force that the OS does not wait for more data before delivering a packet.
Thus if you disable Nagle's algorithm and then send only 512 byte large buffers you kind of achieve what you want. But, this is only a side effect and there is no actual guarantee that the packets do not contain more than 512 byte. If the communication peer does not read fast enough than some outstanding data will be buffered in the senders write buffer and once the receiver is ready the sender will try to send these buffered data in the optimal way, which means larger packets. Similar if data are lost (not acked by the peer) they will not be necessarily delivered in the original packet but can be merged with other outstanding payload.