pythonpython-3.xtcpclienttwisted.internettwistd

Python - Twisted client - Maximum size of data in dataRecived method


I've recently been using and learning twisted for creating TCP client socket.

I got that dataRecived method of Protocol class, returns all data together. It means all data ConCat with each other and finally it returns a long byte data from server.

Code:

from twisted.internet.protocol import Protocol

class ClientProtocol(Protocol):
    def connectionMade(self):
        self.transport.write(b'a')

    def dataReceived(self, data):
        print ('data', data)

So now my questions are:

  1. Has dataRecived method any maximum size for data? Or it's size is unlimited?

  2. If it has maximum size, what is that? And how can i override that?

Note: I'm using Ubuntu 14.04, Python v3.4 and Twisted v15.3.0.


Solution

  • According to Twisted Documentation , the data parameter in dataRecived method, is not unlimited and it is :

    " a string of indeterminate length. Please keep in mind that you will probably need to buffer some data, as partial (or multiple) protocol messages may be received! " .

    So we need to override dataRecived method manually for buffering data. Or using some defined inherit class from protcol.Protocol like: LineReceiver class or NetstringReceiver class for this issue.

    For example dataReceived source of LineRecevier class is as bellow:

     def dataReceived(self, data):
            """Protocol.dataReceived.
            Translates bytes into lines, and calls lineReceived (or
            rawDataReceived, depending on mode.)
            """
            self.__buffer = self.__buffer+data
            while self.line_mode and not self.paused:
                try:
                    line, self.__buffer = self.__buffer.split(self.delimiter, 1)
                except ValueError:
                    if len(self.__buffer) > self.MAX_LENGTH:
                        line, self.__buffer = self.__buffer, ''
                        return self.lineLengthExceeded(line)
                    break
                else:
                    linelength = len(line)
                    if linelength > self.MAX_LENGTH:
                        exceeded = line + self.__buffer
                        self.__buffer = ''
                        return self.lineLengthExceeded(exceeded)
                    why = self.lineReceived(line)
                    if why or self.transport and self.transport.disconnecting:
                        return why
            else:
                if not self.paused:
                    data=self.__buffer
                    self.__buffer=''
                    if data:
                        return self.rawDataReceived(data)