thriftspark-thriftserver

apache thrift monitor a hdfs file


Im planning to use Apache Thrift to monitor any change on my local changes and push that data to the client(the changed data).

When I check the thrift documentation Im seeing multiple transport layers but do not understand which transport layer should I use

http://thrift-tutorial.readthedocs.io/en/latest/thrift-stack.html

Tranport Layer
The transport layer is responsible for reading from and writing to the wire. Thrift supports the following:

TSocket - Uses blocking socket I/O for transport.
TFramedTransport - Sends data in frames, where each frame is preceded by a length. This transport is required when using a non-blocking server.
TFileTransport - This transport writes to a file. While this transport is not included with the Java implementation, it should be simple enough to implement.
TMemoryTransport - Uses memory for I/O. The Java implementation uses a simple ByteArrayOutputStream internally.
TZlibTransport - Performs compression using zlib. Used in conjunction with another transport. Not available in the Java implementation.

Solution

  • There are two kinds of transports:

    The former are the ones that you need (one of them) to write and read data to and from the wire. This could be a TSocket for example.

    The latter are used in addition, and in some cases they are even combined. For example, the TFramedTransport adds a special layer to the data to make memory allocation and I/O more efficient. The zlib transport may be used to compress data.

    One example could be:

    +------------------------------------+
    |  Application code                  |
    +------------------------------------+
    |  TBinaryProtocol                   |
    +------------------------------------+
    |  TZLibTransport                    |
    +------------------------------------+
    |  TFramedTransport                  |
    +------------------------------------+
    |  TSocket transport                 |
    +------------------------------------+
    

    Another one with no layered transports at all:

    +------------------------------------+
    |  Application code                  |
    +------------------------------------+
    |  TBinaryProtocol                   |
    +------------------------------------+
    |  TSocket transport                 |
    +------------------------------------+
    

    PS: What you linked is not official documentation, this is set up by some third-party people and the Apache Thrift project has no influence on the quality of that site.

    Highly recommended is the forthcoming Manning book from Randy Abernethy. He's a Thrift Committer and the book provides valuable insight. And no, I'm not getting anything for recommending it.