protocol-buffers

Does protobuf prevent partial reads?


A neat property of json is that partial reads are prevented.

If a client sends a json like:

[ {"someProperty": 1, "otherProperty": 2}, {"someProperty": 2, "otherProperty": 3}]

And for whatever reason the connection closes mid way and I only receive

[ {"someProperty": 1, "otherProperty": 2}

I know that I will have a json decoding exception and never process that payload.

Can partial reads happen with protobuf?

Meaning, I have some byte array, such that a[0: n] with n != len(a) is valid in the same model but obviously missing data?


Solution

  • No, it does not prevent partial reads.

    Protobuf is not self-terminating, and it is actively intended that append === merge (i.e. later changes / additions simply extend the existing payload). If you're lucky it'll break in the middle of a field, which will typically be detected with some kind of "EOF" failure result/exception; if you're unlucky, it'll break exactly at a field boundary, and it won't be detected - it'll simply only process the fields that were seen.

    Basically: manual framing or other external completeness checks are required with protobuf.