typescriptrestprotocol-buffersprotobufjs

Protobuf.js: the Writer doesn't seems to `finish`


I'm encountering a weird bug while using Protobuf in my TypeScript frontend. I'm using Axios to make calls to my REST API and the protobuf.js package to handle Protobuf in my frontend. I'm new to protobuf, and the issue may stem from my lack of knowledge.

The problem arises when I'm making multiple calls to my API with a payload.

For example, I want to post 3 objects: object_1, object_2, and object_3. Hence, I'm making three post requests. The first request is always correctly handled—object_1 is added to the backend. However, the following ones, to post object_2 and object_3, are posting object_1 again. I've investigated the issue and found that my protobuf is appended to the payload. Meaning that I've got object_1, and object_2 in the second request's payload and object_1, object_2, and object_3 in the third request's payload. My API only reads the first protobuf, i.e., object_1 and adds three times object_1.

I'm using the protobuf.js package as stated in the documentation:

const message = Message.create({ message: 'hello' });
const buffer = Message.encode(message).finish();
await axios.post([...] message [...]);

Does anyone already confront this issue? What am I doing wrong?

Thanks!


Solution

  • The Writer object of protobufjs uses a shared array buffer.

    But axios post doesn't look for the byteOffset and byteLength properties and uses the entire shared buffer, not just the bytes you want.

    You can extract the relevant bytes like this:

    const message = Message.create({ message: 'hello' });
    const x = Message.encode(message).finish();
    const buffer = new Uint8Array(x.buffer, x.byteOffset, x.byteLength);
    

    In my opinion, protobufjs is a bit dated. This is a shameless plug, because I am the author, but if you are not happy with protobufjs, please have a look at protobuf-ts. You would encode your message to binary like this:

    const message = Message.create({ message: 'hello' });
    const bytes: Uint8Array = Message.toBinary(message);
    

    It also supports Angular with auto-generated Twirp clients.