Disclaimer: I am relatively new to Protocol Buffers, and python is not my language of choice. Most of my knowledge of protocol buffers has used them without compiling them, or having an IDE handle it as part of a deployment.
I've been working on updating/fixing an integration with the Steam API. The integration is out of date, and needs a new rpc call, which is in a proto file that has not been compiled to python. I tried the new file and got unexpected results - None of the rpc calls or objects were set up, and all i had was a giant DESCRIPTION blob. To check if this was intended, I tried again, but with a file already in the project, to see if the results were different than the source file. Once again, this appeared to be the case - the newer version has less useful data than the original, and as far as i can tell, is losing information from the .proto file.
I have retrieved the latest .proto files from SteamKit (see https://github.com/steamdatabase/protobufs/tree/ab60110e6ff4766248c73a4dfc1c0beabdad6de5), and followed the instructions from https://protobuf.dev/getting-started/pythontutorial/. I am on Windows, and have run both the CMD version of protoc and the one using python via grpcio and grpcio-tools.
Running protoc -I. --python_out=.\out .\steammessages_base.proto yields https://pastebin.com/bEMwhCgA The expected is https://pastebin.com/iGtPqS9V
I have also run python -m grpc_tools.protoc -I. --python_out=.\out --grpc_python_out=.\grpc_out .\steammessages_base.proto to see if it was something grpc related. The grpc file is empty, except for the class declaration.
I am using Visual Studio as my IDE, but have Apache Netbeans and WSL Ubuntu as well, should either of those options yield better results.
I have checked other answers on SO, and have not found the solution. That said, i may have missed it not knowing what i am looking for
Edit: Probably goes without saying, but if you are on *Nix, use forward slashes in the protoc commands. Also, the out and grpc_out directories already existed before running the commands.
Edit2: The source .proto file can be found at https://github.com/SteamDatabase/Protobufs/blob/ab60110e6ff4766248c73a4dfc1c0beabdad6de5/steam/steammessages_base.proto
Edit3: In case the links are down, the DESCRIPTOR=fileDescriptor... appears identical. The difference is after that. A portion is recorded below as an example
Expected output (original, good file in the project)
_CMSGAUTHTICKET = _descriptor.Descriptor(
name='CMsgAuthTicket',
full_name='CMsgAuthTicket',
filename=None,
file=DESCRIPTOR,
containing_type=None,
create_key=_descriptor._internal_create_key,
fields=[
#... addition python code here
]
)
and what i got:
_CMSGAUTHTICKET._serialized_start=1189
_CMSGAUTHTICKET._serialized_end=1355
These aren't the calls i need, but should demonstrate what's going on.
Is your issue simply that you need to reproduce the exact generated code? If so, the cause is that the version of protoc packaged in the grpcio-tools wheel is newer than the one used to generate the code you're comparing against. The most recent breaking change in the generated code was in grpcio-tools 1.49, which has a dependency on protobuf 4.21.3. Try downgrading to grpcio-tools==1.48.2, which has a dependency on protobuf==3.12.0 and should generate the expected older format. Make sure to pin to protobuf==3.12.0 as well for runtime. grpcio itself is agnostic to protobuf version, so feel free to pull in the latest version.