I am building a file storage client/server using python for the client, go for the server, along with gRPC. I have already successfully built the client in go and it works! I am trying to do the same in python now. Today, I have been working on it all day, yet I have made 0 progress -_-. The request I am sending to my server is probably malformed in some way judging by the error message being spit out of the gRPC library:
File "/Users/xxxxx/Desktop/clients/uploadClient.py", line 60, in upload_file
for res in stream:
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/grpc/_channel.py", line 367, in __next__
return self._next()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/grpc/_channel.py", line 361, in _next
raise self
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.INTERNAL
details = "Exception serializing request!"
debug_error_string = "None"
>
Not too helpful. I have not found any helpful documentation and the hours of reading and searching have not payed off yet. The only thing I can rule out so far is that the problem is server related since the go client has been working great.
Here is my proto (note: I did distill this down a bit and renamed some things):
syntax = "proto3";
import "google/protobuf/timestamp.proto";
package upload;
message Chunk {
message Index {
uint64 as_uint64 = 1;
}
Index index = 1;
bytes sha512 = 2;
bytes data = 3;
}
message Descriptor {
string author = 1; // author
string label = 2; // label
Format format = 3; //format
}
enum Format {
FORMAT_UNKNOWN = 0;
FORMAT_CSV = 1;
FORMAT_XML = 2;
FORMAT_JSON = 3;
FORMAT_PDF = 4;
}
message UploadFile {
message ToClient {
oneof details {
Finished finished = 1;
}
}
message ToService {
oneof details {
Descriptor descriptor = 1;
Chunk chunk = 2;
}
}
}
.
service FileService {
rpc Upload(stream UploadFile.ToService) returns (stream UploadFile.ToClient);
}
Here is the code (note: I did distill this down a bit and renamed some things):
import s_pb2 as s
import s_pb2_grpc as s_grpc
token = 'xxxx'
url = 'xxxx:433'
requestMetadata = [('authorization', 'Bearer ' + token)]
def create_stub():
creds = grpc.ssl_channel_credentials()
channel = grpc.secure_channel(url, creds)
return s_grpc.UploadManagerStub(channel)
def upload_file(src, label, fileFormat):
stub = create_stub()
stream = stub.Upload(
request_iterator=__upload_file_iterator(src, label, fileFormat),
metadata=requestMetadata
)
for res in stream:
print(res)
return stream
def __upload_file_iterator(src, name, fileFormat, chunk_size = 1024):
def descriptor():
to_service = s.UploadFile.ToService
to_service.descriptor = s.Descriptor(
label=label,
format=fileFormat
)
return to_service
yield descriptor()
Yes, I know my iterator is only returning 1 thing, I removed some code to try to isolate the problem.
gRPC is not my strongest skills and I would like to believe that my pea brain is just missing something obvious.
All help is appreciated!
Wrapping the "oneof" details in the constructor fixed the problem:
def chunk(data, index):
return s.UploadFile.ToService(
chunk=s.Chunk(
index=s.Chunk.Index(as_uint64=index),
data=data,
sha512=hashlib.sha512(data).digest()
)
)
def descriptor(name, fileFormat):
return s.UploadFile.ToService(
descriptor=s.Descriptor(
name=name,
format=fileFormat
)
)