I know, this question was probably asked a lot. I consider myself a newbie on Python and sometimes I don't quite get how the importing system works. Especially in my case I have not found any solutions without a workaround I don't want to do. Why? Let me explain:
So I have this "program" which works with Google protobufs. When creating Python code from the given compiler (protoc) it always says "DO NOT EDIT!". So I don't edit the generated code and I don't really want to (There must be a solution without fixing each file by myself every time the code is generated).
Here is how my project is organized:
|root:
|---protos:
| |---API:
| | |---protoApiA.proto
| | |---protoApiB.proto ...
| |---common:
| |---protoCommonA.proto ...
|---src:
| |---__init__.py
| |---foo:
| | |---__init__.py
| | |---bar:
| | | |---main.py
| | | |---protos:
| | | | |---API:
| | | | | |---protoApiA_pb2.py
| | | | | |---protoApiB_pb2-grpc.py ...
| | | | |---common:
| | | | | |---protoCommonA_pb2.py
| | | | | |---....
|---test_script.py
protoApiA.proto imports from protoCommonA.proto.
import "common/protoCommonA.proto";
Which resolves into:
from common import protoCommonA_pb2 as common_dot_protoCommonA__pb2
After I created them using following PowerShell prompt:
& py -3.10 -m grpc_tools.protoc -I./protos --python_out=./src/foo/bar/protos/ --grpc_python_out=./src/foo/bar/protos/ common/protoCommonA.proto API/protoApiA.proto API/protoApiB.proto
So far so good. Here is what the init files contain:
# root/src/__init__.py:
from . import foo
# root/src/foo/__init__.py:
from .bar.main import ... #relevant functions for test_script.py
The file main.py needs to use the proto files for some gRPC related stuff. The file test_script.py is a script which uses some functionalities from main.py mainly for testing.
If I test the functionality over test_script.py it throws a "module not found"-error at protoApiA_pb2.py since it does not know about protoCommonA_pb2.py.
So here is my question: How can I make this work, without editing the import statements in any of the automatically generated Python files?
I informed myself about how the import system works, but I think I do not understand how it really works, since I can't come up with any solution trying the last three days.
Thanks in advance! And hopefully someone can explain to me why I have this issue and how to properly resolve it
So, in the end, I found myself editing the generated proto files automatically right after their creation with a small script.
Although Google uses Protobufs extensively (obviously), they have a flat hierarchy inside their packages or install their proto packages separately. Thus, they do not encounter these problems, and a fix is most likely not made (The issue has persisted since ~2015).