Nanopb: How to define a nanopb message that nests itself?

I'm having trouble defining a nanopb message that has a nested type whose type is the message itself.

If the message doesn't nest itself, everything looks good: proto definition:

syntax = "proto2";
import "nanopb.proto";
option (nanopb_fileopt).max_size = 20;
 
message myMessage {
   optional string string1 = 1;
   optional string string2 = 2;
   optional string string3 = 3;
}

generated code:

typedef struct _myMessage {
    bool has_string1;
    char string1[20];
    bool has_string2;
    char string2[20];
    bool has_string3;
    char string3[20];
/* @@protoc_insertion_point(struct:myMessage) */
} myMessage;

If the message nests itself, there will be errors: proto definition:

syntax = "proto2";
import "nanopb.proto";
option (nanopb_fileopt).max_size = 20;
 
message myMessage {
   optional string string1 = 1;
   optional string string2 = 2;
   optional string string3 = 3;
   optional myMessage submessage = 4;
}

errors:

$ python3 generate_c_files.py test/aaa.proto
Today is a good day to code
Traceback (most recent call last):
  File ".../generator/nanopb_generator.py", line 1850, in <module>
    main_cli()
  File ".../generator/nanopb_generator.py", line 1746, in main_cli
    results = process_file(filename, None, options)
  File ".../generator/nanopb_generator.py", line 1704, in process_file
    headerdata = ''.join(f.generate_header(includes, headerbasename, options))
  File ".../generator/nanopb_generator.py", line 1338, in generate_header
    msize = msg.encoded_size(self.dependencies)
  File ".../generator/nanopb_generator.py", line 1068, in encoded_size
    fsize = field.encoded_size(dependencies)
  File ".../generator/nanopb_generator.py", line 670, in encoded_size
    encsize = submsg.encoded_size(dependencies)
  File ".../generator/nanopb_generator.py", line 1068, in encoded_size
    fsize = field.encoded_size(dependencies)
  File ".../generator/nanopb_generator.py", line 670, in encoded_size
    encsize = submsg.encoded_size(dependencies)
...
File ".../generator/nanopb_generator.py", line 1068, in encoded_size
    fsize = field.encoded_size(dependencies)
  File ".../generator/nanopb_generator.py", line 670, in encoded_size
    encsize = submsg.encoded_size(dependencies)
  File ".../generator/nanopb_generator.py", line 1068, in encoded_size
    fsize = field.encoded_size(dependencies)
  File ".../generator/nanopb_generator.py", line 706, in encoded_size
    encsize = EncodedSize(self.enc_size)
  File ".../generator/nanopb_generator.py", line 160, in __init__
    elif isinstance(value, strtypes + (Names,)):
RecursionError: maximum recursion depth exceeded in __instancecheck__

It seems like, if there's a field in the message whose type is the message itself, you can't set the max_size for a string type field. However I do need to set the max_size for the string, because I need the generated struct member "bool has_string1;".

Solution

It's not possible to statically allocate space for an unbounded recursive structure. You need to either use dynamic allocation or callbacks. This can be accomplished using the type option set to either FT_CALLBACK or FT_POINTER:

message myMessage {
   optional string string1 = 1;
   optional string string2 = 2;
   optional string string3 = 3;
   optional myMessage submessage = 4 [(nanopb).type = FT_CALLBACK];
}

Starting with version 0.4.8, the FT_CALLBACK option will be automatically applied if the generator detects a recursive message. With earlier versions it has to be added manually, in either the .proto file or in a separate .options file.

To use a callback, you need to define a field callback function that will pass the data between your custom storage structure and the protobuf wire format.

With dynamic allocation, the structure will contain the pointer struct myMessage*, which points to the nested message or NULL at the end of recursion. For encoding you can use any kind of allocation for the messages. For decoding, pb_realloc() is used, which by default uses libc heap, but can be overridden to use e.g. custom arena allocator.