How to create python C++ extension with submodule that can be imported

I'm creating a C++ extension for python. It creates a module parent that contains a sub-module child. The child has one method hello(). It works fine if I call it as

import parent
parent.child.hello()
> 'Hi, World!'

If I try to import my function it fails

import parent
from parent.child import hello
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> ModuleNotFoundError: No module named 'parent.child'; 'parent' is not a package

parent.child
> <module 'child'>

here is my code setup.py

from setuptools import Extension, setup
  
# Define the extension module
extension_mod = Extension('parent',
                          sources=['custom.cc'])

# Define the setup parameters
setup(name='parent',
      version='1.0',
      description='A C++ extension module for Python.',
      ext_modules=[extension_mod],
      )

and my custom.cc

#include <Python.h>
#include <string>

std::string hello() {
    return "Hi, World!";
}

static PyObject* hello_world(PyObject* self, PyObject* args) {
    return PyUnicode_FromString(hello().c_str());
}

static PyMethodDef ParentMethods[] = {
    {nullptr, nullptr, 0, nullptr}
};

static PyMethodDef ChildMethods[] = {
    {"hello", hello_world, METH_NOARGS, ""},
    {nullptr, nullptr, 0, nullptr}
};

static PyModuleDef ChildModule = {
    PyModuleDef_HEAD_INIT,
    "child",
    "A submodule of the parent module.",
    -1,
    ChildMethods,
    nullptr,
    nullptr,
    nullptr,
    nullptr

};

static PyModuleDef ParentModule = {
    PyModuleDef_HEAD_INIT,
    "parent",
    "A C++ extension module for Python.",
    -1,
    ParentMethods,
    nullptr,
    nullptr,
    nullptr,
    nullptr
};

PyMODINIT_FUNC PyInit_parent(void) {
    PyObject* parent_module = PyModule_Create(&ParentModule);
    if (!parent_module) {
        return nullptr;
    }
    PyObject* child_module = PyModule_Create(&ChildModule);
    if (!child_module) {
        Py_DECREF(parent_module);
        return nullptr;
    }

    PyModule_AddObject(parent_module, "child", child_module);

    return parent_module;
}

I install and build with python setup.py build install.

So, how do I make sure that my parent is a package?

My code is a toy example but I actually want both modules defined on C++ level. I don't want to split them into several modules - since they are sharing some C++ code.

I'm hoping for something similar to approach of this answer Python extension with multiple modules

Solution

Doing this from within the extension is a "simple" matter of emulating the behavior for modules that the import system recognizes as packages. (Depending on context, it might be nicer to provide an import hook that did the same thing from the outside.) Just a few changes are needed:

Make the name of the child "parent.child".

Make parent a package:

PyModule_AddObject(parent_module, "__path__", PyList_New(0));
PyModule_AddStringConstant(parent_module, "__package__", "parent");

Make child a member of that package:

PyModule_AddStringConstant(child_module, "__package__", "parent");

Update sys.modules as Python would if it had performed the import:

PyDict_SetItemString(PyImport_GetModuleDict(), "parent.child", child_module);

Of course, several of these calls can fail; note that if PyModule_AddObject fails you have to drop the reference to the object being added (which I very much did not do here for clarity).