pythonc++dllpybind11pyd

Cannot import .pyb / .dll file into Python after compiling C++ code with pybind11 on Windows


My OS is Windows 10, I use G++ compiler, VS Code. I attempt to compile an example pybind11 file into a Python object. The C++ code looks as follows:

#include <pybind11/pybind11.h>

int add(int i, int j) {
    return i + j;
}

namespace py = pybind11;

PYBIND11_MODULE(cmake_module, m) {
    m.def("add", &add, R"pbdoc(
        Add two numbers

        Some other explanation about the add function.
    )pbdoc");
};

To compile the project I use the CMake script:

cmake_minimum_required(VERSION 3.30.2)
set(CMAKE_CXX_STANDARD 23)
set(CMAKE_CXX_FLAGS "-Wall -Wextra -pedantic -shared")
set(CMAKE_CXX_FLAGS_DEBUG "-Werror -Og")

# Specify the path to pybind11 if it's not installed globally
set(PYBIND11_DIR "C:\\Users\\User\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\pybind11")

# Specify the path to the Python interpreter
set(PYBIND11_FINDPYTHON "C:\\Users\\User\\AppData\\Local\\Programs\\Python\\Python312\\python.exe")

# Find pybind11
find_package(pybind11 REQUIRED PATHS ${PYBIND11_DIR})

# Find Python components
find_package(Python COMPONENTS Interpreter Development REQUIRED)

# Include directories for Python headers
include_directories(${Python_INCLUDE_DIRS})

# Create the module
set(MODULE_NAME cmake_example)
pybind11_add_module(${MODULE_NAME} src/test.cpp)

After successfully building the file, I get a cmake_example.cp312-win_amd64.pyd file in a build folder. Here, I run import cmake_example, which lead to the following error: ImportError: DLL load failed while importing cmake_example: The specified module could not be found.

I use Python 3.12.4, if it's somehow relevant.

My attempts to solve the issue:


Solution

  • When loading a .pyd file you also need to load the dependencies of the .pyd file otherwise the import will fail. (which for some reason fails with not found error instead of failed to load)

    Add the folder containing MinGW standard libraries (namely libgcc and libstdc++.dll) to the end of your PATH environment variable, it is the same folder that contains gcc.exe and g++.exe

    You can see the dependencies of your dll using dependency walker, those dependencies either need to be in the same folder or on your PATH, ( note: .pyd is a .dll just renamed)

    It is possible to statically link only a few dependencies, but you cannot use the -static flag as python312.dll needs to be dynamically linked, so you need to be picky about what you statically link (only statically link libgcc and libstdc++ and libwinpthread, ie: -static-libgcc ... etc ...)