Things are getting complicated in my world of trying to mesh Python code with my C++.
Essentially, I want to be able to assign a callback function to be used after an HTTP call receives a response, and I want to be able to do this from either C++ or Python.
In other words, I want to be able to call this from C++:
http.get_asyc("www.google.ca", [&](int a) { std::cout << "response recieved: " << a << std::endl; });
and this from Python:
def f(r):
print str.format('response recieved: {}', r)
http.get_async('www.google.ca', f)
I have set up a demo on Coliru that shows exactly what I'm trying to accomplish. Here is the code and the error that I am getting:
#include <boost/python.hpp>
#include <boost/function.hpp>
struct http_manager
{
void get_async(std::string url, boost::function<void(int)> on_response)
{
if (on_response)
{
on_response(42);
}
}
} http;
BOOST_PYTHON_MODULE(example)
{
boost::python::class_<http_manager>("HttpManager", boost::python::no_init)
.def("get_async", &http_manager::get_async);
boost::python::scope().attr("http") = boost::ref(http);
}
import example
def f(r):
print r
example.http.get_async('www.google.ca', f)
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
Boost.Python.ArgumentError: Python argument types in
HttpManager.get_async(HttpManager, str, function)
did not match C++ signature:
get_async(http_manager {lvalue}, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, boost::function<void (int)>)
I'm not sure why the function is not being converted to a boost::function automatically.
I have asked a vaguely similar question on SO before and got an amazing answer. I also wonder if a similar method in the answer given there could be applied to this use-case as well.
Thank you very much for any and all support!
When a function that has been exposed through Boost.Python is invoked, Boost.Python will query its registry to locate a suitable from-Python converter for each of the caller's arguments based on the desired C++ type. If a converter is found that knows how to convert from the Python object to the C++ object, then it will use the converter to construct the C++ object. If no suitable converters are found, then Boost.Python will raise an ArgumentError exception.
The from-Python converters are registered:
int and std::stringboost::python::class<T>. By default, the resulting Python class will hold an embedded instance of a T C++ object, and register to-Python and from-Python converters for the Python class and type T, using the embedded instance.boost::python::converter::registry::push_back()The steps of testing for convertibility and constructing an object occur in two distinct steps. As no from-Python converter has been registered for boost::function<void(int)>, Boost.Python will raise an ArgumentError exception. Boost.Python will not attempt construct the boost::function<void(int)> object, despite boost::function<void(int)> being constructible from a boost::python::object.
To resolve this, consider using an shim function to defer the construction of boost::function<void(int)> until after the boost::python::object has passed through the Boost.Python layer:
void http_manager_get_async_aux(
http_manager& self, std::string url, boost::python::object on_response)
{
return self.get_async(url, on_response);
}
...
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<http_manager>("HttpManager", python::no_init)
.def("get_async", &http_manager_get_async_aux);
...
}
Here is a complete example demonstrating this approach:
#include <boost/python.hpp>
#include <boost/function.hpp>
struct http_manager
{
void get_async(std::string url, boost::function<void(int)> on_response)
{
if (on_response)
{
on_response(42);
}
}
} http;
void http_manager_get_async_aux(
http_manager& self, std::string url, boost::python::object on_response)
{
return self.get_async(url, on_response);
}
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<http_manager>("HttpManager", python::no_init)
.def("get_async", &http_manager_get_async_aux);
python::scope().attr("http") = boost::ref(http);
}
Interactive usage:
>>> import example
>>> result = 0
>>> def f(r):
... global result
... result = r
...
>>> assert(result == 0)
>>> example.http.get_async('www.google.com', f)
>>> assert(result == 42)
>>> try:
... example.http.get_async('www.google.com', 42)
... assert(False)
... except TypeError:
... pass
...
An alternative approach is to explicitly register a from-Python converter for boost::function<void(int)>. This has the benefit that all functions exposed through Boost.Python can use the converter (e.g. one would not need to write a shim for each function). However, a convert would need to be registered for each C++ type. Here is an example demonstrating explicitly registering a custom converter for boost::function<void(int)> and boost::function<void(std::string)>:
#include <boost/python.hpp>
#include <boost/function.hpp>
struct http_manager
{
void get_async(std::string url, boost::function<void(int)> on_response)
{
if (on_response)
{
on_response(42);
}
}
} http;
/// @brief Type that allows for registration of conversions from
/// python iterable types.
struct function_converter
{
/// @note Registers converter from a python callable type to the
/// provided type.
template <typename FunctionSig>
function_converter&
from_python()
{
boost::python::converter::registry::push_back(
&function_converter::convertible,
&function_converter::construct<FunctionSig>,
boost::python::type_id<boost::function<FunctionSig>>());
// Support chaining.
return *this;
}
/// @brief Check if PyObject is callable.
static void* convertible(PyObject* object)
{
return PyCallable_Check(object) ? object : NULL;
}
/// @brief Convert callable PyObject to a C++ boost::function.
template <typename FunctionSig>
static void construct(
PyObject* object,
boost::python::converter::rvalue_from_python_stage1_data* data)
{
namespace python = boost::python;
// Object is a borrowed reference, so create a handle indicting it is
// borrowed for proper reference counting.
python::handle<> handle(python::borrowed(object));
// Obtain a handle to the memory block that the converter has allocated
// for the C++ type.
typedef boost::function<FunctionSig> functor_type;
typedef python::converter::rvalue_from_python_storage<functor_type>
storage_type;
void* storage = reinterpret_cast<storage_type*>(data)->storage.bytes;
// Allocate the C++ type into the converter's memory block, and assign
// its handle to the converter's convertible variable.
new (storage) functor_type(python::object(handle));
data->convertible = storage;
}
};
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<http_manager>("HttpManager", python::no_init)
.def("get_async", &http_manager::get_async);
python::scope().attr("http") = boost::ref(http);
// Enable conversions for boost::function.
function_converter()
.from_python<void(int)>()
// Chaining is supported, so the following would enable
// another conversion.
.from_python<void(std::string)>()
;
}