Boost.Python custom converter

Question

I have a class taking a vector as parameter (a binary file content).

I would like to convert python 'str' type into vector of unsigned char but only for one of my class method.

BOOST_PYTHON_MODULE(hello) {  class_<Hello>("Hello").
     // This method takes a string as parameter and print it
     .def("printChar", &Hello::printChar)
     // This method takes a vector<unsigned char> parameter
     .def("storeFile", &Hello::storeFile) }

Using custom converter seems to be what I need but if I modify my boost::python::converter::registry it will be modify for all my calls to printChar and all python methods passing string as parameter will be converted to vector.

How can I register a per-method converter ?

Answer 1

There are two approaches to this problem:

• Export a helper function as Hello.storeFile that accepts boost::python::str, constructs std::vector<unsigned char> from the string, and delegates to the C++ Hello::storeFile member function.
• Write a custom converter. While converters cannot be registered on a per-function basis, they are fairly well scoped as to not perform any unintended conversions. This approach often provides more reusability.

---

Helper Function

Using a helper function will not affect any other exported function. Thus, the conversion between a python string and std::vector<unsigned char> will only occur for Hello.storeFile.

void Hello_storeFile(Hello& self, boost::python::str str)
{
  std::cout << "Hello_storeFile" << std::endl;
  // Obtain a handle to the string.
  const char* begin = PyString_AsString(str.ptr());
  // Delegate to Hello::storeFile().
  self.storeFile(std::vector<unsigned char>(begin, begin + len(str)));
}

...

BOOST_PYTHON_MODULE(hello)
{
  namespace python = boost::python;

  python::class_<Hello>("Hello")
    // This method takes a string as parameter and print it
    .def("printChar", &Hello::printChar)
    // This method takes a vector<unsigned char> parameter
    .def("storeFile", &Hello_storeFile)
    ;
}

---

Custom Converter

A converter registration has three parts:

• A function that checks if a PyObject is convertible. A return of NULL indicates that the PyObject cannot use the registered converter.
• A construct function that constructs the C++ type from a PyObject. This function will only be called if converter(PyObject) does not return NULL.
• The C++ type that will be constructed.

Therefore, for a given C++ type, if converter(PyObject) returns a non-NULL value, then construct(PyObject) will create the C++ type. The C++ type serves as a key into the registry, so Boost.Python should not perform unintended conversions.

In context of the question, we want a converter for std::vector<unsigned char> where converter(PyObject) returns non-NULL if PyObject is a PyString, and converter(PyObject) will use PyObject to create and populate std::vector<unsigned char>. This conversion will only occur if for exported C++ functions that have a std::vector<unsigned char> (or a const reference) parameter and the argument provided from python is a string. Therefore, this custom converter will not affect exported functions that have std::string parameters.

Here is a complete example. I have opted to make the converter generic to allow multiple types to be constructable from a python string. With its chaining support, it should have the same feel as other Boost.Python types.

#include <iostream>
#include <list>
#include <string>
#include <vector>

#include <boost/foreach.hpp>
#include <boost/python.hpp>

class Hello
{
public:
  void printChar(const std::string& str)
  {
    std::cout << "printChar: " << str << std::endl;
  }

  void storeFile(const std::vector<unsigned char>& data)
  {
    std::cout << "storeFile: " << data.size() << ": ";
    BOOST_FOREACH(const unsigned char& c, data)
      std::cout << c;
    std::cout << std::endl;
  }
};

/// @brief Type that allows for conversions of python strings to
//         vectors.
struct pystring_converter
{

  /// @note Registers converter from a python interable type to the
  ///       provided type.
  template <typename Container>
  pystring_converter&
  from_python()
  {
    boost::python::converter::registry::push_back(
      &pystring_converter::convertible,
      &pystring_converter::construct<Container>,
      boost::python::type_id<Container>());
    return *this;
  }

  /// @brief Check if PyObject is a string.
  static void* convertible(PyObject* object)
  {
    return PyString_Check(object) ? object : NULL;
  }

  /// @brief Convert PyString to Container.
  ///
  /// Container Concept requirements:
  ///
  ///   * Container::value_type is CopyConstructable from char.
  ///   * Container can be constructed and populated with two iterators.
  ///     I.e. Container(begin, end)
  template <typename Container>
  static void construct(
    PyObject* object,
    boost::python::converter::rvalue_from_python_stage1_data* data)
  {
    namespace python = boost::python;
    // Object is a borrowed reference, so create a handle indicting it is
    // borrowed for proper reference counting.
    python::handle<> handle(python::borrowed(object));

    // Obtain a handle to the memory block that the converter has allocated
    // for the C++ type.
    typedef python::converter::rvalue_from_python_storage<Container>
                                                                 storage_type;
    void* storage = reinterpret_cast<storage_type*>(data)->storage.bytes;

    // Allocate the C++ type into the converter's memory block, and assign
    // its handle to the converter's convertible variable.  The C++
    // container is populated by passing the begin and end iterators of
    // the python object to the container's constructor.
    const char* begin = PyString_AsString(object);
    data->convertible = new (storage) Container(
      begin,                          // begin
      begin + PyString_Size(object)); // end
  }
};

BOOST_PYTHON_MODULE(hello)
{
  namespace python = boost::python;

  // Register PyString conversions.
  pystring_converter()
    .from_python<std::vector<unsigned char> >()
    .from_python<std::list<char> >()
    ;

  python::class_<Hello>("Hello")
    // This method takes a string as parameter and print it
    .def("printChar", &Hello::printChar)
    // This method takes a vector<unsigned char> parameter
    .def("storeFile", &Hello::storeFile)
    ;
}

And the example usage:

>>> from hello import Hello
>>> h = Hello()
>>> h.printChar('abc')
printChar: abc
>>> h.storeFile('def')
storeFile: 3: def
>>> h.storeFile([c for c in 'def'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Boost.Python.ArgumentError: Python argument types in
    Hello.storeFile(Hello, list)
did not match C++ signature:
    storeFile(Hello {lvalue}, std::vector<unsigned char, 
                                          std::allocator<unsigned char> >)

For more on custom converters and C++ containers, consider reading this answer.