How to iterate throught C++ sets in Cython?

Question

I am optimizing python code with Cython. A set in C++ is storing all my results, and I do not know how to access the data to move it into a Python object. The structure must be a set. I cannot change it to a vector, list, etc.

I know how to do this in Python and C++, but not in Cython. How are iterators retrieved in Cython? I get STL containers via libcpp.STLContainer as in

from libcpp.vector cimport vector

But, I don't know how iterators work in Cython. What do I need to import? And, is there any change in syntax for using iterators compared to how they work in C++?

Answer 1

Cython should automatically convert a c++ set to a python set when needed, however if you really do need to use iterators on the c++ object you can do that as well.

If we make a very simple example where we construct a set in c++

libset.cc

#include <set>

std::set<int> make_set()
{
    return {1,2,3,4};
}

libset.h

#include <set>

std::set<int> make_set();

We can then write the cython wrapper for this code as, where I've given an example of how to iterate through the set in a nice pythonic way (which uses the c++ iterators in the background) and an example of how to do it directly with iterators.

pyset.pyx

from libcpp.set cimport set
from cython.operator cimport dereference as deref, preincrement as inc

cdef extern from "libset.h":
    cdef set[int] _make_set "make_set"()

def make_set():
    cdef set[int] cpp_set = _make_set()

    for i in cpp_set: #Iterate through the set as a c++ set
        print i

    #Iterate through the set using c++ iterators.
    cdef set[int].iterator it = cpp_set.begin()
    while it != cpp_set.end():
        print deref(it)
        inc(it)

    return cpp_set    #Automatically convert the c++ set into a python set

This can then be compiled with a simple setup.py

setup.py

from distutils.core import setup, Extension
from Cython.Build import cythonize

setup( ext_modules = cythonize(Extension(
            "pyset",
            sources=["pyset.pyx", "libset.cc"],
            extra_compile_args=["-std=c++11"],
            language="c++"
     )))

Answer 2

Very nice answer by Simon. I had to do this for a C++ map to python dict. Here's my rough cython code for the map case:

from libcpp.map cimport map

# code here for _make_map() etc.

def get_map():
    '''
    get_map()
    Example of cython interacting with C++ map.

    :returns: Converts C++ map<int, int> to python dict and returns the dict
    :rtype: dict
    '''
    cdef map[int, int] cpp_map = _make_map()

    pymap = {}
    for it in cpp_map: #Iterate through the c++ map
        pymap[it.first] = it.second

    return pymap