How to release the GIL in Cython for a multithreaded C++ class?

Question

I have a C++ class with some methods that use std::thread that I'm making accessible to Python via Cython. Do you know where in my Cython code I'd want to put the nogill directive? Would I want to put it when I declare the class methods or when I create a Cython wrapper class? I've used the example class from the Cython docs below:

Declaring the class:

cdef extern from "Rectangle.h" namespace "shapes":
    cdef cppclass Rectangle:
        Rectangle() except +
        Rectangle(int, int, int, int) except +
        int x0, y0, x1, y1
        int getArea()
        void getSize(int* width, int* height)
        void move(int, int)

Cython wrapper class:

cdef class PyRectangle:
    cdef Rectangle c_rect      # hold a C++ instance which we're wrapping
    def __cinit__(self, int x0, int y0, int x1, int y1):
        self.c_rect = Rectangle(x0, y0, x1, y1)
    def get_area(self):
        return self.c_rect.getArea()
    def get_size(self):
        cdef int width, height
        self.c_rect.getSize(&width, &height)
        return width, height
    def move(self, dx, dy):
        self.c_rect.move(dx, dy)

Answer 1

You probably don't actually need to use nogil. The GIL only stops multiple Python threads being run simulatenously. However, given you're using C++ threads they can quite happily run in the background irrespective of the GIL, provided they don't try to use PyObjects or run Python code. So my suspicion is that you've misunderstood the GIL and you can get away with not thinking about it.

However, assuming that you do actually want to release it, you need to do 2 things:

1. Mark the C++ functions as nogil to tell Cython that they don't need the GIL. Note that this doesn't actually release it - it just lets Cython know it isn't a problem if it is released:

   cdef cppclass Rectange:
      Rectangle(int, int, int, int) nogil except +
      int getArea() nogil
      # ...
   

2. Use with nogil: blocks in your Cython wrapper class to mark the areas where the GIL is actually released.

   cdef class PyRectangle:
       # ...
       def __cinit__(self, int x0, int y0, int x1, int y1):
           with nogil:
               self.c_rect = Rectangle(x0, y0, x1, y1)
       def get_area(self):
           cdef int result
           with nogil:
               result = self.c_rect.getArea()
           return result
   

   get_area becomes slightly more complicated since the return statement can't exist inside the with nogil block since it involves generating a Python object.