numpy.recfromcsv not releasing GIL

Question

I am using numpy.recfromcsv function to read a large amount of data. This is done in a worker thread so that my GUI (PySide) thread is not blocked. Even though I know of GIL that ensures that only one thread is being interpreted by Python at a time, I had very good experience with numpy operations. Most of them seem to release GIL so that while other processor cores are working on a numpy operation, my GUI thread remains responsive.

However numpy.recfromcsv causes my GUI to become sluggish. Does that mean that it does not release GIL? Or is there another reason? Is there a workaround to keep the GUI responsive even during this operation? I tried multiprocessing by this seems to be overkill and incredible pain with sharing or pickling large amounts of data.

Answer 1

np.recfromcsv uses np.genfromtxt (source) internally, which looks like it's just a pure python function, so I don't think what you're hoping for is possible with numpy.