IronPython throw InsufficientMemoryException when using numpy in threads

Question

I have some IronPython code that being called from within a C# application.
This code worked fine until I decided to change one function to run in a thread.
when numpy functions called in a python thread a InsufficientMemoryException exception is thrown.
I Searched for solutions but didn't find. can someone explain why it is happening and how can I fix it?

I think this is happening only when I have two threads that use numpy

I run code like this:

C#:

_python.functionA(); # _python was created with "Python.CreateEngine()"
_python.functionA(); # twice on purpose

Python:
my_python_script.py

import threading
import time
import numpy

def blah():    
    print numpy.array([100,100,0])

def functionA():
    t = threading.Timer(0,blah)    
    t.start()
    time.sleep(2)

And I got this Exception:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "c:\Program Files\IronPython 2.7.1\Lib\threading.py", line 552, in _Thread__bootstrap_inner
    self.run()
  File "c:\Program Files\IronPython 2.7.1\Lib\threading.py", line 756, in run
    self.function(*self.args, **self.kwargs)
  File "C:\workspace\my_python_script.py", line 113, in blah
    print numpy.array([100,100,0])
MemoryError: Exception of type 'System.InsufficientMemoryException' was thrown.

Thanks

UPDATE 13/07/14

I get this exception even when I run only one thread and via IronPython interpreter, without the C#:

C:\>"c:\Program Files\IronPython 2.7.1\ipy.exe"
IronPython 2.7.1 (2.7.0.40) on .NET 4.0.30319.18063
Type "help", "copyright", "credits" or "license" for more information.
>>> execfile(r"c:\workspace\my_python_script.py")
>>> functionA()
>>> Exception in thread Thread-1:
Traceback (most recent call last):
  File "c:\Program Files\IronPython 2.7.1\Lib\threading.py", line 552, in _Thread__bootstrap_inner
    self.run()
  File "c:\Program Files\IronPython 2.7.1\Lib\threading.py", line 756, in run
    self.function(*self.args, **self.kwargs)
  File "c:\workspace\my_python_script.py", line 6, in blah
    print numpy.array([100,100,0])
MemoryError: Exception of type 'System.InsufficientMemoryException' was thrown.

Answer 1

I believe that numpy is not thread-safe. I had a similar problem where using np.asarray() with threading would crash my program. It seems that the way that numpy's array function builds the array is not thread-safe. The way around this I found was to use np.fromiter() instead. Apparently, it is thread safe. It is slightly slower, which makes since if it is not using threading, but it works. Try putting your data into a list (or some other iterable data structure) and using np.fromiter() to convert it to a numpy array.

Also, just so you know, it actually runs fine on my computer so it could just be that you don't have enough memory to handle threading (or at least not when using numpy).