I have some IronPython
code that being called from within a C#
application.
This code worked fine until I decided to change one function to run in a thread.
when numpy functions called in a python thread a InsufficientMemoryException
exception is thrown.
I Searched for solutions but didn't find. can someone explain why it is happening and how can I fix it?
I think this is happening only when I have two threads that use numpy
I run code like this:
C#:
_python.functionA(); # _python was created with "Python.CreateEngine()"
_python.functionA(); # twice on purpose
Python:
my_python_script.py
import threading
import time
import numpy
def blah():
print numpy.array([100,100,0])
def functionA():
t = threading.Timer(0,blah)
t.start()
time.sleep(2)
And I got this Exception:
Exception in thread Thread-1:
Traceback (most recent call last):
File "c:\Program Files\IronPython 2.7.1\Lib\threading.py", line 552, in _Thread__bootstrap_inner
self.run()
File "c:\Program Files\IronPython 2.7.1\Lib\threading.py", line 756, in run
self.function(*self.args, **self.kwargs)
File "C:\workspace\my_python_script.py", line 113, in blah
print numpy.array([100,100,0])
MemoryError: Exception of type 'System.InsufficientMemoryException' was thrown.
Thanks
UPDATE 13/07/14
I get this exception even when I run only one thread and via IronPython interpreter, without the C#:
C:\>"c:\Program Files\IronPython 2.7.1\ipy.exe"
IronPython 2.7.1 (2.7.0.40) on .NET 4.0.30319.18063
Type "help", "copyright", "credits" or "license" for more information.
>>> execfile(r"c:\workspace\my_python_script.py")
>>> functionA()
>>> Exception in thread Thread-1:
Traceback (most recent call last):
File "c:\Program Files\IronPython 2.7.1\Lib\threading.py", line 552, in _Thread__bootstrap_inner
self.run()
File "c:\Program Files\IronPython 2.7.1\Lib\threading.py", line 756, in run
self.function(*self.args, **self.kwargs)
File "c:\workspace\my_python_script.py", line 6, in blah
print numpy.array([100,100,0])
MemoryError: Exception of type 'System.InsufficientMemoryException' was thrown.
I believe that numpy is not thread-safe. I had a similar problem where using np.asarray()
with threading would crash my program. It seems that the way that numpy's array
function builds the array is not thread-safe. The way around this I found was to use np.fromiter()
instead. Apparently, it is thread safe. It is slightly slower, which makes since if it is not using threading, but it works. Try putting your data into a list (or some other iterable data structure) and using np.fromiter()
to convert it to a numpy array.
Also, just so you know, it actually runs fine on my computer so it could just be that you don't have enough memory to handle threading (or at least not when using numpy).
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With