Take the 2-minute tour ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.

I have some IronPython code that being called from within a C# application.
This code worked fine until I decided to change one function to run in a thread.
when numpy functions called in a python thread a InsufficientMemoryException exception is thrown.
I Searched for solutions but didn't find. can someone explain why it is happening and how can I fix it?

I think this is happening only when I have two threads that use numpy

I run code like this:

C#:

_python.functionA(); # _python was created with "Python.CreateEngine()"
_python.functionA(); # twice on purpose

Python:
my_python_script.py

import threading
import time
import numpy

def blah():    
    print numpy.array([100,100,0])

def functionA():
    t = threading.Timer(0,blah)    
    t.start()
    time.sleep(2)

And I got this Exception:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "c:\Program Files\IronPython 2.7.1\Lib\threading.py", line 552, in _Thread__bootstrap_inner
    self.run()
  File "c:\Program Files\IronPython 2.7.1\Lib\threading.py", line 756, in run
    self.function(*self.args, **self.kwargs)
  File "C:\workspace\my_python_script.py", line 113, in blah
    print numpy.array([100,100,0])
MemoryError: Exception of type 'System.InsufficientMemoryException' was thrown.

Thanks

UPDATE 13/07/14

I get this exception even when I run only one thread and via IronPython interpreter, without the C#:

C:\>"c:\Program Files\IronPython 2.7.1\ipy.exe"
IronPython 2.7.1 (2.7.0.40) on .NET 4.0.30319.18063
Type "help", "copyright", "credits" or "license" for more information.
>>> execfile(r"c:\workspace\my_python_script.py")
>>> functionA()
>>> Exception in thread Thread-1:
Traceback (most recent call last):
  File "c:\Program Files\IronPython 2.7.1\Lib\threading.py", line 552, in _Thread__bootstrap_inner
    self.run()
  File "c:\Program Files\IronPython 2.7.1\Lib\threading.py", line 756, in run
    self.function(*self.args, **self.kwargs)
  File "c:\workspace\my_python_script.py", line 6, in blah
    print numpy.array([100,100,0])
MemoryError: Exception of type 'System.InsufficientMemoryException' was thrown.
share|improve this question
    
How many processors do you have? –  GoBrewers14 Jul 12 '14 at 23:37
    
I'm not sure right now, it on my work computer. But I believe it is at least dual, maybe quad. Is it important how exactly? –  Elisha Jul 12 '14 at 23:40
    
You might read this and this and see if that fixes your issue. –  GoBrewers14 Jul 12 '14 at 23:46
    
quad core, and that didn't fixed the problem :( –  Elisha Jul 13 '14 at 9:42
    
Are you sure that numpy is thread-safe? And the other libraries used? –  Alireza Jul 13 '14 at 10:24

1 Answer 1

I believe that numpy is not thread-safe. I had a similar problem where using np.asarray() with threading would crash my program. It seems that the way that numpy's array function builds the array is not thread-safe. The way around this I found was to use np.fromiter() instead. Apparently, it is thread safe. It is slightly slower, which makes since if it is not using threading, but it works. Try putting your data into a list (or some other iterable data structure) and using np.fromiter() to convert it to a numpy array.

Also, just so you know, it actually runs fine on my computer so it could just be that you don't have enough memory to handle threading (or at least not when using numpy).

share|improve this answer
    
I expected for a solution to this problem and not a workaround so I don't accept this answer yet (maybe solutions will come later). However, It might be that there is no better solution and your answer is the best so far, so you got the bounty –  Elisha Jul 20 '14 at 11:36

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.