up vote 2 down vote favorite
share [g+] share [fb]

I'm using NumPy with Python 2.6.2. I'm trying to create a small (length 3), simple boolean array. The following gives me a MemoryError, which I think it ought not to.

import numpy as np
cond = np.fromiter((x in [2] for x in [0, 1, 2]), dtype = np.bool)

The error it gives me is:

MemoryError: cannot allocate array memory

However, the following method of obtaining a list (as opposed to an ndarray) works fine (without using numpy):

cond = list((x in [2] for x in [0, 1, 2]))

Have I done anything wrong in the Numpy code? My feeling is that it ought to work.

link|improve this question
1  
Your first code works for me as is. Can you post the versions of Python and Numpy? – Muhammad Alkarouri Sep 15 '10 at 12:13
For what it's worth, I can reproduce the problem using python 2.5 and numpy 1.1, but not with anything newer. On the older versions, it works fine if you manually specify the count kwarg (count=3, in this case). However, that defeats the purpose of using np.fromiter in the first place. I assmume it's a bug in numpy that's been fixed somewhere between 1.1 and 1.5? – Joe Kington Sep 15 '10 at 13:32
feedback

2 Answers

up vote 1 down vote accepted

I can reproduce the problem with numpy 1.1 (but not with anything newer). Obviously, upgrading to a more recent version of numpy is your best bet.

Nonetheless, it seems to be related to using np.bool as the dtype when count=-1 (the default: Read all items in the iterator, instead of a set number).

A quick workaround is just to create it as an int array and then convert it to a boolean array:

cond = np.fromiter((x in [2] for x in [0, 1, 2]), dtype=np.int).astype(np.bool)

An alternative is to convert it to a list, and then set count to the length of the list (or just use np.asarray on the list):

items = list((x in [2] for x in [0, 1, 2]))
cond = np.fromiter(items, dtype=np.bool, count=len(items))

Obviously, both of these are suboptimal, but if you can't upgrade to a more recent version of numpy, they will work.

link|improve this answer
+1: nice workaround. – EOL Sep 15 '10 at 13:58
Thank you! I did suspect it was a bug/version thing. – Underflow Sep 20 '10 at 12:59
feedback
up vote 1 down vote

You should not get any error.

With Python 2.6.5 or Python 2.7, and Numpy 1.5.0, I don't get any error. I therefore think that updating your software could very well solve the problem that you observe.

link|improve this answer
Thanks! I can't get a new version of numpy right now so I'll just work around it. – Underflow Sep 20 '10 at 13:00
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.