Tell me more ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.
up vote 1 down vote favorite

I would like to plot an array of 20 millions object, I have 8GB RAM and still I get the following error when I run the following lines:

import matplotlib.pyplot as plt
import numpy as np

d = np.arange(200000000)
plt.plot(d)
plt.show()

Error:

Traceback (most recent call last):
...
  File "C:\Python27\lib\site-packages\matplotlib\axes.py", line 317, in _grab_next_args
    for seg in self._plot_args(remaining, kwargs):
  File "C:\Python27\lib\site-packages\matplotlib\axes.py", line 292, in _plot_args
    x = np.arange(y.shape[0], dtype=float)
MemoryError
share|improve this question
1  
It looks like you try to draw 200 000 000 objects not 20 000 000 – Xavier Combelle Sep 2 '11 at 14:03

2 Answers

up vote 2 down vote

Due to physical limitations of displays and plotters, you won't be able to plot 20,000,000 points anyway. So you could reduce your array by sampling it or by using means of slices:

>>> m = 20000000
>>> a = np.arange(m)
>>> n = 100 # <- reducing to 100 points
>>> s = m/n # <- size of slices to compress
>>> reduced = []
>>> for i in xrange(n):
...     slice = a[i*s:(i+1)*s]
...     reduced.append(np.mean(slice))
>>> reduced
[99999.5, 299999.5, ..., 19699999.5, 19899999.5]

.. assuming np.mean makes sence on the objects you're plotting.

share|improve this answer
up vote 2 down vote

MemoryError isn't lying--you had a failed memory allocation. This isn't really that unreasonable considering you're trying to plot 200 million points (note that the number you posted was 200 million, not 20 million).

It seldom or never makes sense to plot millions of points. When I have large datasets, I preprocess my data so that I am plotting no more than thousands of points. A simple regular sample would be fine for this for data like you have, but peakfinding can be necessary for great depictions of other data.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.