Take the 2-minute tour ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.

I've got a total of around 7000 images from which I'm extracted HoG features. I then want to convert the list into an np array for further processing. But I get a memory error during the convertion.

Here's the relevant part of my code:

from skimage import data, io, filter, color, exposure
from skimage.feature import hog
from skimage.transform import resize
import matplotlib.pyplot as plt
import numpy as np

tmp_hogs = [] # this is the list I need to convert into a numpy array
for group in samplegroups:
    for myimg in group:
        curr_img = np.array(myimg, dtype=float)
        imgs.append(curr_img)
        fd, hog_image = hog(curr_img, orientations=8, pixels_per_cell=(4, 4),
                 cells_per_block=(1, 1), visualise=True, normalise=True)
        tmp_hogs.append(fd)

img_hogs = np.array(tmp_hogs, dtype =float) 

The error I get is:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Users\app\anacondasoftware\lib\threading.py", line 810, in __bootstrap_inner
    self.run()
  File "C:\Users\app\anacondasoftware\lib\site-packages\spyderlib\widgets\externalshell\monitor.py", line 582, in run
    already_pickled=True)
  File "C:\Users\app\anacondasoftware\lib\site-packages\spyderlib\utils\bsdsocket.py", line 45, in write_packet
    nsend -= temp_fail_retry(socket.error, sock.send, sent_data)
  File "C:\Users\app\anacondasoftware\lib\site-packages\spyderlib\utils\bsdsocket.py", line 25, in temp_fail_retry
    return fun(*args)
error: [Errno 10054] An existing connection was forcibly closed by the remote host

Traceback (most recent call last):
  File "C:\Users\app\Documents\Python Scripts\gbc_carclassify.py", line 63, in <module>
    img_hogs = np.array(tmp_hogs, dtype =float) 
MemoryError

How can I fix it?

share|improve this question
    
What size is tmp_hogs? And is it 1D? –  Davidmh Jun 21 at 13:51
    
How much memory do you have? How much memory does one hog occupy? Are you using 32 or 64-bit python? –  DrV Jun 21 at 17:34
    
32-bit python (Anaconda's spyder) on a 64-bit machine. –  user961627 Jun 22 at 9:34

1 Answer 1

For RGB or RGBA images you need only 8 bits per value, and when using float you are alocating 64 bits per value. Try using np.uint8 instead:

img_hogs = np.array(tmp_hogs, dtype=np.uint8)
share|improve this answer
2  
another improvement will be to create an empty array and store the images in this array directly, avoiding the intermediate list –  Saullo Castro Jun 21 at 14:15

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.