current community

your communities

more stack exchange communities

Stack Exchange
tour help
Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.
up vote 3 down vote favorite
1

I'm running a rather simple script with ArcPy, which extracts values of raster to points of a shapefile:

def raster_overlay(input_files):
    for raster in input_files:
        in_shp = "temp/shapefiles/points.shp"
        in_raster = raster
        print in_raster
        ExtractMultiValuesToPoints(in_shp, in_raster, "BILINEAR")

raster_overlay(raster_files)

The variable raster_files is a list with strings pointing to a number of raster files on the local hard disk.

The shapefile contains around 50 points, and each raster is around 15MB.

If the list of raster is small (more or less <20), everything works as expected. However, I have around 50 raster images. Using the complete list, Python crashes ("python.exe has stopped working") after processing 21 images or so.

The values of the first 20 raster are correctly appended to the shapefile (one column for each raster).

I monitored python.exe in the Windows 7 Resource Monitor. Memory usage of Python starts around 15 MB and then gradually increases to around 700 MB (and then Python crashes). So, memory leakage might be the reason for the crashes (although the PC has 8 GB of RAM).

I tried to include gc.collect() and del in_shp and del in_raster inside the loop, but nothing changed. What else could I try?

PS: I just tried changing BILINEAR to NONE, and the script didn't crash. However, memory usage still increased in the same way (up to around 1.1 GB). And I would like to use bilinear interpolation, so the question is still relevant.

Edit: Cross-posted here to the ESRI support. I'll update this question if they can give a solution.

share|improve this question
2  
Sounds like a bug in the ExtractMultiValuesToPoints tool. I've had terrible problems in the past with raster processing in arcpy and looping. Try firing this off to ESRI support? –  Hornbydd Sep 10 '13 at 16:10
    
FYI, your reference to your shapefile should be made out side of your loop. –  artwork21 Sep 10 '13 at 16:11
1  
You may also want to try writing to a geodatabase feature class instead of a shapefile. See if that generates the same error. –  artwork21 Sep 10 '13 at 16:13
1  
Wrap ExtractMultiValuesToPoints with a Try, Except to find the images causing the error, etc. –  klewis Sep 10 '13 at 17:05
    
@klewis try: except: won't catch a hard crash/segfault/core dump like the OP describes. –  Luke Sep 11 '13 at 1:14

1 Answer 1

up vote 1 down vote

What is your memory use when you replace your script with

def raster_overlay(input_files):
    for in_raster in input_files:
        in_shp = "temp/shapefiles/points.shp"
        #in_raster = raster
        #print in_raster
        ExtractMultiValuesToPoints(in_shp, in_raster, "BILINEAR")

raster_overlay(raster_files)

You may be eating up memory by creating 2 copies of the raster then printing it

share|improve this answer
    
You should also move in_shp outside of the for loop or make it a parameter to pass to the function. –  Paul Sep 10 '13 at 18:31
    
I just tried your suggestion, but the result is the same. raster_files is just a list of strings, so using print shouldn't cause any memory leakage in my understanding. Also, moving in_shp outside of the loop didn't change anything. –  Jon Snow Sep 11 '13 at 7:59

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.