current community

your communities

more stack exchange communities

Stack Exchange
tour help
Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.
up vote 5 down vote favorite
1

I am starting to have concerns about the ability of ArcGIS 10 to fulfill a deliverable of mine. It got me to wondering what exactly else there is, that can perform similar functionality?

ET Geowizards has been tried, although it doesn't have the same capability of ArcGIS e.g. I cannot aggregate all of the points I have with ET, to the scale I have them plotted.

I have no real experience of open source GIS, so am asking what is there, and what limitations do they have, as a synopsis.

Thanks in advance.

EDIT

To specifically state issues. There is a memory leak, which means looping through 700 objects, performing:

  1. Agg Points.
  2. Buffer.
  3. Add Field.
  4. Update cursor.

Starts off taking 5-9 secs per object, and continues on to 2 mins per (similarly sized) object.

In SP2, it appears AggPoints no longer works to create an FC on the fly. There is more, but it's too long a list to compile!

EDIT # 2

Code, simplied with no buffer, add field or cursor. geom is a collection or arcpy points

def createGeom(geom, scratchDB):
    filetime = (str(time.time())).split(".")
    outfile = "fc" + filetime[0]+filetime[1]
    outpath = scratchDB + "tmpV.gdb/Polygon/"  
    outFeatureAggClass = outpath+outfile +"_Agg"
    arcpy.AggregatePoints_cartography(geom, outFeatureAggClass,"124000 meters")

EDIT #3

Seems to be that the size of the FGDB we're writing to is one of the main issues; I think the problem is that performance degrades as the local file gdb fills up - significantly.

Any ideas how to tune local fgdb?

share|improve this question
    
can you be more specific or give some examples highlighting your concerns with ArcGIS 10? –  artwork21 Aug 1 '11 at 13:00
    
Comment added. I would love to do a concave hull, but the only code kicking around isn't suitable for the grid of points I am using, as it's simply too large. I am using world grids of around 60km each. –  Hairy Aug 1 '11 at 13:47
    
downvote because the Q is too broad. I think people are upvoting on the title alone. It's a theme many are interested in (myself included!), but it doesn't make a good question. –  matt wilkie Aug 2 '11 at 16:50
1  
In terms of cursors, I have found ArcGIS to be a nightmare when working in Python. C# has been much more amenable to the task of large cursor processes, though. –  Nathanus Aug 2 '11 at 17:55
    
The cursor is small, and I am using it to update one object. I have also removed the cursor to show it isn't that, which it isn't. Calling arcpy causes the leaks. –  Hairy Aug 3 '11 at 6:58

1 Answer 1

up vote 2 down vote accepted

The question as stated is too broad, though the edit enumerating the outline does help, a little. Looking for a reason to use something other than arcgis for job X? then you need to describe what Job X is. In many format conversion cases / kicks butt on arcgis performance wise. On the other hand gdal/ogr sucks at making maps!

Want help fixing memory leaks? Then show (the simplest possible form of) the code. It might benefit from using a dictionary instead of cursor, or in-memory workspace, or a change the workflow (for example aggregate & buffer then cursor instead of cursor then aggregrate & buffer [ref]), or... In any case, start here: Performance of ArcGISScripting and large spatial data sets

share|improve this answer
    
Matt, I though I had made it clear the workflow? The cursor only ever updates one record, the record it has created with the agg points and buffer; I cannot update the data, without first adding the fields, and creating the dataset, which I am doing with the agg points, and buffer. I don't find this sueful at all matt, to be honest. –  Hairy Aug 3 '11 at 6:56
    
Matt's suggestions are sound. When you say there is a memory leak, how much memory is it actually using? If you are using an in-memory workspace, take care to either overwrite or delete anything you create there when it is no longer needed so that you don't fill up your RAM and start hitting the pagefile instead. We would need to see how you are calling your createGeom in a loop to say whether/how that could be improved. –  blah238 Aug 3 '11 at 7:55
    
Also are you hitting an SDE feature class in this script? I ran into an actual memory leak in a very specific situation involving a direct connection to an SDE feature class which I described here. It's possible, though unlikely, that the same leak is happening here. –  blah238 Aug 3 '11 at 8:01
    
I think you can see no in-memory workspace is being used. It's using a set of points, to create a featureClass. They are all being written to a local FGDB. I said Matt wasn't being useful, as I had described the workflow clearly, before he wrote his post. He also got the order wrong using the cursor. –  Hairy Aug 3 '11 at 12:27
    
@Hairy apologies for lack of usefulness. It appears there's enough of that to flow in both directions, as what was described as a clear workflow for you was anything but for me. ;-) In any case, on with the troubleshooting! (in additional comments) –  matt wilkie Aug 3 '11 at 17:54

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.