Tell me more ×
Code Review Stack Exchange is a question and answer site for peer programmer code reviews. It's 100% free, no registration required.

I'm trying to optimize this very slow loop (list comprehension, map..?) - would really appreciate any insights. x is an object with three attributes, w, y and z. xs is a list of x's. alpha and beta are constants.

for x in xs:
    x.y = function(x.z)
    x.w = alpha * x.y * x.y * (1+x.z)**beta
share|improve this question
Are all these attributes floats? – DSM Feb 9 '12 at 22:13
4  
not enough information. – Hamish Feb 9 '12 at 22:13
1  
How many x do you have in xs? What does function(x.z) do? – dgraziotin Feb 9 '12 at 22:16
7  
@user1021819: did you profile the code? If function is a numerical integration, it's probably taking more time than anything else, so optimizing anything else wouldn't help much. – DSM Feb 9 '12 at 22:24
4  
Maybe if function is smooth, you can sample it at, say, 1000 equally spaced points (instead of 300,000), memoize those results, and interpolate between those points to estimate function(x.z). – unutbu Feb 9 '12 at 22:33
show 8 more comments

migrated from stackoverflow.com Feb 9 '12 at 23:35

3 Answers

Check out http://numpy.scipy.org/ This minimal Python example won't get much faster without C-frameworks.

share|improve this answer

If you're trying to speed up getting something that cosmocalc returns -- say a zage -- then as unutbu suggests you should be able to get away with an approximation, because the functions are nice and smooth. For example, if you cared about the zage, then while the function itself is very slow:

>>> import timeit
>>> 
>>> setup = """
... from cosmocalc import cosmocalc
... def zage(z): 
...     return cosmocalc(z)['zage']
... """
>>> 
>>> timeit.timeit(setup=setup, stmt='zage(5.4)',number=100)
0.38316917419433594

A spline can be both fast and accurate:

>>> import cosmocalc
>>> from scipy import linspace
>>> from scipy.interpolate import InterpolatedUnivariateSpline
>>> zs = linspace(0, 20, 100)
>>> zages = [cosmocalc.cosmocalc(z)['zage'] for z in zs]
>>> ZZ = InterpolatedUnivariateSpline(zs, zages)
>>> 
>>> # check the errors
... zcheck = linspace(0, 20, 1000)
>>> approx = ZZ(zcheck)
>>> real = [cosmocalc.cosmocalc(z)['zage'] for z in zcheck]
>>> abserr = abs((real-approx))
>>> relerr = abs((real-approx)/real)
>>> print max(abserr), max(relerr)
0.000144758262362 0.000157948469763

with less than 5 seconds to compute 300000 terms now:

In [19]: zs = linspace(0, 20, 3*10**5)

In [20]: time zcalc = [ZZ(z) for z in zs]
CPU times: user 4.56 s, sys: 0.05 s, total: 4.61 s
Wall time: 4.87 s

or if we build a numpy array instead of a list:

In [22]: time zcalc = ZZ(zs)
CPU times: user 0.05 s, sys: 0.00 s, total: 0.05 s
Wall time: 0.08 s

Caveat: if you're using the results for something else, say if you're trying to take a numerical derivative, then you should be careful. But if you only care about the values themselves, then a spline should more than suffice, whatever output of the cosmocalc routine you're interested in. Always make sure to look at the results and the errors visually to catch any hijinks the function might be trying to pull.

Other options include: (1) using a chebyshev fit, say from mpmath; (2) fixing the numerical integration routines in cosmocalc themselves-- they're designed for portability, not for speed.

share|improve this answer

Without more information (a dataset, and what function does), it's hard for us to say. However, absent any more information I would try running it under PyPy: http://pypy.org/, which can often execute Python code significantly faster than CPython.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.