NumPy is the fundamental package for scientific computing with the programming language Python.
0
votes
0answers
5 views
Cython - Fastest way to iterate over Numpy array
I wrote a function to calculate the gamma coefficient of a clustering. The bottleneck is the comparison of values from dist_withing to dist_between. To speed this up, I tried to adapt and compile it ...
2
votes
1answer
21 views
Help improve my my python code for Exponential Smoothing Sumitted to Statsmodel
I just submitted this pull request to statsmodels (a Python toolkit for statistical and econometric modeling). It adds support for exponential smoothing of time series.
I have only been programming ...
0
votes
1answer
69 views
Fastest way for working with itertools.combinations
I need to speed up the function below:
import numpy as np
import itertools
def combcol(myarr):
ndims = myarr.shape[0]
solutions = []
for idx1, idx2, idx3, idx4, idx5, idx6 in ...
1
vote
2answers
65 views
Elegantly avoid columns containing NaNs?
The next bit of code does exactly what i want it to do, but it is 'rather' unelegant and unpythonic, so the question is quite simple: how can this code be changed into something more elegant
import ...
4
votes
1answer
42 views
Organization of complex empirical equations
I am working on coding up a set of models that are based on empirical models. These models tend to be quite complicated. In some cases, the logic used by the equations makes it impossible to compute ...
1
vote
1answer
29 views
Downsampling n-dimensional data from bins in one dimension
So I'm writing some code to perform a quite specific task given a large numpy array with N rows and 3 columns representing points in 3D. The 3D points are to be binned along one of the dimensions ...
0
votes
1answer
45 views
How to reduce runtime of gene data processing program?
This program takes a huge data set as input, processes it, calculates and then writes the output to an array. Most calculations may be quite simple, such as summation. In the input file, there are ...
1
vote
2answers
32 views
How can this code be made more Pythonic?
This next part of code does exactly what I want it to do. dem_rows and dem_cols contain float values for a number of things i can identify in an image, but i need to get the nearest pixel for each of ...
1
vote
3answers
209 views
NumPy Scipy optimization
I have an optimization issue and I'm not sure if I can improve the overall speed of my function.
The function draw_w is my actual implementation that gives me the right update for my vector w. You ...
4
votes
1answer
141 views
Optimizing NumPy code
I'm try to implement SSIM method. This method has already been implemented in Python (source code), but my goal is implement it with using only Python and NumPy. Also my goal is to using this method ...
4
votes
1answer
131 views
Optimizing numpy/python for speed
This is my first real python script after "Hello, World!" I'd really appreciate feedback to make it faster and more efficient.
The purpose of this script is to calculate the nonlinear reflection ...
2
votes
1answer
92 views
What would be the optimal way to address these arrays
I think the arrays here are slowing down my code due to incorrect usage.
There is a lot of stuff going on here, it's understandable if nobody responds. Just looking for some pointers. Thanks.
for ...
4
votes
2answers
289 views
Is this the fastest way to find the closest point to a list of points using numpy?
I'm trying to find the closest point (Euclidean distance) from a user inputed point to a list of 50,000 points that I have. Note that the list of points changes all the time. and the closest distance ...
2
votes
0answers
33 views
Summing 2D NumPy array by multiple lables
Each element of the array is a coordinate (x, y).
Each coordinate has two labels.
Goal: sum the elements that have the same two labels.
How can this be faster?
>>> import numpy
...
2
votes
1answer
68 views
Eliminate for loops in numpy implementation
I have the following dataset in numpy
indices | real data (X) |targets (y)
| |
0 0 | 43.25 665.32 ... |2.4 } 1st block
0 0 | 11.234 |-4.5 }
0 ...
5
votes
2answers
308 views
Critiques on a trivially easy to use Python CSV class
I have been working on a project where I needed to analyze multiple, large datasets contained inside many CSV files at the same time. I am not a programmer but an engineer so I did a lot of searching ...
2
votes
1answer
244 views
Using numpy polynomial module - is there a better way?
I'm working on a project where I need to solve for one of the roots of a quartic polymonial many, many times. Is there a better, i.e., faster way to do this? Should I write my own C-library? The ...
2
votes
1answer
55 views
How can I speed up this correlation function Python
I am writing a specialized version of the cross correlation function as used in neuroscience. The function below is supposed to take a time series data and ask how many of its values fall in specified ...
3
votes
1answer
262 views
Are these list-comprehensions written the fastest possible way?
This is a simple repeatable question regarding the usage of Python3's comprehensions:
Could the Python3 syntax be used in another way to speed up the process, further so the gap between Python3 and ...
2
votes
1answer
126 views
Performance problems with 1D cellular automata experiment in Python with Numpy
I'm an experienced programmer but relatively new to Python (a month or so) so it's quite likely that I've made some gauche errors. I very much appreciate your time spent taking a look at my code.
My ...
0
votes
1answer
94 views
Figuring out the structure of a shape tuple
I have some code that uses a multidimensional look-up table (LUT) to do interpolation. The typical application is a color space conversion, where 3D inputs (RGB) are converted to 4D (CMYK), but the ...
2
votes
3answers
236 views
A pythonic way of de-interleaving a list (i.e. data from a generator), into multiple lists
I've recently discovered the wonders of the Python world, and am quickly learning. Coming from Windows/C#/.NET, I find it refreshing working in Python on Linux. A day you've learned something new is ...
2
votes
0answers
199 views
Is there better way to read from a file and connect all data in one big data than to use generators?
Is there better way to read from a file and connect all data in one big data than to use generators?
At the moment, I do the following:
use generators to read data from files.
use numpy to pack all ...
3
votes
1answer
151 views
How can I make this code faster using Numpy?
I have been reading some Numpy guides but can't seem to figure it out. my TA told me I should be able to speed up my code by using a numpy array instead of a for loop in the following segment of code.
...
4
votes
1answer
171 views
Optimizing Python / Numpy Code
So I'm implementing a version of the mean shift image processing algorithm for color segmentation in python/numpy.
I've written a pure numpy version of the actual mean shifting per pixel (which I ...
6
votes
2answers
545 views
Python / Numpy running 15x slower than MATLAB - am I using Numpy effeciently?
Here is some code I wrote in Python / Numpy that I pretty much directly translated from MATLAB code. When I run the code in MATLAB on my machine, it takes roughly 17 seconds. When I run the code in ...
2
votes
2answers
124 views
Simple arithmetic in Python
As a beginner, I wrote the following python script that solves warrant 1 of this document - pp 436-438.
My solution, although works, seems to me as poorly designed and highly unmaintanable. I was ...
1
vote
1answer
124 views
Python comparison numpy matrix
I need a numpy matrix with comparisons among all elements in a sequence:
from numpy import matrix
seq = [0, 1, 2, 3]
matrix([[cmp(a, b) for a in seq] for b in seq])
I'd like to know if there is a ...
1
vote
1answer
320 views
Data cleaning - am I using classes correctly here?
I have a data processing application that does some data cleaning as the first step. This is the module I import for that purpose.
The only 'public' part of the api is the 'clean' function which ...
2
votes
1answer
108 views
Simple utility/convenience module - am I doing it right?
I have a large library (four packages) of data processing code that I have written over the last 12 months. I am quite new to python and would like to see if I am doing things correctly.
This ...
1
vote
1answer
418 views
Linear Regression and data manipulation
How could I improve the following code that runs a simple linear regression using matrix algebra? I import a .csv file (link here) called 'cdd.ny.csv', and perform the matrix calculations that solve ...
6
votes
1answer
250 views
Code Review of small scientific project, particuarly array vs. list perform
I'm new to python but not to programming in general. This is my first project beyond the basics.
I was wondering if I could get some feedback on the code in general, in particular on any bad Java/C++ ...
1
vote
2answers
376 views
Python: optimize this rolling loop
I have a numpy array of about 2500 data points. This function is called on a rolling basis where 363 data points is passed at a time.
def fcn(data):
a = [data[i]/np.mean(data[i-2:i+1])-1 for i in ...
4
votes
5answers
519 views
Python function speedup - how to?
I wrote a simple function that calculates and returns the maximum drawdown of a set of returns. I am trying to squeeze as much efficiency out of the code as possible for speed. I've got it down to ...
