Tagged Questions
0
votes
0answers
20 views
Sampling at pre-defined time values
This is a follow-up question to this other question: Causal resampling: Sum over the last X <time_unit>
Say I have the following time series:
money_spent
...
1
vote
0answers
25 views
Numpy fast check for complete array equality, like Matlabs isequal
In Matlab, the builtin isequal does a check if two arrays are equal. If they are not equal, this might be very fast, as the implementation presumably stops checking as soon as there is a difference:
...
0
votes
3answers
29 views
Numpy only way to read text file and keep comments
Here is a minimal working example of a text file:
# A B C
1 7 9
7 2 10
10 20 30
Loading this file using numpy.loadtxt will discard the commented line. Is there a nice way to map the columns stored ...
0
votes
0answers
14 views
Pandas/Numpy: Calculate current state series based on binary signals
I have 2 timeseries of binary "signals", let's call them "entry" and "stay".
Entry==1 means add 1 to current state (for some maximum amount of time) and stay==0 means set current state to 0.
entry:
...
1
vote
1answer
15 views
Can lambdify return an array with dytpe np.float128?
I am solving a large non-linear system of equations and I need a high degree of numerical precision. I am currently using sympy.lambdify to convert symbolic expressions for the system of equations and ...
1
vote
2answers
35 views
Re-arranging numpy array in place
I am trying to modify a numpy array "in-place". I am interested in re-arranging the array in-place (instead of return:ing a re-arranged version of the array).
Here is an example code:
from numpy ...
0
votes
0answers
12 views
Measure the autocorrelation at lag 1 in pyhon
I am struggling to measure the (statistical) autocorrelation at lag-1 of the following dataset (time - depth):
http://s17.postimg.org/ta77u890f/figure_1.png
I would like to estimate the first value ...
1
vote
1answer
24 views
Piecewise functions on Numpy Arrays
What is efficient (speed) way to apply Piecewise functions on Numpy Array?
Say, for example, Piecewise functions are like
For (1) : x<=2 f(x) = 2*x + x^2
(2) : x>2 f(x) = -(x^2 + 2)
...
-1
votes
0answers
11 views
data processing, correlation calculation
I have product purchase count data which looks likes this:
user item1 item2
a 2 4
b 1 3
c 5 6
... ... ...
These data are imported into python using ...
-2
votes
1answer
20 views
Numpy multiply each column of array A by the corresponding row of array B, without looping
Suppose I have two 2-d arrays, A and B. A is of size 3x3 and B 3x5. I would like to np.dot the 1st column of A (size 3x1) with the 1st row of B (size 1x5), and the 2nd column of A with the 2nd row of ...
0
votes
0answers
20 views
Logarithmic colormap in matplotlib
Is it possible to define a matplitlib colormap which logarithmically interpolates between two given colors?
Background: When plotting arrays in matplotlib via the imshow(...) command, colormaps are ...
2
votes
3answers
74 views
Can I vectorise this python code?
I have written this python code to get neighbours of a label (a set of pixels sharing some common properties). The neighbours for a label are defined as the other labels that lie on the other side of ...
0
votes
1answer
22 views
Combining multiple numpy arrays into one
I have this O/P as shown in the below pic
My O/P consists of around 100+ numpy arrays like the one shown above. I'm trying to combine all these 100+ numpy arrays into an single numpy array for ...
0
votes
3answers
39 views
Numpy: Return 0 with divide by zero
I'm trying to perform an element wise divide in python, but if a zero is encountered, I need the quotient to just be zero.
I could always just use a for-loop through my data, but to really utilize ...
1
vote
2answers
78 views
Insertion of non aligned elements in 3-dimensional matrices in numpy
I'm working with 3-dimensional matrices using numpy 1.9 and python 2.7.5.
Here is an example:
>>> A
array([[[ 1., 1., 1.],
[ 1., 1., 1.],
[ 1., 1., 1.],
[ 1., ...
1
vote
1answer
15 views
Numpy - Creating a function whose inputs are two matrices and whose output is a binary matrix in Python
I am trying to write a function that take two m x n matrices as input and gives a binary matrix output, where 0 is returned if an element m,n is less than zero and returns 1 if else. I want these ...
0
votes
1answer
22 views
Reducing pandas series with multiple nan values to a set gives multiple nan values
I'm expecting to get set([nan,0,1]) but I get set([nan, 0.0, nan, 1.0]):
>>> import numpy as np
>>> import pandas as pd
>>> l= [np.nan,0,1,np.nan]
>>> ...
0
votes
1answer
15 views
Install python module on synology - pip error
I'm trying to install additional python module on my Synology DS112J. I installed pip and now i'm trying to install modules. It works with the module "requests" but don't with matplotlib and numpy.
...
1
vote
1answer
30 views
Time differentiation in Pandas
Say I have a dataframe with several timestamps and values. I would like to measure Δ values / Δt every 2.5 seconds. Does Pandas provide any utilities for time differentiation?
...
1
vote
1answer
46 views
Save data from plot to numpy array
I'm wondering how could I save the data content of a plot generated using Matplotlib to a Numpy array.
As a example, suppose I generated a contour plot with the following code:
import matplotlib
...
0
votes
4answers
78 views
Algorithmic help in Python, find pair (x,y) where y/x > const
I'm building a rather huge real-time odds system, and my bottleneck right now is the actual computation. I have a huge amount of sorted lists, and for each list, I need to find each pair (x,y) where ...
1
vote
3answers
56 views
Numpy: vector of indices and value
I guess I'm having a slow day and can't figure this one out. I have an m x n numpy array and want to convert it to a vector where each element is a 3 dimensional vector containing the row number, ...
2
votes
2answers
44 views
in-place numpy array sorting according to given index
There are some questions that come close, but I haven't found a specific answer to this. I'm trying to do some in-place sorting of a numpy 3D array along a given axis. I don't want simple sorting ...
2
votes
3answers
42 views
Creating numpy array with different steps
I would like to create an array like the following:
# 0 | 4 | 8 | 16 | 32
In which, each element except the first, is the double of the previous one.
I can create this smaller through an iteration ...
0
votes
0answers
20 views
Update PyGTK plot of numpy array at a time interval
I've been stuck on this for a little while. I'm new to PyGTK (and GUIs in general), so please forgive me if this is a particularly dumb question. Due to circumstances beyond my control, I'm having to ...
0
votes
0answers
22 views
Duplicate values from array for new array with python [duplicate]
My input array is: [ 37. -18. 27.]
Now I wanted to create such array (every value four times):
[ 37. 37. 37. 37. -18. -18. -18. -18. 27. 27. 27. 27.]
import numpy as np
abc = (37.0, ...
2
votes
1answer
26 views
Using np.savetxt and np.loadtxt with multi-dimensional arrays
What is a generalized way of storing a more-than-2 dimensional array (ndim > 2) to file and retrieving it in same format (dimension) using np.savetxt and np.loadtxt?
My concern is if I give any ...
0
votes
0answers
25 views
Illegal instruction when computing big matrix with numpy.cov
Everything is in the title. I'm trying to compute a covariance matrix with numpy.cov. I've got N samples of 50 observables. I'm simply doing this (as a test):
input_data=np.ones((50,N))
cov_matrix = ...
2
votes
2answers
71 views
Why is Numpy outperforming this cython routine by a factor 3
I have just started experimenting with cython and as a first exercise I created the following (re)implementation of a function computing the sin for each element of an array. So here's my sin.pyx
...
1
vote
1answer
20 views
Calculate mannwhitneyu over specified numpy axis
The Mann-Witney-U-test is the non-parametric equivalent to the t-test which can only be used for normally distributed data. In Scipy both test are available:
t-test: ...
-2
votes
0answers
39 views
Calculate mean of a certain group of elements
I have a time series data with values every 5 sec
Is it possible to calculate the means of elements, for each 15 minutes for example?
Format of data:
06/08/14
Format of time:
00:00:00.00
3
votes
3answers
81 views
Efficent insertion of not aligned elements in a numpy array
I'm using numpy 1.9 to work on a set of arrays. Assuming I have something like that I have two 2d arrays A and B and a 1-d array C, that looks like that:
>>> A
array([[ 1., 1., 1., 1., ...
2
votes
2answers
46 views
Huge sparse matrix in python
I need to iteratively construct a huge sparse matrix in numpy/scipy. The intitialization is done within a loop:
from scipy.sparse import dok_matrix, csr_matrix
def foo(*args):
dim_x = ...
-1
votes
3answers
32 views
Declare empty numpy.ndarray and fill it
Given a numpy array 'x' and a hop size 'N', I have to create a function that will return a numpy.ndarray with the values of 'x' that fit the hop size, for example, if x = [0,1,2,3,4,5,6,7,8,9] and N = ...
1
vote
3answers
26 views
Numpy Image Arrays: How to efficiently switch from RGB to Hex
I have been using nested for loops to turn RGB images into an image array of Hex values, but it is too slow for large images. Does anyone know a quick way and or a library that can help me switch back ...
2
votes
1answer
24 views
Subclassing and extending numpy.ndarray
I need some basic data class representations and I want to use existing numpy classes, since they already offer great functionality.
However, I'm not sure if this is the way to do it (although it ...
0
votes
0answers
44 views
Python 3.3 import matplotlib in Windows don't get along
Well I have Python 3.3 and I need matplotlib to function properly.
I used the pip install matplotlib to install it.
Have installed Six, dateutils, Scipy, numpy and even pyparsing for it to work.
...
0
votes
0answers
14 views
Pandas Creating Panel from Multiple DataFrames Buffer Has Wrong Dimesnions
This is a continuation of an early SO for which a solution was found to create DataFrames, my issue now is making the Panel. I'm doing the following:
df1 = DataFrame(gps_data, index=gps_time, ...
1
vote
2answers
25 views
pick TxK numpy array from TxN numpy array using TxK column index array
This is an indirect indexing problem.
It can be solved with a list comprehension.
The question is whether, or, how to solve it within numpy,
When
data.shape is (T,N)
and
c.shape is (T,K)
...
1
vote
2answers
53 views
Python / Cython / Numpy optimization of np.nonzero
I have a piece of code that I am trying to optimize. The majority of the code execution time is taken by cdef np.ndarray index = np.argwhere(array==1)
Where array is a numpy is a 512x512,512 numpy ...
1
vote
0answers
30 views
python: change numpy.array access method to start at 1 instead of 0
I'm porting some code from R to python (note: in R lists start at the 1st, not 0th element) and instead of changing every place I access an array I want to create an child class of numpy.array so ...
1
vote
1answer
22 views
plotting a histogram of a numpy array by timestamp
My array has pairs of unix timestamps and values.
[[ 1.40170249e+09 9.00000000e+01]
[ 1.40170249e+09 9.10000000e+01]
[ 1.40170249e+09 9.20000000e+01]
...,
[ 1.41149703e+09 ...
0
votes
2answers
24 views
How to Use Pandas with Multiple Column Numpy Array
Okay I'm stumped on this I've looked at the Pandas documentation but I can't figure out the right way to do it and I think I'm just making a mess. Basically I have data which are numpy arrays e.g.
...
0
votes
2answers
37 views
Python operator overloading : commutative or right only?
How is the operator overloading parsed?
I have a class object o and want to do -1 * o with the overloaded __mul__ operator.
Would that be parsed correctly when the left operand is a -1? ...
-1
votes
0answers
40 views
Pandas Multiindex: what am I doing wrong?
I have a program where I have a large pandas dataframe of pairwise interactions (in rows) that I do random walks through. The list of options for each successive step is narrowed down from the entire ...
0
votes
2answers
24 views
Get numpy 2D array from user input
I am trying to get a numpy two-dimensional array from user input except it does not work properly as input() returns a 'str' type while the numpy array() method expects a tuple:
import numpy as N
...
0
votes
0answers
19 views
call a function when there's an overflow using numpy.seterr in python
I'm using
np.seterr(under="raise", over ="raise", invalid="raise")
at the top of my code.
Then I also have these lines:
try:
weights[i,j] = math.exp(np.clip((beta[j,i] + oneminusbeta[j,i]), ...
0
votes
0answers
17 views
Python c-api function returning null
I am trying to create some python objects and use NumPy from C++. I have modified some examples I have found. My code below finds the NumPy module, the function numpy.array but returns null for ...
0
votes
1answer
17 views
RuntimeWarning: overflow encountered in exp in computing the logistic function
I'm getting this error when I try to compute the logistic function for a data mining method I'm implementing:
RuntimeWarning: overflow encountered in exp
My code:
def logistic_function(x):
# x ...
2
votes
2answers
69 views
+50
Simple way to find median
I have a data file and I perform few operations on the data. I can get solutions for all other operations just fine. I am not able to calculate the median only.
Input: Few lines from huge input.
...
