Tell me more ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.
up vote 1 down vote favorite

I'm trying to call the R function loess via Rpy2 in Python on this datafile: http://filebin.ca/azuz9Piv0z8/test.data

It works when I use a subset of the data (the first 1000 points) but when I try to use the entire file, I get an error. My code:

import pandas
from rpy2.robjects import r
import rpy2.robjects as robjects
data = pandas.read_table(os.path.expanduser("~/test2.data"), sep="\t").values
small_data = data[0:1000, :]
print "small data loess:"
a, b = robjects.FloatVector(list(small_data[:, 0])), \
       robjects.FloatVector(list(small_data[:, 1]))
df = robjects.DataFrame({"a": a, "b": b})
loess_fit = r.loess("b ~ a", data=df)
print loess_fit

print "large data loess:"
a, b = robjects.FloatVector(list(data[:, 0])), \
       robjects.FloatVector(list(data[:, 1]))
df = robjects.DataFrame({"a": a, "b": b})
loess_fit = r.loess("b ~ a", data=df)
print loess_fit

Fitting on small_data works but not data. I get the error:

Error in simpleLoess(y, x, w, span, degree, parametric, drop.square, normalize,  : 
  NA/NaN/Inf in foreign function call (arg 1)
    loess_fit = r.loess("b ~ a", data=df)
  File "/usr/local/lib/python2.7/dist-packages/rpy2-2.3.3-py2.7-linux-x86_64.egg/rpy2/robjects/functions.py", line 86, in __call__
    return super(SignatureTranslatedFunction, self).__call__(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rpy2-2.3.3-py2.7-linux-x86_64.egg/rpy2/robjects/functions.py", line 35, in __call__
    res = super(Function, self).__call__(*new_args, **new_kwargs)
rpy2.rinterface.RRuntimeError: Error in simpleLoess(y, x, w, span, degree, parametric, drop.square, normalize,  : 
  NA/NaN/Inf in foreign function call (arg 1)

How can this be fixed? I'm not sure if it's a problem with the R function loess or with the Rpy2 interface to it? thanks.

share|improve this question

1 Answer

up vote 2 down vote accepted

The problem are -Inf values in your data:

DF <- read.table('http://filebin.ca/azuz9Piv0z8/test.data')
DF[!is.finite(DF[,1]) | !is.finite(DF[,2]),]
#        V1   V2
# 5952 -Inf -Inf
share|improve this answer
thank you! just noticed it - weird because I did dropna() before in pandas. is there a way to make loess be ok with NA values or should I just pre-remove them – user248237dfsf Mar 21 at 21:00
Inf is not the same as NA. loess deals just fine with the latter (see its na.action argument). You could use DF[!is.finite(DF[,1]) | !is.finite(DF[,2]),] <- NA. – Roland Mar 21 at 21:03
1  
inf and -inf are not considered as missing values in pandas, see: pandas.pydata.org/pandas-docs/stable/missing_data.html, so dropna() will not have any effect. – herrfz Mar 21 at 21:04

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.