current community

your communities

more stack exchange communities

company blog
Stack Exchange Inbox Reputation and Badges
tour help
Code Review Stack Exchange is a question and answer site for peer programmer code reviews. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top
up vote 1 down vote favorite

I'm running the following code on quite large data frames. I've rewritten it for the iris dataset to make it reproducable.
I'm quite unexperienced with the apply functions and I find them a pain in the bum to apply them.

Is there any ways to drastically improve the performance of this process?

lmfit <- lm(iris$Petal.Width ~ iris$Sepal.Length + iris$Sepal.Width)
out_index <- 1
TableWithResiduals <- data.frame(matrix(ncol = ncol(iris) +1, nrow = nrow(iris)))
for (row in 1:length(resid(lmfit))){
  TableWithResiduals[out_index,] <- cbind(iris[row,],resid(lmfit)[row])
  out_index <- out_index +1  
}
colnames(TableWithResiduals) <- colnames(iris)
colnames(TableWithResiduals)[length(TableWithResiduals)] <- "Residual_value"
share|improve this question
up vote 2 down vote accepted

If you look at the doc for cbind, which you already use, you will see that it can take whole matrices, data.frames, and vectors as inputs. This means you can just do:

TableWithResiduals <- cbind(iris, Residual_value = resid(lmfit))

You could also have done:

TableWithResiduals <- iris
TableWithResiduals$Residual_value <- resid(lmfit)

If it were not for these solutions, there are a few things that could be improved in your code. First, you could have used row directly instead of the out_index variable you created. Second, the last two lines of your code could have been merged into one: names(TableWithResiduals) <- c(names(iris), "Residual_value"). Also, if you look at the doc for lm, you could have saved yourself some typing by doing lm(Petal.Width ~ Sepal.Length + Sepal.Width, iris)

share|improve this answer
    
Thanks alot! Using the TableWithResiduals$Residual_value <- resid(lmfit) the improvement in speed is almost a thousand times according to microbenchmark, it's almost instantaneously now :) – Bas Nov 2 '15 at 12:43

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.