Error in calling `lm` in a `lapply` with `weights` argument

Question

I've encounter a weird behavior when calling lm within a lapply using the weights argument.

My code consist of a list of formula on which I run a linear model that I call in lapply. So far it was working:

dd <- data.frame(y = rnorm(100),
                 x1 = rnorm(100),
                 x2 = rnorm(100),
                 x3 = rnorm(100),
                 x4 = rnorm(100),
                 wg = runif(100,1,100))

ls.form <- list(
  formula(y~x1+x2),
  formula(y~x3+x4),
  formula(y~x1|x2|x3),
  formula(y~x1+x2+x3+x4)
)

res.no.wg <- lapply(ls.form, lm, data = dd)

However, when I add the weights argument, I get a weird error:

res.with.wg <- lapply(ls.form, lm, data = dd, weights = dd[,"wg"])
Error in eval(extras, data, env) : 
  ..2 used in an incorrect context, no ... to look in

It's like if the ... from lapply was conflicting with the ... of the lm call but only because of the weights argument.

Any idea was is the cause of this problem and how to fix it?

NOTE: using the call without the lapply works as expected:

lm(ls.form[[1]], data = dd, weights = dd[,"wg"] )

Call:
lm(formula = ls.form[[1]], data = dd, weights = dd[, "wg"])

Coefficients:
(Intercept)           x1           x2  
   -0.12020      0.06049     -0.01937  

EDIT The final call is a lapply within a function of the type:

f1 <- function(samp, dat, wgt){
res.with.wg2 <- lapply(ls.form, function(x) {lm(formula = x, data=dat[samp,], weights=dat[samp,wgt])})
}

f1(1:66, dat=dd, wgt = "wg")

Answer 1

I am not really sure why it is not working, but I do think I have a fix for you:

res.with.wg2 <- lapply(ls.form, 
                   function(x) {lm(formula = x, data=dd, weights=dd[,"wg"])})

Hope this helps!

Answer 2

There is a note in the help file for lapply:

For historical reasons, the calls created by lapply are unevaluated, and code has been written (e.g., bquote) that relies on this. This means that the recorded call is always of the form FUN(X[[i]], ...), with i replaced by the current (integer or double) index. This is not normally a problem, but it can be if FUN uses sys.call or match.call or if it is a primitive function that makes use of the call. This means that it is often safer to call primitive functions with a wrapper, so that e.g. lapply(ll, function(x) is.numeric(x)) is required to ensure that method dispatch for is.numeric occurs correctly.

lm uses match.call twice in its opening lines:

cl <- match.call()
mf <- match.call(expand.dots = FALSE)

The solution noted in the help file and by @Florian is to use an anonymous function wrapper.

Update

For this specific problem of changing the model formula, you can rewrite to avoid calling lm within the lapply by using update instead:

# create base model (the formula here doesn't really matter, but we can specify the weights safely here)
baselm <- lm(y+x1,data=dd,weights=dd[,"wg"])
# update with lapply
lapply(ls.form,update,object=baselm)
[[1]]

Call:
lm(formula = y ~ x1 + x2, data = dd, weights = dd[, "wg"])

Coefficients:
(Intercept)           x1           x2  
    0.07561      0.16111      0.15014  

...