Subtly different behaviour between with() and attach() in R?

Question

People often use the attach() and detach() functions to set up "search paths" for variable names in R, but because this alters global state that's hard to keep track of, people recommend using with() instead, which sets up a temporary alteration to the search path for the duration of a single expression.

However I just noticed that unlike attach(), with() apparently doesn't "resolve through" functions. For example, let's first set up a dummy function that will access a variable named x:

f <- function { print(x) }

Now,

with(list(x=42), f())

fails even though

with(list(x=42), print(x))

and

attach(list(x=42))
f()

both succeed! :(

Can anyone tell me why? I would like with() to behave exactly like attach() does here, to enable me to effectively pass a big parameter list to a function by setting up an environment containing the parameter values with with(). I see this approach as having several benefits over the alternatives (the two I've considered are (a) laboriously passing all the parameters into the function, and (b) explicitly passing in a list/frame of parameters as a function argument and having the function itself call with()), but it doesn't work. I find this discrepancy pretty troubling to be honest! Any explanation/help would be appreciated.

I'm using R 2.11.1.

Answer 1

The difference between what with(list(x = 42), f()) is doing and what you expect is the difference between lexical scoping (which is what R uses) and dynamic scoping (which seems to be what you are expecting).

Lexical scoping means that free variables (like the variable x in f) are looked up in the environment where f is defined -- not the environment f is called from.

f is defined in the global environment so that is where x is looked up.

It doesn't matter that with has been invoked to create a new environment from which f is called since the environment from which its called is not involved in looking up free variables.

To get this to work the way you want create a copy of f and reset its environment since that is what R uses to search for free variables:

with(list(x = 42), { environment(f) <- environment(); f() })

Admittedly this is a bit onerous but you could simplify it somewhat by using the proto package since proto resets the environment of each function that is explicitly inserted into a proto object:

library(proto)
with(proto(x = 42, f = f), f())

ADDED:

Note that if your aim is to do object oriented programming (as per your comment to another response) then you might want to look into proto further at the proto home page. For example, we could define the proto object p and redefine f so that its a method of p (in which case it must accept the object in argument 1) like this:

library(proto)
p <- proto(x = 42, f = function(.) print(.$x))
p$f()

ADDED 2:

With the attached case, running f() first looks in the global environment since that is where f is defined. Since x is not found in the global environment it looks into the parent of the global environment and in this case it finds it there. We can discover the parent of the global environment using parent.env and here we see that the attached environment has become the parent of the global environment.

> attach(list(x = 42))
> parent.env(.GlobalEnv)
<environment: 0x048dcdb4>
attr(,"name")
[1] "list(x = 42)"

We can view the global environment and all its ancestors in order like this:

> search()
 [1] ".GlobalEnv"        "list(x = 42)"      "package:stats"    
 [4] "package:graphics"  "package:grDevices" "package:utils"    
 [7] "package:datasets"  "package:methods"   "Autoloads"        
[10] "package:base"   

Thus "list(x = 42)" is the parent of the global environment, stats is the parent of "list(x = 42)" and so on.

Answer 2

I think this is due to you not defining any arguments to f, and thence how the x required for print(x) is looked for.

In normal use f() will look for x in the global environment if it isn't supplied (and it isn't nor can it be as f takes no arguments).

Inside with(), what happens is that any arguments needed for f will be taken from the data argument. But because your f doesn't take any arguments, the x in the list is never used. Instead, R reverts to usual behaviour and looks up x in the environment of f, the global environment, and of course it isn't there. The difference with attach() is that it explicitly adds an object containing an x to the search path that is in the global environment.

If you write your function properly, following the mantra that you pass in any and all arguments you use within the function, then everything works as one would expect:

> F <- function(x) print(x)
> with(list(x = 42), F(x))
[1] 42
> ls()
[1] "f" "F"

If you already have the list or arguments need for the call, perhaps consider do.call() to set up the call for you, instead of using with. For example:

> do.call(F, list(x = 42))
[1] 42

You still need your function to be properly defined with arguments, as your f doesn't work:

> do.call(f, list(x = 42))
Error in function ()  : unused argument(s) (x = 42)

Answer 3

The description from ?with states:

Evaluate an R expression in an environment constructed from data, possibly modifying the original data.

This means that the function is run in an evironment where the data exists, but the environment is not on the search path. Thus any functions run in this environment will not find data that isn't passed to them (since they are run in their own environment), unless instructed to look at the parent.frame. Consider the following:

> f <- function() print(x)
> f()
Error in print(x) : object 'x' not found
> with(list(x=42),f())
Error in print(x) : object 'x' not found
> x <- 13
> f()
[1] 13
> with(list(x=42),f())
[1] 13
> f2 <- function(x) print(get("x",parent.frame()))
> f2()
[1] 13
> with(list(x=42),f2())
[1] 42

To clarify how with is normally used, the data is passed to the function in the environment with creates, and not normally referred to as a "global" variable by the function being called.