Why doesn't 'with' pass variable scope through nested functions?

Question

In R, if I create an environment and then use with to evaluate a function in that enviroment, the function has normally access to the variables. However, if I nest functions, for some reason, they go out of scope. Can you explain to me why this is the case?

Example:

Create a new environment with a variable called x

E = new.env();
E$x = c(1,2,3)

Using with I can print this variable:

with(E, print(x));
#[1] 1 2 3

But now if I nest this function, it no longer works:

printMe = function() { print(x); }
with(E, printMe())
#Error in print(x) : object 'x' not found

I know I can make it work again like so:

printMe = function(x) { print(x); }
with(E, printMe(x))
#[1] 1 2 3

But I don't understand -- if with creates an environment, why can't the nested function see the x? It works if you attach it:

attach(E)
printMe()
#[1] 1 2 3

I think I'm just missing something about scoping, but what's the recommended way of doing this? Or, to put my question another way: why can't nested functions in with access free variables?

Answer 1

Basically, when you use with, you're doing the same as

printMe = function() { print(x); }
local({
    x=1:3
    printMe()
})
# Error in print(x) : object 'x' not found

which also does not work. This has to do with how free variables are resolved in a function. When you call printMe, it will look to resolve variables in the enclosure itself, then it looks in the parent frame where the function was defined (it does not look where the function was called). Here, printMe is defined in the global environment. However, x is not defined in the global environment. If you do

printMe = function() { print(x); }
x=1:3
printMe()
# [1] 1 2 3

Then both x and printMe are defined in the global environment so it works. You could also change the environment of the printMe function

 environment(printMe) <- E
 printMe()
 # [1] 1 2 3

or define the function in the same environment as the with

with(E,{printMe <- function() {print(x)}; printMe()})
# [1] 1 2 3

The main point is, it doesn't matter what environment you call functions from, it matters what environments they were defined in.

You might want to check out the Advanced R material on functional programming which goes over these properties.

Answer 2

R is lexically scoped not dynamically scoped. The environment of printMe is the Global Environment, because it was defined in the Global Environment:

environment(printMe)
<environment: R_GlobalEnv>

So when you call:

with(E, printMe())

The function printMe tries to find x locally. It doesn´t. Then it tries to find x in its environment, which is the Global Enviroment and not the local environment of with. It does not find it again, then it throws an error.

To illustrate the point, see that if you define printMe inside with, the environment of printMe will be the local environment of with and it will find x:

with(E,{
  printMe <- function() {
    print(x)
  } 
  print(environment(printMe))
  printMe()
  })
<environment: 0x29785678>
[1] 1 2 3

Or, you could change the environment of printMe inside with:

with(E, {
  environment(printMe) <- environment()
  printMe()
})
[1] 1 2 3

Regarding your second example, when you attach the environment E, you are making the objects of E (i.e, x) available to the Global Environment. So when you call printMe in this situation, it will look for x in the Global Environment and, since E is attached, it will find it. That´s why it works.

I used to have the same doubts you are having, so this question may help you further: Environments in R, mapply and get .

This may also help: Understanding lexical scoping in R