Understanding lexical scoping in R

Question

I am reading this paper (ungated copy) on evaluating the design of the R programming language, and am not able to understand a particular example on lexical scoping (or the absence thereof).

On page 4, the authors provide the following example of the use of the with function:

with(formaldehyde, carb*optden)

They go on to say:

The astute reader will have noticed that the above example clashes with our claim that R is lexically scoped. As is often the case, R is lexically scoped up to the point it is not. R is above all a dynamic language with full reflective access to the running program’s data and representation. In the above example, the implementation of with sidesteps lexical scoping by reflectively manipulating the environment. This is done by a combination of lazy evaluation, dynamic name lookup, and the ability turn code into text and back:

with.default <- function(env, expr, ...)
  eval(substitute(expr),env, enclose=parent.frame())

The function uses substitute to retrieve the unevaluated parse tree of its second argument, then evaluates it with eval in the environment constituted by composing the first argument with the lexically enclosing environment. The ‘...’ is used to discard any additional arguments.

How is the use of the with function in this case a violation of the principles of lexical scoping?

Answer 1

Normally when discussed in the context of R lexical scoping means that free variables in a function (i.e. variables that are used in a function but not defined in the function) are looked up in the parent environment of the function, as opposed to the environment of the caller (also referred to as the parent frame) but there are no free variables in with.default so the example does not illustrate a violation of lexical scoping in that sense.

For example, this illustrates lexical scoping:

x <- 1
f <- function() x
g <- function() { x <- 0; f() }
g() # 1

The answer is 1 because 1 is defined in the environment that f is defined in. Had R used dynamic scoping rather than lexical scoping the answer would have been 0 (using the environment of the caller). We can illlustrate how R can emulate dynamic scoping like this:

f <- function() eval.parent(quote(x))
g() # 0

ADDED:

In a comment below @hadley suggested that the authors may have been referring to the fact that the second actual argument to with.default is not evaluated lexically and this interpretation seems likely. Instead of being evaluated relative to the surrounding lexical environment the second actual argument of with.default is read into the with.default function as an expression using substitute and then evaluated relative to the first argument using eval. There is some question of what the definition of lexical scoping ought to be as it is rarely defined even when extensively discussed but typical discussions in relation to R refer to it as the treatment of free variables. See for example Gentleman & Ihaka.