R: disentangling scopes

Question

My question is about avoiding namespace pollution when writing modules in R.

Right now, in my R project, I have functions1.R with doFoo() and doBar(), functions2.R with other functions, and main.R with the main program in it, which first does source('functions1.R'); source('functions2.R'), and then calls the other functions.

I've been starting the program from the R GUI in Mac OS X, with source('main.R'). This is fine the first time, but after that, the variables that were defined the first time through the program are defined for the second time functions*.R are sourced, and so the functions get a whole bunch of extra variables defined.

I don't want that! I want an "undefined variable" error when my function uses a variable it shouldn't! Twice this has given me very late nights of debugging!

So how do other people deal with this sort of problem? Is there something like source(), but that makes an independent namespace that doesn't fall through to the main one? Making a package seems like one solution, but it seems like a big pain in the butt compared to e.g. Python, where a source file is automatically a separate namespace.

Any tips? Thank you!

Answer 1

I would explore two possible solutions to this.

a) Think more in a more functional manner. Don't create any variables outside of a function. so, for example, main.R should contain one function main(), which sources in the other files, and does the work. when main returns, none of the clutter will remain.

b) Clean things up manually:

#main.R
prior_variables <- ls()
source('functions1.R')
source('functions2.R')

#stuff happens

rm(list = setdiff(ls(),prior_variables))`

Answer 2

The main function you want to use is sys.source(), which will load your functions/variables in a namespace ("environment" in R) other than the global one. One other thing you can do in R that is fantastic is to attach namespaces to your search() path so that you need not reference the namespace directly. That is, if "namespace1" is on your search path, a function within it, say "fun1", need not be called as namespace1.fun1() as in Python, but as fun1(). [Method resolution order:] If there are many functions with the same name, the one in the environment that appears first in the search() list will be called. To call a function in a particular namespace explicitly, one of many possible syntaxes - albeit a bit ugly - is get("fun1","namespace1")(...) where ... are the arguments to fun1(). This should also work with variables, using the syntax get("var1","namespace1"). I do this all the time (I usually load just functions, but the distinction between functions and variables in R is small) so I've written a few convenience functions that loads from my ~/.Rprofile.

  name.to.env <- function(env.name)
    ## returns named environment on search() path
    pos.to.env(grep(env.name,search()))

  attach.env <- function(env.name)
    ## creates and attaches environment to search path if it doesn't already exist
    if( all(regexpr(env.name,search())<0) ) attach(NULL,name=env.name,pos=2)

  populate.env <- function(env.name,path,...) {
    ## populates environment with functions in file or directory
    ## creates and attaches named environment to search() path 
    ##        if it doesn't already exist
    attach.env(env.name)
    if( file.info(path[1])$isdir )
      lapply(list.files(path,full.names=TRUE,...),
             sys.source,name.to.env(env.name)) else
    lapply(path,sys.source,name.to.env(env.name))
    invisible()
  }

Example usage:

populate.env("fun1","pathtofile/functions1.R")
populate.env("fun2","pathtofile/functions2.R")

and so on, which will create two separate namespaces: "fun1" and "fun2", which are attached to the search() path ("fun2" will be higher on the search() list in this case). This is akin to doing something like

attach(NULL,name="fun1")
sys.source("pathtofile/functions1.R",pos.to.env(2))

manually for each file ("2" is the default position on the search() path). The way that populate.env() is written, if a directory, say "functions/", contains many R files without conflicting function names, you can call it as

populate.env("myfunctions","functions/")

to load all functions (and variables) into a single namespace. With name.to.env(), you can also do something like

with(name.to.env("fun1"), doStuff(var1))

or

evalq(doStuff(var1), name.to.env("fun1"))

Of course, if your project grows big and you have lots and lots of functions (and variables), writing a package is the way to go.