using python virtual env in R

Question

I am using 'rPython' package for calling python within R but I am unable to make R refer to my python's virtual environment.

In R, I have tried using

system('. /home/username/Documents/myenv/env/bin/activate')

but after running the above my python library path does not change (which I check via python.exec(print sys.path)). When I run

python.exec('import nltk')

I am thrown the error:

Error in python.exec("import nltk") : No module named nltk

although it is there in my virtual env.

I am using R 3.0.2, Python 2.7.4 on Ubuntu 13.04.

Also, I know I can change the python library path from within R by using

python.exec("sys.path='\your\path'")

but I don't want this to be entered manually over and over again whenever a new python package is installed.

Thanks in advance!

Answer 1

Use the "activate" bash script before running R, so that the R process inherits the changed environment variables.

$ source myvirtualenv/bin/activate

$ R

Now rPython should be able to use the packages in your virtualenv.

Works for me. May behave strangely if the Python version you made the virtualenv with is different to the one rPython links into the R process.

Answer 2

Expanding on @PaulHarrison's answer, you can mimic what .../activate is doing directly in the environment (before starting python from R).

Here's one method for determining what vars are modified:

$ set > pyenv-pre
$ . /path/to/venv/activate
(venvname) $ set > pyenv-post
(venvname) $ diff -uw pyenv-pre pyenv-post

This gave me something like:

--- pyenv-pre 2018-12-02 15:16:43.093203865 -0800
+++ pyenv-post 2018-12-02 15:17:34.084999718 -0800
@@ -33,10 +33,10 @@
 OPTERR=1
 OPTIND=1
 OSTYPE=linux-gnu
-PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
+PATH=/path/to/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
 PIPESTATUS=([0]="0")
 PPID=325990
-PS1='\[\e]0;\u@\h: \w\a\]${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
+PS1='(venvname) \[\e]0;\u@\h: \w\a\]${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
 PS2='> '
 PS4='+ '
 PWD=/
@@ -50,10 +50,13 @@
 TERM=xterm
 UID=3000019
 USER='helloworld'
+VIRTUAL_ENV=/path/to/venv
 XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
 XDG_RUNTIME_DIR=/run/user/3000019
 XDG_SESSION_ID=27577
-_=set
+_=/path/to/venv/bin/activate
+_OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
+_OLD_VIRTUAL_PS1='\[\e]0;\u@\h: \w\a\]${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
 __git_printf_supports_v=yes
 __grub_script_check_program=grub-script-check
 _backup_glob='@(#*#|*@(~|.@(bak|orig|rej|swp|dpkg*|rpm@(orig|new|save))))'
@@ -2390,6 +2393,31 @@
         fi;
     fi
 }
+deactivate () ... rest of this function snipped for brevity

So it appears that the important envvars to update are:

• PATH: prepend the venv bin directory to the existing paths
• VIRTUAL_ENV: set to /path/to/venv

I believe the other changes (OLD_VIRTUAL_* and deactivate () ...) are optional and really only used to back-out the venv activation.

Looking at the .../activate script verifies these are most of the steps taken. Another step is unset PYTHONHOME if set, which may not be shown in the diff above if you didn't have it set previously.

To R-ize this:

Sys.setenv(
  PATH = paste("/path/to/venv/bin", Sys.getenv("PATH"), sep = .Platform$path.sep),
  VIRTUAL_ENV = "/path/to/venv"
)
Sys.unsetenv("PYTHONHOME") # works whether previously set or not