mix variables/results between chunks in knitr defined in different languages?

Question

Language engines is a great knitr feature . We can use any languages in knitr, including but not limited to R. But what if I want to use results/variable defined in a one chunck , in another chunck either with the same language or in another language (more sexy option)?

Better to explain my idea with some code. Here I define 4 chunks, 2 in python and 2 in R.

First I define a variable x in python:

## I define engien path explicitly here (TODO: set it globally)
```{r,engine='python',engine.path='C:/Anaconda/python.exe' }
x = 1
print x
```
## 1

Now Try to use x in new python chunck:

```{r,engine='python',engine.path='C:/Anaconda/python.exe' }
x = x +1
print x

```

No error but amazing result, looks like x is NULL here. Now if I Try to use x in new R chunck:

```{r fig.width=7, fig.height=6}
x +1
y = 2
```
## Error: object 'x' not found 

I get an error. Now if I try to use y in new R chunk and it works fine. the r engine can use variable defined in the previous R chunck. Note this doesn't work with python.

```{r fig.width=7, fig.height=6}
y+3
```
## [1] 5

Why there is difference in behavior between R and python here? Is it structural due to R scoping rules or just a future not already implemented in knitr? or maybe a bug?

Answer 1

This is the documented behaviour.

see http://yihui.name/knitr/demo/engines/

Except engine='R' (default), all chunks are executed in separate sessions, so the variables cannot be directly shared. If we want to make use of objects created in previous chunks, we usually have to write them to files (as side effects). For the bash engine, we can use Sys.setenv() to export variables from R to bash (example)

Answer 2

You could put something in a startup (PYTHONSTARTUP) file for python. An elaborate startup file can export all known variables in a file when exiting leveraging atexit.

Here is a recipe that prints all new globals:

>>> import atexit
>>> def list_globals(known_globals=globals().keys()):
...     new_globals = set(globals().keys())
...     new_globals -= set(known_globals)
...     for key in new_globals:
...        print '%s=%s' % (key, globals()[key])
... 
>>> atexit.register(list_globals)
<function list_globals at 0x107140e60>
>>> del list_globals
>>> del atexit
>>> 

Here is an example session after the above code has been run.

>>> 
>>> def foo():
...    a  = 1
... 
>>> b = 2
>>> ^D
b=2
foo=<function foo at 0x107140d70>

You could instead use a file / socket / message queue / etc.