how to get output from a pipe connection before closing it in R?

Question

In R, we can open a pipe connection using pipe() and write to it. I observed the following situation that I do not quite understand. Let's use a python pipe for example:

z = pipe('python', open='w+')

cat('x=1\n', file=z)
cat('print(x)\n', file=z)
cat('print(x+2)\n', file=z)
cat('print(x+2\n', file=z)
cat(')\n', file=z)

close(z)

What I was expecting was the output from print() would be immediately shown in the R console, but the fact is the output comes only after I close the pipe connection:

> z = pipe('python', open='w+')
> 
> cat('x=1\n', file=z)
> cat('print(x)\n', file=z)
> cat('print(x+2)\n', file=z)
> cat('print(x+2\n', file=z)
> cat(')\n', file=z)
> 
> close(z)
1
3
3

So my question is, how can I get the output before I close the connection? Note that it does not seem to be possible to capture the output using capture.output(), either:

> z = pipe('python', open='w+')
> 
> cat('x=1\n', file=z)
> cat('print(x)\n', file=z)
> cat('print(x+2)\n', file=z)
> cat('print(x+2\n', file=z)
> cat(')\n', file=z)
> 
> x = capture.output(close(z))
1
3
3
> x
character(0)

The background of this question is the knitr engines. For the interpreted languages like Python, I wish I can open a persistent "terminal" so that I can keep on writing code into it and get output from it. I'm not sure if pipe() is the correct way to go, though.

Answer 1

Python notices that the input is not interactive and waits until the connection is closed to parse and execute the code. You can use the -i option to force it to stay in interactive mode. (but the output is a bit mangled).

z = pipe('python -i', open='w')
cat('x=1\n', file=z)
cat('print(x)\n', file=z)
cat('print(x+2)\n', file=z)
cat('print(x+2\n', file=z)
cat(')\n', file=z)
Sys.sleep(2)
# Python 2.7.4 (default, Apr 19 2013, 18:28:01) 
# [GCC 4.7.3] on linux2
# Type "help", "copyright", "credits" or "license" for more information.
# >>> >>> 1
# >>> 3
# >>> ... 3
# >>> 
close(z)

Your actual problem is more complicated: you need to both read and write to the same connection. I do not know how to do that in a portable way, but you can use a pipe and a named pipe (a "fifo") on platforms that support them.

stopifnot( capabilities("fifo") )
system('mkfifo /tmp/Rpython.fifo')
output <- fifo('/tmp/Rpython.fifo', 'r')
input  <- pipe('python -i > /tmp/Rpython.fifo', 'w')
python_code <- "
x=1
print(x)
print(x+2)
print(x+2
)
"
cat( python_code, file = input )
flush( input )
Sys.sleep(2) # Wait for the results
result <- readLines(output)
result
# [1] "1" "3" "3"