Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Capture stdout from a script?

Tags:

python

stdout

sys

suppose there is a script doing something like this:

# module writer.py
import sys

def write():
    sys.stdout.write("foobar")

Now suppose I want to capture the output of the write function and store it in a variable for further processing. The naive solution was:

# module mymodule.py
from writer import write

out = write()
print out.upper()

But this doesn't work. I come up with another solution and it works, but please, let me know if there is a better way to solve the problem. Thanks

import sys
from cStringIO import StringIO

# setup the environment
backup = sys.stdout

# ####
sys.stdout = StringIO()     # capture output
write()
out = sys.stdout.getvalue() # release output
# ####

sys.stdout.close()  # close the stream 
sys.stdout = backup # restore original stdout

print out.upper()   # post processing
like image 315
Paolo Avatar asked Feb 27 '11 22:02

Paolo


People also ask

How do you capture stdout?

To capture a tool's standard output stream, add the stdout field with the name of the file where the output stream should go. Then add type: stdout on the corresponding output parameter.

How do I capture all stdout in python?

To capture stdout output from a Python function call, we can use the redirect_stdout function. to call redirect_stdout with the f StringIO object. Then we call do_something which prints stuff to stdout. And then we get the value printed to stdout with f.

How do I redirect to stdout?

Understanding the concept of redirections and file descriptors is very important when working on the command line. To redirect stderr and stdout , use the 2>&1 or &> constructs.


4 Answers

For future visitors: Python 3.4 contextlib provides for this directly (see Python contextlib help) via the redirect_stdout context manager:

from contextlib import redirect_stdout
import io

f = io.StringIO()
with redirect_stdout(f):
    help(pow)
s = f.getvalue()
like image 116
nodesr Avatar answered Oct 21 '22 05:10

nodesr


Setting stdout is a reasonable way to do it. Another is to run it as another process:

import subprocess

proc = subprocess.Popen(["python", "-c", "import writer; writer.write()"], stdout=subprocess.PIPE)
out = proc.communicate()[0]
print out.upper()
like image 37
Matthew Flaschen Avatar answered Oct 21 '22 03:10

Matthew Flaschen


Here is a context manager version of your code. It yields a list of two values; the first is stdout, the second is stderr.

import contextlib
@contextlib.contextmanager
def capture():
    import sys
    from cStringIO import StringIO
    oldout,olderr = sys.stdout, sys.stderr
    try:
        out=[StringIO(), StringIO()]
        sys.stdout,sys.stderr = out
        yield out
    finally:
        sys.stdout,sys.stderr = oldout, olderr
        out[0] = out[0].getvalue()
        out[1] = out[1].getvalue()

with capture() as out:
    print 'hi'
like image 52
Jason Grout Avatar answered Oct 21 '22 05:10

Jason Grout


Starting with Python 3 you can also use sys.stdout.buffer.write() to write (already) encoded byte strings to stdout (see stdout in Python 3). When you do that, the simple StringIO approach doesn't work because neither sys.stdout.encoding nor sys.stdout.buffer would be available.

Starting with Python 2.6 you can use the TextIOBase API, which includes the missing attributes:

import sys
from io import TextIOWrapper, BytesIO

# setup the environment
old_stdout = sys.stdout
sys.stdout = TextIOWrapper(BytesIO(), sys.stdout.encoding)

# do some writing (indirectly)
write("blub")

# get output
sys.stdout.seek(0)      # jump to the start
out = sys.stdout.read() # read output

# restore stdout
sys.stdout.close()
sys.stdout = old_stdout

# do stuff with the output
print(out.upper())

This solution works for Python 2 >= 2.6 and Python 3. Please note that our sys.stdout.write() only accepts unicode strings and sys.stdout.buffer.write() only accepts byte strings. This might not be the case for old code, but is often the case for code that is built to run on Python 2 and 3 without changes.

If you need to support code that sends byte strings to stdout directly without using stdout.buffer, you can use this variation:

class StdoutBuffer(TextIOWrapper):
    def write(self, string):
        try:
            return super(StdoutBuffer, self).write(string)
        except TypeError:
            # redirect encoded byte strings directly to buffer
            return super(StdoutBuffer, self).buffer.write(string)

You don't have to set the encoding of the buffer the sys.stdout.encoding, but this helps when using this method for testing/comparing script output.

like image 15
JonnyJD Avatar answered Oct 21 '22 05:10

JonnyJD