Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Create and pipe a file-like object as input for a command

I'm looking for a better way to do this, if possible:

import subprocess

f = open('temp.file', 'w+')
f.write('hello world')
f.close()

out = subprocess.check_output(['cat', 'temp.file'])

print out

subprocess.check_output(['rm', 'temp.file'])

In this example I'm creating a file and passing it as input to cat (in reality it's not cat I'm running but some other program that parses an input pcap file).

What I'm wondering is, is there a way in Python I can create a 'file-like object' with some content, and pipe this file-like object as input to a command-line program. If it is possible, I reckon it would be more efficient than writing a file to the disk and then deleting that file.

like image 609
Juicy Avatar asked Jun 17 '15 12:06

Juicy


People also ask

How do you create a file like an object in Python?

To create a file object in Python use the built-in functions, such as open() and os. popen() . IOError exception is raised when a file object is misused, or file operation fails for an I/O-related reason. For example, when you try to write to a file when a file is opened in read-only mode.

What is pipe file in Linux?

A pipe is a form of redirection (transfer of standard output to some other destination) that is used in Linux and other Unix-like operating systems to send the output of one command/program/process to another command/program/process for further processing.

How do you use the pipe command in Python?

pipe() method in Python is used to create a pipe. A pipe is a method to pass information from one process to another process. It offers only one-way communication and the passed information is held by the system until it is read by the receiving process.

What is the pipe command in Linux?

The Pipe is a command in Linux that lets you use two or more commands such that output of one command serves as input to the next. In short, the output of each process directly as input to the next one like a pipeline. The symbol '|' denotes a pipe.


2 Answers

check_output takes a stdin input argument to specify a file-like object to connect to the process's standard input.

with open('temp.file') as input:
    out = subprocess.check_output(['cat'], stdin=input)

Also, there's no need to shell out to run rm; you can remove the file directly from Python:

os.remove('temp.file')
like image 129
chepner Avatar answered Sep 23 '22 14:09

chepner


You can write to a TemporaryFile

import subprocess
from tempfile import TemporaryFile
f = TemporaryFile("w")
f.write("foo")
f.seek(0)
out = subprocess.check_output(['cat'],stdin=f)

print(out)
b'foo'

If you just want to write to a file like object and get the content:

from io import StringIO
f = StringIO()
f.write("foo")

print(f.getvalue())
like image 25
Padraic Cunningham Avatar answered Sep 23 '22 14:09

Padraic Cunningham