Ruby pipes: How do I tie the output of two subprocesses together?

Question

Is there an automated way to do shell piping in Ruby? I'm trying to convert the following shell code to Ruby:

a | b | c... > ...

but the only solution I have found so far is to do the buffer management myself (simplified, untested, hope it gets my meaning across):

a = IO.popen('a')
b = IO.popen('b', 'w+')
Thread.new(a, b) { |in, out|
    out.write(in.readpartial(4096)) until in.eof?
    out.close_write
}
# deal with b.read...

I guess what I'm looking for is a way to tell popen to use an existing stream, instead of creating a new one? Or alternatively, an IO#merge method to connect a's output to b's input? My current approach becomes rather unwieldly when the number of filters grows.

I know about Kernel#system('a | b') obviously, but I need to mix Ruby filters with external program filters in a generic way.

Answer 1

Old question, but since its one of the first result on Google, here is the answer : http://devver.wordpress.com/2009/10/12/ruby-subprocesses-part_3/ (method 8)

In short :

sh = Shell.new
sh.system("a") | sh.system("b") | sh.system("c")

And you can do more complicated things like

sh.echo(my_string) | sh.system("wc") > "file_path"
xml = (sh.echo(html) | sh.system("tidy", "-q")).to_s

Answer 2

Using plain ruby, spawn has redirection options that you can use to connect processes with pipes.

1) Create a pipe

r,w = IO.pipe

2) Use it to connect two spawned processes

spawn(*%w[echo hello world], out: w)
spawn(*%w[tr a-z A-Z], in: r)
# => HELLO WORLD

Of course, you can encapsulate this in something like sh.system from the mentioned Shell library, and create a |() method for doing the interconnecting.

The open3 module of the standard library has some really nice tools for this kind of stuff, including the creation of complete pipelines.