Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do you pipe output from a Ruby script to 'head' without getting a broken pipe error

Tags:

unix

ruby

pipe

I have a simple Ruby script that looks like this

require 'csv'
while line = STDIN.gets
  array = CSV.parse_line(line)
  puts array[2]
end

But when I try using this script in a Unix pipeline like this, I get 10 lines of output, followed by an error:

ruby lib/myscript.rb < data.csv  | head

12080450
12080451
12080517
12081046
12081048
12081050
12081051
12081052
12081054
lib/myscript.rb:4:in `write': Broken pipe - <STDOUT> (Errno::EPIPE)

Is there a way to write the Ruby script in a way that prevents the broken pipe exception from being raised?

like image 328
dan Avatar asked May 16 '10 21:05

dan


People also ask

What causes a broken pipe error?

This error generally means means that the data stopped flowing to us and we were unable to start the transfer again. Often times this is caused by a wireless internet connection with fluctuating signal strength, a firewall or other security software.

What causes a broken pipe error in Linux?

Broken pipe occurs when a process prematurely exits from either end and the other process has not yet closed the pipe. Example use case: A user has just recently reinstalled RVM (Ruby Version Manager) after he performed a fresh install of Ubuntu.

What is Brokenpipeerror?

A broken Pipe Error is generally an Input/Output Error, which is occurred at the Linux System level. The error has occurred during the reading and writing of the files and it mainly occurs during the operations of the files.


2 Answers

head is closing the standard output stream after it has read all the data it needs. You should handle the exception and stop writing to standard output. The following code will abort the loop once standard output has been closed:

while line = STDIN.gets
  array = CSV.parse_line(line)
  begin
    puts array[2]
  rescue Errno::EPIPE
    break
  end
end
like image 66
Phil Ross Avatar answered Nov 15 '22 23:11

Phil Ross


The trick I use is to replace head with sed -n 1,10p.

This keeps the pipe open so ruby (or any other program that tests for broken pipes and complains) doesn't get the broken pipe and therefore doesn't complain. Choose the value you want for the number of lines.

Clearly, this is not attempting to modify your Ruby script. There almost certainly is a way to do it in the Ruby code. However, the 'sed instead of head' technique works even where you don't have the option of modifying the program that generates the message.

like image 27
Jonathan Leffler Avatar answered Nov 16 '22 00:11

Jonathan Leffler