Ruby zip a stream

Question

I am trying to write a ruby fcgi script which compresses files in a directory on the fly and sends the output blockwise as an http response. It is very important that this compression is done as a stream operation, otherwise the client will get a timeout for huge directories.

I have the following code:

d="/tmp/delivery/"

# send zip header
header(MimeTypes::ZIP)

# pseudocode from here on
IO.open(d) { |fh|
    block=fh.readblock(1024)
    #send zipped block as http response
    print zip_it(block)
}

How do I achieve what I've written as pseudo-ruby in the above listing?

Answer 1

Tokland's idea of using the external zip command works pretty well. Here's a quick snippet that should work with Ruby 1.9 on Linux or similar environments. It uses an array parameter to popen() to avoid any shell quoting issues and sysread/syswrite to avoid buffering. You could display a status message in the empty rescue block if you like -- or you could use read and write, though I haven't tested those.

#! usr/bin/env ruby
d = '/tmp/delivery'
output = $stdout
IO.popen(['/usr/bin/zip', '-', d]) do |zip_output|
  begin
    while buf = zip_output.sysread(1024)
      output.syswrite(buf)
    end
    rescue EOFError
  end
end

Answer 2

AFAYK Zip format is not streamable, at end of compression it writes something in the file header.

gz or tar.gz is better option.