How do I block on reading a named pipe in Ruby?

Question

I'm trying to set up a Ruby script that reads from a named pipe in a loop, blocking until input is available in the pipe.

I have a process that periodically puts debugging events into a named pipe:

# Open the logging pipe
log = File.open("log_pipe", "w+") #'log_pipe' created in shell using mkfifo
...
# An interesting event happens
log.puts "Interesting event #4291 occurred"
log.flush
...

I then want a separate process that will read from this pipe and print events to the console as they happen. I've tried using code like this:

input = File.open("log_pipe", "r+") 
while true
  puts input.gets  #I expect this to block and wait for input
end
# Kill loop with ctrl+c when done

I want the input.gets to block, waiting patiently until new input arrives in the fifo; but instead it immediately reads nil and loops again, scrolling off the top of the console window.

Two things I've tried:

1. I've opened the input fifo with both "r" and "r+"--I have the same problem either way;

2. I've tried to determine if my writing process is sending EOF (which I've heard will cause the read fifo to close)--AFAIK it isn't.

SOME CONTEXT:

If it helps, here's a 'big picture' view of what I'm trying to do:

I'm working on a game that runs in RGSS, a Ruby based game engine. Since it doesn't have good integrated debugging, I want to set up a real-time log as the game runs--as events happen in the game, I want messages to show up in a console window on the side. I can send events in the Ruby game code to a named pipe using code similar to the writer code above; I'm now trying to set up a separate process that will wait for events to show up in the pipe and show them on the console as they arrive. I'm not even sure I need Ruby to do this, but it was the first solution I could think of.

Note that I'm using mkfifo from cygwin, which I happened to have installed anyway; I wonder if that might be the source of my trouble.

If it helps anyone, here's exactly what I see in irb with my 'reader' process:

irb(main):001:0> input = File.open("mypipe", "r")
=> #<File:mypipe>
irb(main):002:0> x = input.gets
=> nil
irb(main):003:0> x = input.gets
=> nil

I don't expect the input.gets at 002 and 003 to return immediately--I expect them to block.

Answer 1

I found a solution that avoids using Cygwin's unreliable named pipe implementation entirely. Windows has its own named pipe facility, and there is even a Ruby Gem called win32-pipe that uses it.

Unfortunately, there appears to be no way to use Ruby Gems in an RGSS script; but by dissecting the win32-pipe gem, I was able to incorporate the same idea into an RGSS game. This code is the bare minimum needed to log game events in real time to a back channel, but it can be very useful for deep debugging.

I added a new script page right before 'Main' and added this:

module PipeLogger
  # -- Change THIS to change the name of the pipe!
  PIPE_NAME = "RGSSPipe"

  # Constant Defines
  PIPE_DEFAULT_MODE        = 0            # Pipe operation mode
  PIPE_ACCESS_DUPLEX       = 0x00000003   # Pipe open mode
  PIPE_UNLIMITED_INSTANCES = 255          # Number of concurrent instances
  PIPE_BUFFER_SIZE         = 1024         # Size of I/O buffer (1K)
  PIPE_TIMEOUT             = 5000         # Wait time for buffer (5 secs)
  INVALID_HANDLE_VALUE     = 0xFFFFFFFF   # Retval for bad pipe handle

  #-----------------------------------------------------------------------
  # make_APIs
  #-----------------------------------------------------------------------
  def self.make_APIs
    $CreateNamedPipe     = Win32API.new('kernel32', 'CreateNamedPipe', 'PLLLLLLL', 'L')
    $FlushFileBuffers    = Win32API.new('kernel32', 'FlushFileBuffers', 'L', 'B')
    $DisconnectNamedPipe = Win32API.new('kernel32', 'DisconnectNamedPipe', 'L', 'B')
    $WriteFile           = Win32API.new('kernel32', 'WriteFile', 'LPLPP', 'B')
    $CloseHandle         = Win32API.new('kernel32', 'CloseHandle', 'L', 'B')
  end

  #-----------------------------------------------------------------------
  # setup_pipe
  #-----------------------------------------------------------------------
  def self.setup_pipe
    make_APIs
    @@name = "\\\\.\\pipe\\" + PIPE_NAME

    @@pipe_mode = PIPE_DEFAULT_MODE
    @@open_mode = PIPE_ACCESS_DUPLEX
    @@pipe         = nil
    @@buffer       = 0.chr * PIPE_BUFFER_SIZE
    @@size         = 0
    @@bytes        = [0].pack('L')

    @@pipe = $CreateNamedPipe.call(
      @@name,
      @@open_mode,
      @@pipe_mode,
      PIPE_UNLIMITED_INSTANCES,
      PIPE_BUFFER_SIZE,
      PIPE_BUFFER_SIZE,
      PIPE_TIMEOUT,
      0
    )

    if @@pipe == INVALID_HANDLE_VALUE
      # If we could not open the pipe, notify the user
      # and proceed quietly
      print "WARNING -- Unable to create named pipe: " + PIPE_NAME
      @@pipe = nil
    else
      # Prompt the user to open the pipe
      print "Please launch the RGSSMonitor.rb script"
    end
  end

  #-----------------------------------------------------------------------
  # write_to_pipe ('msg' must be a string)
  #-----------------------------------------------------------------------
  def self.write_to_pipe(msg)
    if @@pipe
      # Format data
      @@buffer = msg
      @@size   = msg.size

      $WriteFile.call(@@pipe, @@buffer, @@buffer.size, @@bytes, 0)
    end
  end

  #------------------------------------------------------------------------
  # close_pipe
  #------------------------------------------------------------------------
  def self.close_pipe
    if @@pipe
      # Send kill message to RGSSMonitor
      @@buffer = "!!GAMEOVER!!"
      @@size   = @@buffer.size
      $WriteFile.call(@@pipe, @@buffer, @@buffer.size, @@bytes, 0)

      # Close down the pipe
      $FlushFileBuffers.call(@@pipe)
      $DisconnectNamedPipe.call(@@pipe)
      $CloseHandle.call(@@pipe)
      @@pipe = nil
    end
  end
end

To use this, you only need to make sure to call PipeLogger::setup_pipe before writing an event; and call PipeLogger::close_pipe before game exit. (I put the setup call at the start of 'Main', and add an ensure clause to call close_pipe.) After that, you can add a call to PipeLogger::write_to_pipe("msg") at any point in any script with any string for "msg" and write into the pipe.

I have tested this code with RPG Maker XP; it should also work with RPG Maker VX and later.

You will also need something to read FROM the pipe. There are any number of ways to do this, but a simple one is to use a standard Ruby installation, the win32-pipe Ruby Gem, and this script:

require 'rubygems'
require 'win32/pipe'
include Win32

# -- Change THIS to change the name of the pipe!
PIPE_NAME = "RGSSPipe"

Thread.new { loop { sleep 0.01 } } # Allow Ctrl+C

pipe = Pipe::Client.new(PIPE_NAME)
continue = true

while continue
  msg = pipe.read.to_s
  puts msg

  continue = false if msg.chomp == "!!GAMEOVER!!"
end

I use Ruby 1.8.7 for Windows and the win32-pipe gem mentioned above (see here for a good reference on installing gems). Save the above as "RGSSMonitor.rb" and invoke it from the command line as ruby RGSSMonitor.rb.

CAVEATS:

1. The RGSS code listed above is fragile; in particular, it does not handle failure to open the named pipe. This is not usually an issue on your own development machine, but I would not recommend shipping this code.
2. I haven't tested it, but I suspect you'll have problems if you write a lot of things to the log without running a process to read the pipe (e.g. RGSSMonitor.rb). A Windows named pipe has a fixed size (I set it here to 1K), and by default writes will block once the pipe is filled (because no process is 'relieving the pressure' by reading from it). Unfortunately, the RPGXP engine will kill a Ruby script that has stopped running for 10 seconds. (I'm told that RPGVX has eliminated this watchdog function--in which case, the game will hang instead of abruptly terminating.)

Answer 2

What's probably happening is the writing process is exiting, and as there are no other writing processes, EOF is sent to the pipe which causes gets to return nil, and so your code loops continually.

To get around this you can usually just open the pipe read-write at the reader end. This works for me (on a Mac), but isn't working for you (you've tried "r" and "r+"). I'm guessing this is to due with Cygwin (POSIX says opening a FIFO read-write is undefined).

An alternative is to open the pipe twice, once read-only and once write-only. You don't use the write-only IO for anything, it's just so that there's always an active writer attached to the pipe so it doesn't get closed.

input = File.open("log_pipe", "r")      # note 'r', not 'r+'
keep_open = File.open("log_pipe", "w")  # ensure there's always a writer
while true
  puts input.gets
end