Can OptionParser skip unknown options, to be processed later in a Ruby program?

Question

Is there any way to kick off OptionParser several times in one Ruby program, each with different sets of options?

For example:

$ myscript.rb --subsys1opt a --subsys2opt b

Here, myscript.rb would use subsys1 and subsys2, delegating their options handling logic to them, possibly in a sequence where 'a' is processed first, followed by 'b' in separate OptionParser object; each time picking options only relevant for that context. A final phase could check that there is nothing unknown left after each part processed theirs.

The use cases are:

1. In a loosely coupled front-end program, where various components have different arguments, I don't want 'main' to know about everything, just to delegate sets of arguments/options to each part.

2. Embedding some larger system like RSpec into my application, and I'd to simply pass a command-line through their options without my wrapper knowing those.

I'd be OK with some delimiter option as well, like -- or --vmargs in some Java apps.

There are lots of real world examples for similar things in the Unix world (startx/X, git plumbing and porcelain), where one layer handles some options but propagates the rest to the lower layer.

Out of the box, this doesn't seem to work. Each OptionParse.parse! call will do exhaustive processing, failing on anything it doesn't know about. I guess I'd happy to skip unknown options.

Any hints, perhaps alternative approaches are welcome.

Answer 1

I needed a solution that wouldn't throw OptionParser::InvalidOption ever, and couldn't find an elegant solution among the current answers. This monkey patch is based on one of the other answers but cleans it up and makes it work more like the current order! semantics. But see below for an unsolved issue inherent to multiple-pass option parsing.

class OptionParser
  # Like order!, but leave any unrecognized --switches alone
  def order_recognized!(args)
    extra_opts = []
    begin
      order!(args) { |a| extra_opts << a }
    rescue OptionParser::InvalidOption => e
      extra_opts << e.args[0]
      retry
    end
    args[0, 0] = extra_opts
  end
end

Works just like order! except instead of throwing InvalidOption, it leaves the unrecognized switch in ARGV.

RSpec tests:

describe OptionParser do
  before(:each) do
    @parser = OptionParser.new do |opts|
      opts.on('--foo=BAR', OptionParser::DecimalInteger) { |f| @found << f }
    end
    @found = []
  end

  describe 'order_recognized!' do
    it 'finds good switches using equals (--foo=3)' do
      argv = %w(one two --foo=3 three)
      @parser.order_recognized!(argv)
      expect(@found).to eq([3])
      expect(argv).to eq(%w(one two three))
    end

    it 'leaves unknown switches alone' do
      argv = %w(one --bar=2 two three)
      @parser.order_recognized!(argv)
      expect(@found).to eq([])
      expect(argv).to eq(%w(one --bar=2 two three))
    end

    it 'leaves unknown single-dash switches alone' do
      argv = %w(one -bar=2 two three)
      @parser.order_recognized!(argv)
      expect(@found).to eq([])
      expect(argv).to eq(%w(one -bar=2 two three))
    end

    it 'finds good switches using space (--foo 3)' do
      argv = %w(one --bar=2 two --foo 3 three)
      @parser.order_recognized!(argv)
      expect(@found).to eq([3])
      expect(argv).to eq(%w(one --bar=2 two three))
    end

    it 'finds repeated args' do
      argv = %w(one --foo=1 two --foo=3 three)
      @parser.order_recognized!(argv)
      expect(@found).to eq([1, 3])
      expect(argv).to eq(%w(one two three))
    end

    it 'maintains repeated non-switches' do
      argv = %w(one --foo=1 one --foo=3 three)
      @parser.order_recognized!(argv)
      expect(@found).to eq([1, 3])
      expect(argv).to eq(%w(one one three))
    end

    it 'maintains repeated unrecognized switches' do
      argv = %w(one --bar=1 one --bar=3 three)
      @parser.order_recognized!(argv)
      expect(@found).to eq([])
      expect(argv).to eq(%w(one --bar=1 one --bar=3 three))
    end

    it 'still raises InvalidArgument' do
      argv = %w(one --foo=bar)
      expect { @parser.order_recognized!(argv) }.to raise_error(OptionParser::InvalidArgument)
    end

    it 'still raises MissingArgument' do
      argv = %w(one --foo)
      expect { @parser.order_recognized!(argv) }.to raise_error(OptionParser::MissingArgument)
    end
  end
end

Problem: normally OptionParser allows abbreviated options, provided there are enough characters to uniquely identify the intended option. Parsing options in multiple stages breaks this, because OptionParser doesn't see all the possible arguments in the first pass. For example:

describe OptionParser do
  context 'one parser with similar prefixed options' do
    before(:each) do
      @parser1 = OptionParser.new do |opts|
        opts.on('--foobar=BAR', OptionParser::DecimalInteger) { |f| @found_foobar << f }
        opts.on('--foo=BAR', OptionParser::DecimalInteger) { |f| @found_foo << f }
      end
      @found_foobar = []
      @found_foo = []
    end

    it 'distinguishes similar prefixed switches' do
      argv = %w(--foo=3 --foobar=4)
      @parser1.order_recognized!(argv)
      expect(@found_foobar).to eq([4])
      expect(@found_foo).to eq([3])
    end
  end

  context 'two parsers in separate passes' do
    before(:each) do
      @parser1 = OptionParser.new do |opts|
        opts.on('--foobar=BAR', OptionParser::DecimalInteger) { |f| @found_foobar << f }
      end
      @parser2 = OptionParser.new do |opts|
        opts.on('--foo=BAR', OptionParser::DecimalInteger) { |f| @found_foo << f }
      end
      @found_foobar = []
      @found_foo = []
    end

    it 'confuses similar prefixed switches' do
      # This is not generally desirable behavior
      argv = %w(--foo=3 --foobar=4)
      @parser1.order_recognized!(argv)
      @parser2.order_recognized!(argv)
      expect(@found_foobar).to eq([3, 4])
      expect(@found_foo).to eq([])
    end
  end
end

Answer 2

Assuming the order in which the parsers will run is well defined, you can just store the extra options in a temporary global variable and run OptionParser#parse! on each set of options.

The easiest way to do this is to use a delimiter like you alluded to. Suppose the second set of arguments is separated from the first by the delimiter --. Then this will do what you want:

opts = OptionParser.new do |opts|
  # set up one OptionParser here
end

both_args = $*.join(" ").split(" -- ")
$extra_args = both_args[1].split(/\s+/)
opts.parse!(both_args[0].split(/\s+/))

Then, in the second code/context, you could do:

other_opts = OptionParser.new do |opts|
  # set up the other OptionParser here
end

other_opts.parse!($extra_args)

Alternatively, and this is probably the "more proper" way to do this, you could simply use OptionParser#parse, without the exclamation point, which won't remove the command-line switches from the $* array, and make sure that there aren't options defined the same in both sets. I would advise against modifying the $* array by hand, since it makes your code harder to understand if you are only looking at the second part, but you could do that. You would have to ignore invalid options in this case:

begin
    opts.parse
rescue OptionParser::InvalidOption
    puts "Warning: Invalid option"
end

---

The second method doesn't actually work, as was pointed out in a comment. However, if you have to modify the $* array anyway, you can do this instead:

tmp = Array.new

while($*.size > 0)
    begin
        opts.parse!
    rescue OptionParser::InvalidOption => e
        tmp.push(e.to_s.sub(/invalid option:\s+/,''))
    end
end

tmp.each { |a| $*.push(a) }

It's more than a little bit hack-y, but it should do what you want.

Answer 3

I've got the same problem, and I found the following solution:

options = ARGV.dup
remaining = []
while !options.empty?
  begin
    head = options.shift
    remaining.concat(parser.parse([head]))
  rescue OptionParser::InvalidOption
    remaining << head
    retry
  end
end

Answer 4

For posterity, you can do this with the order! method:

option_parser.order!(args) do |unrecognized_option|
  args.unshift(unrecognized_option)
end

At this point, args has been modified - all known options were consumed and handled by option_parser - and can be passed to a different option parser:

some_other_option_parser.order!(args) do |unrecognized_option|
  args.unshift(unrecognized_option)
end

Obviously, this solution is order-dependent, but what you are trying to do is somewhat complex and unusual.

One thing that might be a good compromise is to just use -- on the command line to stop processing. Doing that would leave args with whatever followed --, be that more options or just regular arguments.