Specify options and arguments dynamically

Question

I'd like to load arguments and options from a database. I am allowing users to define their own options and arguments. Users can call a remote api on the command line. They specify the URL and parameters to the end point. Here is what the data from database looks like

[
    {
        "name": "thename1",
        "short": "a",
        "long": "ace"
        "type": "string",
        "required": false
    },
    {
        "name": "thename2",
        "short": "b",
        "long": "bravo"
        "type": "number",
        "required": true
    },
    {
        "name": "thename3",
        "short": "c",
        "long": "candy"
        "type": "array",
        "required": true
    }
]

These parameters align with what the remote end point needs. Each API endpoint has different parameters hence the reason they need to be dynamic. Here is how the command line will look.

command run www.mysite.com/api/get -a testparam --bravo testpara2 -c item1 item2

The param and value will be mapped in a URL. Is there a way to setup dynamic parameters in click?

Answer 1

This can be done by building a custom decorator that calls the click.option decorator multiple times after translating the given data structure into click equivalents.

Code:

import click

def options_from_db(options):
    map_to_types = dict(
        array=str,
        number=float,
        string=str,
    )
    def decorator(f):
        for opt_params in reversed(options):
            param_decls = (
                '-' + opt_params['short'],
                '--' + opt_params['long'],
                opt_params['name'])
            attrs = dict(
                required=opt_params['required'],
                type=map_to_types.get(
                    opt_params['type'], opt_params['type'])
            )
            if opt_params['type'] == 'array':
                attrs['cls'] = OptionEatAll
                attrs['nargs'] = -1

            click.option(*param_decls, **attrs)(f)
        return f
    return decorator

Using the options_from_db decorator:

To use the new decorator, decorate the command, and pass the option data from the db like:

@options_from_db(run_options)
def command(*args, **kwargs):
    ....

How does this work?

The @click.option() decorator is, like all decorators, a function. In this case it annotates the decorated function, and returns the same function. So we can simply call it multiple times to annotate our decorated function.

Note: your array parameter violates the click requirement to not allow nargs < 0 on options. But there is another answer that enables this, and this answer uses the code from there.

Code from other answer:

class OptionEatAll(click.Option):

    def __init__(self, *args, **kwargs):
        self.save_other_options = kwargs.pop('save_other_options', True)
        nargs = kwargs.pop('nargs', -1)
        assert nargs == -1, 'nargs, if set, must be -1 not {}'.format(nargs)
        super(OptionEatAll, self).__init__(*args, **kwargs)
        self._previous_parser_process = None
        self._eat_all_parser = None

    def add_to_parser(self, parser, ctx):

        def parser_process(value, state):
            # method to hook to the parser.process
            done = False
            value = [value]
            if self.save_other_options:
                # grab everything up to the next option
                while state.rargs and not done:
                    for prefix in self._eat_all_parser.prefixes:
                        if state.rargs[0].startswith(prefix):
                            done = True
                    if not done:
                        value.append(state.rargs.pop(0))
            else:
                # grab everything remaining
                value += state.rargs
                state.rargs[:] = []
            value = tuple(value)

            # call the actual process
            self._previous_parser_process(value, state)

        retval = super(OptionEatAll, self).add_to_parser(parser, ctx)
        for name in self.opts:
            our_parser = parser._long_opt.get(
                name) or parser._short_opt.get(name)
            if our_parser:
                self._eat_all_parser = our_parser
                self._previous_parser_process = our_parser.process
                our_parser.process = parser_process
                break
        return retval

Test Code:

run_options = [
    {
        "name": "thename1",
        "short": "a",
        "long": "ace",
        "type": "string",
        "required": False
    }, {
        "name": "thename2",
        "short": "b",
        "long": "bravo",
        "type": "number",
        "required": True
    }, {
        "name": "thename3",
        "short": "c",
        "long": "candy",
        "type": "array",
        "required": True
    }
]

@click.group()
def cli():
    pass

@cli.command()
@options_from_db(run_options)
@click.argument('url')
def run(*args, **kwargs):
    click.echo('args: {}'.format(args) )
    click.echo('kwargs: {}'.format(kwargs))


if __name__ == "__main__":
    commands = (
        'run www.mysite.com/api/get -a testparam --bravo 5 -c item1 item2',
        '',
        '--help',
        'run --help',
    )

    import sys, time

    time.sleep(1)
    print('Click Version: {}'.format(click.__version__))
    print('Python Version: {}'.format(sys.version))
    for cmd in commands:
        try:
            time.sleep(0.1)
            print('-----------')
            print('> ' + cmd)
            time.sleep(0.1)
            cli(cmd.split())

        except BaseException as exc:
            if str(exc) != '0' and \
                    not isinstance(exc, (click.ClickException, SystemExit)):
                raise

Results:

Click Version: 6.7
Python Version: 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)]
-----------
> run www.mysite.com/api/get -a testparam --bravo 5 -c item1 item2
args: ()
kwargs: {'thename1': 'testparam', 'thename2': 5.0, 'thename3': ('item1', 'item2'), 'url': 'www.mysite.com/api/get'}
-----------
> 
Usage: test.py [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  run
-----------
> --help
Usage: test.py [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  run
-----------
> run --help
Usage: test.py run [OPTIONS] URL

Options:
  -a, --ace TEXT
  -b, --bravo FLOAT  [required]
  -c, --candy TEXT   [required]
  --help             Show this message and exit.