Common lisp: Can I define a function with arbitrary number of args and optional keyword args?

Question

I am fairly new to CL coming from R and Python.

I want to define a function to which I can pass an arbitrary number of arguments and also have keywords with default values that I can set if I want different values from the defaults.

In R I can do this like so:

foo <- function(..., a = 1, b = 2){
    list(a = a, b = b, ...)
}

foo(1, 2, 3)
foo(1, 2, 3, a = 2)
foo(b = 10, 1, 2, 3)

In PCL it says you can combine &key and &rest arguments but it doesn't work in the way I would expect if I try something like

(defun foo (&rest rest &key (a 1) (b 2))
   (list rest a b))

Here, I get an unknown &key error if I specify anything other than the two keyword args:

(foo :a 100 12 3)
>> unknown &KEY argument: 12

What I want is similar functionality to this:

(defun bar (&optional (a 1) (b 2) &rest rest)
   (list rest a b))

(bar 5 4 1 2 3 4)
>>((1 2 3 4) 5 4)

But I want to choose whether or not I supply to argument to :a and :b. I am using sbcl.

Answer 1

There is no standard way to do this in CL. You could make the function take everything in the rest parameter, and parse the keyword arguments yourself, if you don't want to design the API differently.

That said, here are a few points for further exploration. They might be useful in specific use cases, but are rather limited and exploit implementation-dependent behavior.

You can use &allow-other-keys in the lambda list or :allow-other-keys t when calling foo to prevent the unknown key error, but rest will also include the keys and values of your keyword arguments:

CL-USER> (defun foo (&rest rest &key (a 1) (b 2))
           (list rest a b))
FOO
CL-USER> (foo)
(NIL 1 2)
CL-USER> (foo :a 100 12 3 :allow-other-keys t)
((:A 100 12 3 :ALLOW-OTHER-KEYS T) 100 2)
CL-USER> (defun foo (&rest rest &key (a 1) (b 2) &allow-other-keys)
           (list rest a b))
FOO
CL-USER> (foo)
(NIL 1 2)
CL-USER> (foo :a 100 12 3)
((:A 100 12 3) 100 2)

As acelent correctly points out in a comment below, this might signal an error. It works for me in CLISP, SBCL, and CCL under the default optimize settings, but by the standard, keyword argument names (i.e. the first of each pair of arguments) must be symbols. Whether or not this works depends on the safety level and is implementation-dependent. It should signal an error (on conforming implementations) in safe code (safety level of 3).

In general, allowing other keys can be useful for passing keyword arguments through, but is not exactly what you wanted. One quick and dirty way might be filtering for keyword parameters in rest and just dropping them and their succeeding elements. Something like this:

CL-USER> (defun foo (&rest rest &key (a 1) (b 2) &allow-other-keys)
           (let ((rest (loop for (key value) on rest by #'cddr
                             unless (keywordp key)
                               append (list key value))))
             (list rest a b)))
FOO
CL-USER> (foo)
(NIL 1 2)
CL-USER> (foo :a 100 12 3)
((12 3) 100 2)

Which will, alas, by the standard only work for even numbers of arguments, as coredump points out in his answer. (It might work with an odd number of arguments under some implementations for some safety levels, but it didn't work in the implementations I tested.) Also, obviously it isn't robust in other ways (different positions of keyword arguments etc.), and not meant for production use, but just as a starting point for possible exploration.

The proper solution would involve writing your own keyword argument parser, and use it with rest, or, as pointed out in coredump's answer, using a different API. Another point worth mentioning is that in CL, applying large numbers of arguments is generally not a good idea since it might lead to inefficient code. What's worse, it is also not very reliable, since the number of allowed arguments is implementation-dependent. For example, under CCL on my system, call-arguments-limit is 65536. It may be significantly – even orders of magnitude – smaller under other implementations and systems. So, in general, prefer reduceing to applying large numbers of arguments.

Answer 2

Basically, here is how variable numbers of arguments are handled in conjunction with keywords:

• First, mandatory and optional arguments are bound to variables.
• Then, the remaining supplied arguments are bound to the &rest parameter, if given; let's say the parameter is named rest. It is the list of all remaining arguments.
• Now, if &key was also supplied in the lambda form, keywords are extracted from rest, which is then expected to have an even number of arguments (cf. specification), where keywords and values alternate, as in (:k1 v1 :k2 v2 ...).
• Unless you append &allow-other-keys in the lambda form definition or supply :allow-other-keys T when calling it, the number of supplied arguments must match exactly the number (and name) of expected keywords parameters you can only provide the keyword parameters declared in the lambda form (see @acelant's comments for details).

So, what can you do?

The simplest approach would be to define your functions so that all parameters are bound to keywords, with default values. You may also allow the caller to pass additional arguments:

    (defun foo (&rest rest &key (a 1) (b 2) &allow-other-keys) ...)

This is very similar to your definition, but you cannot just pass values; all arguments must be given along with a key:

    (foo :a 100 :max-depth 12 :max-try 3) ;; for example

CL is not R nor Python: maybe you don't need to pass so many arguments to your functions, maybe you can use special variables (dynamic scope), generic methods, ... Functions in existing packages usually mix mandatory, optional and keyword parameters. Take the time to read standard APIs (e.g. cl-ppcre, drakma) and see how you could define your functions in a more idiomatic way.

To conclude, maybe you really need to define functions with such R-like arguments. If this is the case, you can try to define your own macros for lambda lists in function definitions forms, so that

    (r-defun FOO ((a 1) (b 2) &rest rest) ...)

gets translated to something like:

    (defun FOO (&rest args)
       (let* ((rest (copy-seq args)) ;; must not modify "args"
              (a (find-keyword-and-remove-from-rest :a rest :default 1)
              (b (find-keyword-and-remove-from-rest :b rest :default 2))
          ... ;; function body
    ))

Do this if you want to have fun with macroexpansion, but really, I'am pretty sure this is not needed.

---

EDIT I originally modified the args argument (previously named rest) instead of doing a copy, but that was a bad example. The specification says :

When the function receives its arguments via &rest, it is permissible (but not required) for the implementation to bind the rest parameter to an object that shares structure with the last argument to apply. Because a function can neither detect whether it was called via apply nor whether (if so) the last argument to apply was a constant, conforming programs must neither rely on the list structure of a rest list to be freshly consed, nor modify that list structure.