How to use a "taken" word as a CLOS generic

Question

Generics seem to offer a nice facility for pulling out a common word and letting it act on things according to the types you pass it, with extensibility after-the-fact.

But what about common words that are already taken and aren't defined as generic? If I try to define REMOVE, for instance:

(defclass reticulator () (splines))

(defmethod remove ((item reticulator)
     sequence &key from-end test test-not start end count key))

I get an error in SBCL:

COMMON-LISP:REMOVE already names an ordinary function or a macro.

Is there an idiom or approved way to "generic-ify" one of these built-in functions? Do people do this?

Just to see what would happen I tried overriding REMOVE with a generic:

(defgeneric remove (item sequence &key from-end test test-not start end count key))

WARNING: redefining COMMON-LISP:REMOVE in DEFGENERIC

I don't know if there's a "good" way of doing that, which would pipe to the old implementation with allowance of overloading for specific types that want to give the word new meaning.

Answer 1

Why are things how they are?

The first version of Common Lisp was designed starting in 1981/82 and the result was published as the book Common Lisp the Language in 1984. The language itself was largely designed based on Lisp Machine Lisp (aka Zetalisp). Zetalisp was much larger than the then published Common Lisp and included an early object system called Flavors. Much of the stuff in Zetalisp was implemented in an object-oriented fashion - which did cost performance and on Lisp Machines the performance hit wasn't so large - but they had specialized processors which had optimized instruction sets. So Common Lisp did not include any object-system and thus it was slightly optimized for performance on typical processors of that time. In a later version of Common Lisp, an object-system was to be added - when there was enough experience with object-oriented extensions to Lisp - remember, we are talking about the early 80s.

This 1984 Common Lisp had a limited form of generic behavior. For example the function REMOVE works on sequences - and sequences is a new type, which has vectors and lists as subtypes.

Later Common Lisp was standardized starting in 1986 and one were looking for an object-system for Common Lisp - none of the proposed were good enough - so a new one was developed based on New Flavors (from Symbolics, a newer version of the above mentioned Flavors) and Common Loops (from Xerox PARC). Those had already generic functions, but single dispatch. CLOS then added multiple dispatch.

It was decided to not replace basic functions with CLOS generic functions - one reason for that is performance: CLOS generic functions need a relatively complex dispatch mechanism and this dispatch is decided at runtime. There is no CLOS feature to have static compile-time dispatch and there is also no standardized feature to make the parts of the classes 'sealed' -> so that they cannot be changed. Thus a highly-dynamic system like CLOS has runtime costs.

Some functions are defined as CLOS generic functions (like PRINT-OBJECT) and some implementations have large parts of Common Lisp with CLOS implementations (streams, conditions, ...) - but this is implementation specific and not required by the standard. There are also several libraries, which provide CLOS based functionality of built-in CL features: like I/O with extensible CLOS-based streams.

Also note that redefining existing Common Lisp functions is undefined behavior.

So Common Lisp chose to provide a powerful object-system, but leave it to the individual implementations how much they want to use CLOS in the base language - with the limitations that functions which are standardized as normal non-CLOS-generic functions usually should not be replaced with CLOS functions by users.

Some Lisp dialects/implementations tried to deal with these problems and tried to define a faster CLOS variant, which then would be the base for much of the language. See for example Apple's Dylan language. For some newer approach see the language Julia.

Your own improved Common Lisp

Packages (-> symbol namespaces) allow you to define your own improved CL: here we define a new package which has all CL symbols, with only cl:remove being shadowed by its own symbol. We then define a CLOS generic function named bettercl::remove and write two example methods.

CL-USER 165 > (defpackage "BETTERCL" (:use "CL") (:shadow cl:remove))
#<The BETTERCL package, 1/16 internal, 0/16 external>

CL-USER 166 > (in-package "BETTERCL")
#<The BETTERCL package, 1/16 internal, 0/16 external>

BETTERCL 167 > (defgeneric remove (item whatever))
#<STANDARD-GENERIC-FUNCTION REMOVE 4060000C64>

BETTERCL 168 > (defmethod remove (item (v vector)) (cl:remove item v))
#<STANDARD-METHOD REMOVE NIL (T VECTOR) 40200AB12B>

BETTERCL 169 > (remove 'a #(1 2 3 a b c))
#(1 2 3 B C)

BETTERCL 170 > (defmethod remove ((digit integer) (n integer))
                 "remove a digit from an integer, returns a new integer"
                 (let ((s (map 'string
                               (lambda (item)
                                 (character (princ-to-string item)))
                               (cl:remove digit
                                          (map 'vector
                                               #'digit-char-p
                                               (princ-to-string n))))))
                   (if (= (length s) 0) 0 (read-from-string s))))
#<STANDARD-METHOD REMOVE NIL (INTEGER INTEGER) 40200013C3>

BETTERCL 171 > (remove 8 111888111348111)
11111134111

Now you could also export the symbols from BETTERCL, such that you can use this package in your application packages, instead of the package CL.

This approach has been used before. For example CLIM (the Common Lisp Interface Manager) defines a package CLIM-LISP, which is used as the dialect to program in.

Common Lisp provides sometimes both a function and a related CLOS generic function

See the standard function DESCRIBE, which can be extended by writing methods for the standard CLOS generic function DESCRIBE-OBJECT.

Improvement experiments for Common Lisp

Another way to make standard Common Lisp functions extensible, is to replace them with versions which then use an extensible CLOS-based protocol. Note that HOW to replace standard functions is implementation specific and the effects may also be implementation specific. If a compiler for example has inlined a built-in function into code, then redefining will have no effect on the already inlined code.

For an example of this approach see the paper (PDF) User-extensible sequences in Common Lisp by Christophe Rhodes.

When to use generic functions?

There are a few things to keep in mind:

• define CLOS generic functions when there are multiple related methods, which ideally benefit from a common extension mechanism and which are likely to be extended

• think about the performance hit

• don't use a single CLOS generic function for functions which have a similar arglist and even the same name, but which work for very different domains

This means that you should not define functions in a single CLOS generic function like:

; do some astrophysics calculations
(defmethod rotate-around ((s star) (u galaxy)) ...)

; do some computation with graphics objects
(defmethod rotate-around (shape (a axis)) ...)

For example writing :before, :around and :after methods might not lead to useful results.

One could have two different rotate-around generic functions, one in a package ASTRO-PHYSICS and the other one in a package GRAPHICS-OBJECTS. Thus the methods would not be in the same CLOS generic function and extending those would be probably easier.