Lisp / Clojure: Is it a good idea to write function generating macros?

Question

This question asks to create a Clojure macro to generate several functions. We figured out a way to do this but were stuck with the question of "Is this a good idea?".

My initial reaction is not really, for two reasons

1. You then have functions that are not defined in your code, and this can complicate understanding your code quite a bit! (Imagine somebody has a problem with one of your functions and looks at the source code only to not find it anywhere).
2. It is better to factor out the commonality of the code in a function or macro. Letting your computer write a bunch of functions that are very alike is a poor approach to that.

What do you think? When does generating functions in a Lisp make sense? Should it ever be 'on the fly' or would you prefer to have it in a file somewhere?

Answer 1

The complaint about code complexity have been following macros around for years. Every abstraction is there to hide complexity, whether it's a macro, or function, or whatever.

The value of factoring to a function is reuse, as functions are more reusable than macros. Not just in the case of being able to use "apply", but in the literal case of shared code. A shared function is simply a pointer to implementation of the function. A shared and reused macro results in several COPIES of functions or code or whatever, and while the abstraction is there, the code is not shared at all within the system.

Now, you could make a really clever macro that checks for a function definition at expansion, and if it doesn't find it, then it can create the function on the fly, or do some other clever thing.

But even with the functions factored out, they're still going to be ostensibly hidden from the user since that the basic premise behind the macro in the first place. Putting those helper functions in to some hidden package doesn't make then any more visible to the consumer unless they know to even look within the source code (assuming they even have the source code).

Ideally, the functions would be uninteresting to the developer, as they have "no user serviceable parts inside". If they do, then the macro is not sufficient support or documentation for these function in the first place.

Answer 2

TL;DR: It depends.

Can the commonality, or a subset, be abstracted into a function (or functions), with only the more dynamic bits defined through macros? When possible, it's best to make the macro-y parts as limited in scope as possible.

What's the nature of the functions/macros? If they're part of a well-documented system aspect, it doesn't really matter where they come from.

Are they poorly-understood, and do they require frequent inspection to understand or verify behavior? If so, then leaving them as real functions may make more sense. If they're not, and are more or less "stock" system aspects, do whatever is cleaner.

Are the functions/macros maintained by everybody, or by someone more focused on an underlying system implementation? If they're mostly consumed, it matters less how/where/when they're implemented.

Answer 3

Good tools will take care of concern #1. In Emacs/SLIME, just press M-. on a symbol and it takes you to wherever it was defined. Of course, if your macro is very complicated that may not help you much but that's another concern.

As for #2, that's very much a design consideration. You can write functions "on the fly" using no macros at all and it won't make much difference.

My personal main concern re macros, especially macros defining vars, is that they turn out to be less composable than I want them to be. I think I agree with Dave Newton here that you should limit the scope of what any particular macro does and try to factor out as much code into public functions (and maybe a few simple macros) as possible. Especially if your macros create some object and assign it to a var (i.e. anything that starts with def...) you probably should make sure there is also a way to create that object "anonymously" and if you don't need a macro for that, even better.