How to write similar functions in Common Lisp?

Question

I'm learning Common Lisp from Practical Common Lisp. It has an example of helper functions for reading and writing binary files in Chapter 24. Here's one example:

(defun read-u2 (in)
  (+ (* (read-byte in) 256) (read-byte in)))

I can write functions for reading other kinds of binary numbers likewise. But I thought that doing so violates the DRY principle. Besides, these functions are going to be similar, so I tried to generate the functions with macros.

(defmacro make-read (n be)
  `(defun ,(intern (format nil "READ~d~:[L~;B~]E" n be))
       (&optional (stream *standard-input*))
     (logior ,@(loop for i from 0 below n collect
                `(ash (read-byte stream)
                      ,(* 8 (if be (- n 1 i) i)))))))

(defmacro make-read-s (n be)
  `(defun ,(intern (format nil "READ~d~:[L~;B~]E-S" n be))
       (&optional (stream *standard-input*))
     (let ((a (,(intern (format nil "READ~d~:[L~;B~]E" n be)) stream)))
       (if (zerop (logand a ,(ash 1 (1- (* 8 n)))))
       a
       (logior a ,(ash -1 (* 8 n)))))))

(defmacro make-write (n be)
  `(defun ,(intern (format nil "WRITE~d~:[L~;B~]E" n be))
       (n &optional (stream *standard-output*))
     (setf n (logand n ,(1- (ash 1 (* 8 n)))))
     ,@(loop for i from 0 below n collect
        `(write-byte (ldb (byte 8 ,(* 8 (if be (- n 1 i) i))) n)
                     stream))))

(eval-when (:compile-toplevel :load-toplevel :execute)
  (dolist (cat '("READ" "READ-S" "WRITE"))
    (dolist (be '(nil t))
      (dolist (n '(1 2 4 8))
        (eval `(,(intern (format nil "MAKE-~a" cat)) ,n ,be))))))

It works. It generates functions for reading and writing unsigned and signed integers in sizes of 1, 2, 4, and 8. SLIME understands it. But I wonder if there are better ways.

What's the best way to write a bunch of similar functions in Common Lisp?

Answer 1

There are some issues with this code, though the general approach to have macros generating functions is fine.

Naming

The macros should not be named make-..., because they are not functions which make something, but macros which define a function.

Code generation

The EVAL-WHEN ... EVAL code is really bad and should not be used this way.

The better way is to write macro which expands into a progn with the function definitions.

If I wanted to use EVAL, then I would not need to write code generating macros, but simply code generating functions. But I don't want to use EVAL, I want to create code for the compiler directly. If I have code generating macros, then I don't need EVAL.

EVAL is not a good idea, because it is not clear that the code would be compiled - which would be implementation dependent. Also the evaluation would take place at compile time and load time. It would be better to compile the functions at compile time and only load them at load time. A file compiler also might miss possible optimizations for the evaluated functions.

(defmacro def-read-fun (n be)
  `(defun ,(intern (format nil "READ~d~:[L~;B~]E" n be))
          (&optional (stream *standard-input*))
     (logior ,@(loop for i from 0 below n collect
                     `(ash (read-byte stream)
                           ,(* 8 (if be (- n 1 i) i)))))))

(defmacro def-read-s-fun (n be)
  `(defun ,(intern (format nil "READ~d~:[L~;B~]E-S" n be))
          (&optional (stream *standard-input*))
     (let ((a (,(intern (format nil "READ~d~:[L~;B~]E" n be)) stream)))
       (if (zerop (logand a ,(ash 1 (1- (* 8 n)))))
           a
         (logior a ,(ash -1 (* 8 n)) )))))

(defmacro def-write-fun (n be)
  `(defun ,(intern (format nil "WRITE~d~:[L~;B~]E" n be))
          (n &optional (stream *standard-output*))
     (setf n (logand n ,(1- (ash 1 (* 8 n)))))
     ,@(loop for i from 0 below n collect
             `(write-byte (ldb (byte 8 ,(* 8 (if be (- n 1 i) i))) n)
                          stream))))

Instead of the EVAL-WHEN ... EVAL we define another macro and then we use it later:

(defmacro def-reader/writer-functions (cat-list be-list n-list)
  `(progn
     ,@(loop for cat in cat-list append
             (loop for be in be-list append
                   (loop for n in n-list
                         collect `(,(intern (format nil "DEF-~a-FUN" cat))
                                   ,n
                                   ,be))))))

Now we can use above macro to generate all the functions:

(def-reader/writer-functions
 ("READ" "READ-S" "WRITE")
 (nil t)
 (1 2 4 8))

You can see the expansion here:

CL-USER 173 > (pprint (macroexpand-1 '(def-reader/writer-functions
                                       ("READ" "READ-S" "WRITE")
                                       (nil t)
                                       (1 2 4 8))))

(PROGN
  (DEF-READ-FUN 1 NIL)
  (DEF-READ-FUN 2 NIL)
  (DEF-READ-FUN 4 NIL)
  (DEF-READ-FUN 8 NIL)
  (DEF-READ-FUN 1 T)
  (DEF-READ-FUN 2 T)
  (DEF-READ-FUN 4 T)
  (DEF-READ-FUN 8 T)
  (DEF-READ-S-FUN 1 NIL)
  (DEF-READ-S-FUN 2 NIL)
  (DEF-READ-S-FUN 4 NIL)
  (DEF-READ-S-FUN 8 NIL)
  (DEF-READ-S-FUN 1 T)
  (DEF-READ-S-FUN 2 T)
  (DEF-READ-S-FUN 4 T)
  (DEF-READ-S-FUN 8 T)
  (DEF-WRITE-FUN 1 NIL)
  (DEF-WRITE-FUN 2 NIL)
  (DEF-WRITE-FUN 4 NIL)
  (DEF-WRITE-FUN 8 NIL)
  (DEF-WRITE-FUN 1 T)
  (DEF-WRITE-FUN 2 T)
  (DEF-WRITE-FUN 4 T)
  (DEF-WRITE-FUN 8 T))

Each of the subforms then will be expanded into the function definitions.

This way the compiler runs the macros to generate all the code at compile time and the compiler can then generate code for all the functions.

Efficiency / Defaults

In a lowest-level function I may not want to use an &optional parameter. The default call would get the value from a dynamic binding and, worse, *standard-input* / *standard-output* may not be a stream for which READ-BYTE or WRITE-BYTE works. Not in every implementation you can use a standard input/output stream as a binary stream.

LispWorks:

CL-USER 1 > (write-byte 13 *standard-output*)

Error: STREAM:STREAM-WRITE-BYTE is not implemented for this stream type: #<SYSTEM::TERMINAL-STREAM 40E01D110B>
  1 (abort) Return to level 0.
  2 Restart top-level loop.

I also may want to declare all generated functions to be inlined.

Type declarations would be another thing to think about.

Summmary: don't use EVAL.

Answer 2

Generally, I'd prefer to just add the number of bytes to read as another parameter to the function:

(defun read-integer (stream bytes)
  (check-type bytes (integer 1 *))
  (loop :repeat bytes
        :for b := (read-byte stream)
        :for n := b :then (+ (* n 256) b)
        :finally (return n)))

Signedness and endianness could be added as keyword arguments. This way of programming is good for understandable code that is also easily navigated through tools like SLIME.

Unrolling this through macros is a valid optimization strategy, and I defer to Rainer's answer.

In the specific case of reading numbers from a stream, optimization is likely a valid goal from the start, since this tends to get used a lot in tight loops.

If you do this, however, you should also thoroughly document what gets generated. If a reader of the code sees an operator read8bes, he cannot easily find out where it was defined. You need to help him.