CFFI, callbacks, and void * (how do I create a pointer to a native object in common lisp/cffi?)

Question

Let's say I have a a callback in a physics library in C that generates collision information for two objects that are nearby/touching. This callback takes a (void *)data parameter which I'd like to use to pass in one of my lisp objects.

How do I make use of this (void *) parameter? From what I can think of (I'm relatively new to CFFI but getting the hang of it pretty quickly) I could either

1. Somehow cast the object (a CLOS object) as a pointer and pass this pointer in to the callback. I've tried (cffi:convert-to-foreign my-obj :pointer) but it just returns my-obj, not a pointer, and I can't call the callback with it.
2. Create a closure around the callback so it can reference the data without needing the void* pointer. I've searched google a bunch and haven't found any info on this. Can I just wrap defcallback in a defun?
3. Shove the object I want to pass into the global scope. This seems really, really dirty and I'd like to avoid it at all costs.

Any ideas on a cross-platform way to do this? I know a lot of callbacks in C take a void* parameter, so someone must have thought of this. I'm using Clozure CL, but like I said, the more cross-platform/cross-implementation the better. Thanks!

Answer 1

Sorry if my answer is a bit late but I have come across the same problem recently. The solution I used was to store all the lisp objects that are to be accessed by callbacks in a hash table using an integer key. All I have to do then is pass the key around to the c functions, cast to void* if needed.

Answer 2

I'm not sure this would work. It would be really ugly if the GC moves the object that you captured in the closure. Maybe you would have to disable the GC. http://www.sbcl.org/manual/Calling-Lisp-From-C.html#Calling-Lisp-From-C

Perhaps an option is to call another Lisp function from within the callback to set a variable but for this to work the address of the Lisp function shouldn't change. I don't know if the address of the lisp function can change during garbage collection or when you redefine the function.

Callbacks are not portable: http://common-lisp.net/project/cffi/manual/html_node/Tutorial_002dCallbacks.html

You can look here if your implementation supports callbacks: http://common-lisp.net/project/cffi/manual/html_node/Implementation-Support.html#Implementation-Support

I modified an example from the CFFI help. The callback function is the comparison operator and it calls a function to print out each comparison it does. Instead of printing the list PAR you could just push it into a global variable (or closure).

Note that I am not convinced that this usage is save. If for some reason the position of STORE changes in memory and the code in the callback function isn't updated accordingly, that could lead to bad results.

(require :cffi)

(defpackage :run
  (:use :cl :cffi))
(in-package :run)

(defcfun "qsort" :void
    (base :pointer)
    (nmemb :int)
    (size :int)
    (fun-compar :pointer))


(defun store (par)
  (format t "I'm comparing ~a~%" par))


(defcallback < :int ((a :pointer) (b :pointer))
    (let ((x (mem-ref a :int))
          (y (mem-ref b :int)))
      (store (list x y))
      (cond ((> x y) 1)
            ((< x y) -1)
            (t 0))))

(with-foreign-object (array :int 10)
  ;; Initialize array.
  (loop for i from 0 and n in '(7 2 10 4 3 5 1 6 9 8)
     do (setf (mem-aref array :int i) n))
          ;; Sort it.
  (qsort array 10 (foreign-type-size :int) (callback <))
  ;; Return it as a list.
  (loop for i from 0 below 10
     collect (mem-aref array :int i)))

This is the output of the example:

I'm comparing (7 2)
I'm comparing (4 3)
I'm comparing (10 3)
I'm comparing (10 4)
I'm comparing (2 3)
I'm comparing (7 3)
I'm comparing (7 4)
I'm comparing (7 10)
I'm comparing (5 1)
I'm comparing (9 8)
I'm comparing (6 8)
I'm comparing (1 6)
I'm comparing (5 6)
I'm comparing (2 1)
I'm comparing (2 5)
I'm comparing (3 5)
I'm comparing (4 5)
I'm comparing (7 5)
I'm comparing (7 6)
I'm comparing (7 8)
I'm comparing (10 8)
I'm comparing (10 9)

I searched for callback on the freenode #lisp channel logs logs:

http://ccl.clozure.com/irc-logs/lisp/2010-04/lisp-2010.04.28.txt
Best comment there:
12:56:53 <pkhuong> and if you find out that threads are indeed an issue, I'd code a C wrapper to build a message queue.

Maybe these codesamples have been corrected (I can't check right now because the server is down):

http://paste.lisp.org/display/98482
http://paste.lisp.org/display/98495