Passing a callback through cl-function

Question

I'm trying to use the excellent request.el library request data from a REST API:

(request
 "http://httpbin.org/get"
 :params '(("key" . "value") ("key2" . "value2"))
 :parser 'json-read
 :success (function*
           (lambda (&key data &allow-other-keys)
             (message "I sent: %S" (assoc-default 'args data)))))

Which works nicely. Being a lisp-newbie I don't really know what the function* does here, I just got that from the request.el-examples.

I then tried to wrap this call in a function to reduce boilerplate like so:

(defun my/do-request (callback)
  (request
   "http://httpbin.org/get"
   :params '(("key" . "value") ("key2" . "value2"))
   :parser 'json-read
   :success callback))

(my/do-request (lambda (data)
                 (message "got data: %s" data)))

But the callback is not being called? I also tried passing the callback like this:

(defun my/do-request (callback)
  (request
   "http://httpbin.org/get"
   :params '(("key" . "value") ("key2" . "value2"))
   :parser 'json-read
   :success (function*
             (lambda (&key data &allow-other-keys)
               (callback data)))))

with the same result. I thought I might need lexical binding here, but that also didn't help.

How can I reduce the boilerplate code here?

Answer 1

When calling the new function, it is generally a good practice to try it with the exact same value you originally had:

(my/do-request
 (function*
  (lambda (&key data &allow-other-keys)
   (message "I sent: %S" (assoc-default 'args data)))))

The above prints the desired message.

First approach

You called the code as follows, and nothing was printed:

(my/do-request (lambda (data)
                 (message "got data: %s" data)))

It turns out there is an error, but unfortunately it does not reach the user. In case of doubt, you should enable the debugger:

(setf debug-on-error t)

You can eval the above in the *scratch* buffer or in the minibuffer after doing M-: (eval-expression).

Then, when you reevaluate the call, the following should be displayed:

Debugger entered--Lisp error: (wrong-number-of-arguments (lambda (data) (message "got data: %s" data)) 8)
  (lambda (data) (message "got data: %s" data))(:data ((args (key . "value") (key2 . "value2")) (headers (Accept . "*/*") (Accept-Encoding . "deflate, gzip") (Connection . "close") (Host . "httpbin.org") (User-Agent . "curl/7.55.1")) (origin . .....) (url . "http://httpbin.org/get?key=value&key2=value2")) :symbol-status success :error-thrown nil :response [cl-struct-request-response 200 nil ((args (key . "value") (key2 . "value2")) (headers (Accept . "*/*") (Accept-Encoding . "deflate, gzip") (Connection . "close") (Host . "httpbin.org") (User-Agent . "curl/7.55.1")) (origin . .....) (url . "http://httpbin.org/get?key=value&key2=value2")) nil success "http://httpbin.org/get?key=value&key2=value2" nil (:params (("key" . "value") ("key2" . "value2")) :parser json-read :success (lambda (data) (message "got data: %s" data)) :error #[128 "\302\300\303\301\"\"\207" [request-default-error-callback ("http://httpbin.org/get") apply append] 6 "\n\n(fn &rest ARGS2)"] :url "http://httpbin.org/get?key=value&key2=value2" :response #0) #<killed buffer> "HTTP/1.1 200 OK\nConnection: keep-alive\nServer: gunicorn/19.8.1\nDate: Thu, 05 Jul 2018 11:44:15 GMT\nContent-Type: application/json\nContent-Length: 249\nAccess-Control-Allow-Origin: *\nAccess-Control-Allow-Credentials: true\nVia: 1.1 vegur\n" nil curl nil])
  apply((lambda (data) (message "got data: %s" data)) (:data ((args (key . "value") (key2 . "value2")) (headers (Accept . "*/*") (Accept-Encoding . "deflate, gzip") (Connection . "close") (Host . "httpbin.org") (User-Agent . "curl/7.55.1")) (origin . ...) (url . "http://httpbin.org/get?key=value&key2=value2")) :symbol-status success :error-thrown nil :response [cl-struct-request-response 200 nil ((args (key . "value") (key2 . "value2")) (headers (Accept . "*/*") (Accept-Encoding . "deflate, gzip") (Connection . "close") (Host . "httpbin.org") (User-Agent . "curl/7.55.1")) (origin . ...) (url . "http://httpbin.org/get?key=value&key2=value2")) nil success "http://httpbin.org/get?key=value&key2=value2" nil (:params (("key" . "value") ("key2" . "value2")) :parser json-read :success (lambda (data) (message "got data: %s" data)) :error #[128 "\302\300\303\301\"\"\207" [request-default-error-callback ("http://httpbin.org/get") apply append] 6 "\n\n(fn &rest ARGS2)"] :url "http://httpbin.org/get?key=value&key2=value2" :response #1) #<killed buffer> "HTTP/1.1 200 OK\nConnection: keep-alive\nServer: gunicorn/19.8.1\nDate: Thu, 05 Jul 2018 11:44:15 GMT\nContent-Type: application/json\nContent-Length: 249\nAccess-Control-Allow-Origin: *\nAccess-Control-Allow-Credentials: true\nVia: 1.1 vegur\n" nil curl nil]))
  apply(apply (lambda (data) (message "got data: %s" data)) (:data ((args (key . "value") (key2 . "value2")) (headers (Accept . "*/*") (Accept-Encoding . "deflate, gzip") (Connection . "close") (Host . "httpbin.org") (User-Agent . "curl/7.55.1")) (origin . ...) (url . "http://httpbin.org/get?key=value&key2=value2")) :symbol-status success :error-thrown nil :response [cl-struct-request-response 200 nil ((args (key . "value") (key2 . "value2")) (headers (Accept . "*/*") (Accept-Encoding . "deflate, gzip") (Connection . "close") (Host . "httpbin.org") (User-Agent . "curl/7.55.1")) (origin . ...) (url . "http://httpbin.org/get?key=value&key2=value2")) nil success "http://httpbin.org/get?key=value&key2=value2" nil (:params (("key" . "value") ("key2" . "value2")) :parser json-read :success (lambda (data) (message "got data: %s" data)) :error #[128 "\302\300\303\301\"\"\207" [request-default-error-callback ("http://httpbin.org/get") apply append] 6 "\n\n(fn &rest ARGS2)"] :url "http://httpbin.org/get?key=value&key2=value2" :response #1) #<killed buffer> "HTTP/1.1 200 OK\nConnection: keep-alive\nServer: gunicorn/19.8.1\nDate: Thu, 05 Jul 2018 11:44:15 GMT\nContent-Type: application/json\nContent-Length: 249\nAccess-Control-Allow-Origin: *\nAccess-Control-Allow-Credentials: true\nVia: 1.1 vegur\n" nil curl nil]))
  request--safe-apply((lambda (data) (message "got data: %s" data)) (:data ((args (key . "value") (key2 . "value2")) (headers (Accept . "*/*") (Accept-Encoding . "deflate, gzip") (Connection . "close") (Host . "httpbin.org") (User-Agent . "curl/7.55.1")) (origin . ...) (url . "http://httpbin.org/get?key=value&key2=value2")) :symbol-status success :error-thrown nil :response [cl-struct-request-response 200 nil ((args (key . "value") (key2 . "value2")) (headers (Accept . "*/*") (Accept-Encoding . "deflate, gzip") (Connection . "close") (Host . "httpbin.org") (User-Agent . "curl/7.55.1")) (origin . ...) (url . "http://httpbin.org/get?key=value&key2=value2")) nil success "http://httpbin.org/get?key=value&key2=value2" nil (:params (("key" . "value") ("key2" . "value2")) :parser json-read :success (lambda (data) (message "got data: %s" data)) :error #[128 "\302\300\303\301\"\"\207" [request-default-error-callback ("http://httpbin.org/get") apply append] 6 "\n\n(fn &rest ARGS2)"] :url "http://httpbin.org/get?key=value&key2=value2" :response #1) #<killed buffer> "HTTP/1.1 200 OK\nConnection: keep-alive\nServer: gunicorn/19.8.1\nDate: Thu, 05 Jul 2018 11:44:15 GMT\nContent-Type: application/json\nContent-Length: 249\nAccess-Control-Allow-Origin: *\nAccess-Control-Allow-Credentials: true\nVia: 1.1 vegur\n" nil curl nil]))
  request--callback(#<killed buffer> :params (("key" . "value") ("key2" . "value2")) :parser json-read :success (lambda (data) (message "got data: %s" data)) :error #[128 "\302\300\303\301\"\"\207" [request-default-error-callback ("http://httpbin.org/get") apply append] 6 "\n\n(fn &rest ARGS2)"] :url "http://httpbin.org/get?key=value&key2=value2" :response [cl-struct-request-response 200 nil ((args (key . "value") (key2 . "value2")) (headers (Accept . "*/*") (Accept-Encoding . "deflate, gzip") (Connection . "close") (Host . "httpbin.org") (User-Agent . "curl/7.55.1")) (origin . ...) (url . "http://httpbin.org/get?key=value&key2=value2")) nil success "http://httpbin.org/get?key=value&key2=value2" nil (:params (("key" . "value") ("key2" . "value2")) :parser json-read :success (lambda (data) (message "got data: %s" data)) :error #[128 "\302\300\303\301\"\"\207" [request-default-error-callback ("http://httpbin.org/get") apply append] 6 "\n\n(fn &rest ARGS2)"] :url "http://httpbin.org/get?key=value&key2=value2" :response #0) #<killed buffer> "HTTP/1.1 200 OK\nConnection: keep-alive\nServer: gunicorn/19.8.1\nDate: Thu, 05 Jul 2018 11:44:15 GMT\nContent-Type: application/json\nContent-Length: 249\nAccess-Control-Allow-Origin: *\nAccess-Control-Allow-Credentials: true\nVia: 1.1 vegur\n" nil curl nil])
  apply(request--callback #<killed buffer> (:params (("key" . "value") ("key2" . "value2")) :parser json-read :success (lambda (data) (message "got data: %s" data)) :error #[128 "\302\300\303\301\"\"\207" [request-default-error-callback ("http://httpbin.org/get") apply append] 6 "\n\n(fn &rest ARGS2)"] :url "http://httpbin.org/get?key=value&key2=value2" :response [cl-struct-request-response 200 nil ((args (key . "value") (key2 . "value2")) (headers (Accept . "*/*") (Accept-Encoding . "deflate, gzip") (Connection . "close") (Host . "httpbin.org") (User-Agent . "curl/7.55.1")) (origin . ...) (url . "http://httpbin.org/get?key=value&key2=value2")) nil success "http://httpbin.org/get?key=value&key2=value2" nil #0 #<killed buffer> "HTTP/1.1 200 OK\nConnection: keep-alive\nServer: gunicorn/19.8.1\nDate: Thu, 05 Jul 2018 11:44:15 GMT\nContent-Type: application/json\nContent-Length: 249\nAccess-Control-Allow-Origin: *\nAccess-Control-Allow-Credentials: true\nVia: 1.1 vegur\n" nil curl nil]))
  request--curl-callback(#<process request curl> "finished\n")

Based on the documentation, the callback function should be variadic, which means the following works:

(lambda (&rest args) (message "got data: %s" args))

But then, you will see too much data.

Arguments are passed as key/values pairs. In order to get the data associated with the :data symbol, you would have to do:

(lambda (&rest args)
  (message "got data: %s" (getf args :data)))

The resulting value is an association list, from which you can access the 'args entry like done above (i.e. (assoc-default 'args data)).

But, instead of doing (getf args :data), you can also write:

(lambda (&key data) ...)

The special &key symbol is used to automatically access the value associated with :data in the implicit list of arguments. But, keyword parameters are from Common Lisp, Emacs Lisp does not know how to handle &key out of the box. That's why there is a (function* ...) macro that wraps around the lambda. You then have the choice of using the function* macro or deal with the argument list by yourself, as shown above. It depends on what you need. As suggested by the documentation, if you use &key, you should also use &allow-other-keys.

Second approach

In your second approach, having debug-on-entry set to t indicates that callback is not a known function:

Debugger entered--Lisp error: (void-function callback)

This is due to Emacs Lisp being a Lisp-2, i.e. you cannot call a function given in argument simply by putting it as the first element of a function call. Emacs Lisp understands the syntax as "call the function named callback", not "call the function object bound to variable callback". You'd need to use funcall:

(function*
  (lambda (&key data &allow-other-keys)
    (funcall callback data)))

But with the above, now the error is that callback is an undefined variable. And now, this is due to dynamic scoping. If you evalute the following line and reevaluate the defun, the code works as expected:

(setf lexical-binding t)

Alternatively, you can put the following as the first line of you file:

;; -*- lexical-binding: t -*-