How should carmine's wcar macro be used?

Question

I'm confused by how calls with carmine should be done. I found the wcar macro described in carmine's docs:

(defmacro wcar [& body] `(car/with-conn pool spec-server1 ~@body))

Do I really have to call wcar every time I want to talk to redis in addition to the redis command? Or can I just call it once at the beginning? If so how?

This is what some code with tavisrudd's redis library looked like (from my toy url shortener project's testsuite):

(deftest test_shorten_doesnt_exist_create_new_next 
  (redis/with-server test-server
    (redis/set "url_counter" 51)
    (shorten test-url)
    (is (= "1g" (redis/get (str "urls|" test-url))))
    (is (= test-url (redis/get "shorts|1g")))))

And now I can only get it working with carmine by writing it like this:

(deftest test_shorten_doesnt_exist_create_new_next
  (wcar (car/set "url_counter" 51))
    (shorten test-url)
    (is (= "1g" (wcar (car/get (str "urls|" test-url)))))
    (is (= test-url (wcar (car/get "shorts|1g")))))

So what's the right way of using it and what underlying concept am I not getting?

Answer 1

Dan's explanation is correct.

Carmine uses response pipelining by default, whereas redis-clojure requires you to ask for pipelining when you want it (using the pipeline macro).

The main reason you'd want pipelining is for performance. Redis is so fast that the bottleneck in using it is often the time it takes for the request+response to travel over the network.

Clojure destructuring provides a convenient way of dealing with the pipelined response, but it does require writing your code differently to redis-clojure. The way I'd write your example is something like this (I'm assuming your shorten fn has side effects and needs to be called before the GETs):

(deftest test_shorten_doesnt_exist_create_new_next
  (wcar (car/set "url_counter" 51))
  (shorten test-url)
  (let [[response1 response2] (wcar (car/get (str "urls|" test-url))
                                    (car/get "shorts|1g"))]
    (is (= "1g" response1))
    (is (= test-url response2))))

So we're sending the first (SET) request to Redis and waiting for the reply (I'm not certain if that's actually necessary here). We then send the next two (GET) requests at once, allow Redis to queue the responses, then receive them all back at once as a vector that we'll destructure.

At first this may seem like unnecessary extra effort because it requires you to be explicit about when to receive queued responses, but it brings a lot of benefits including performance, clarity, and composable commands.

I'd check out Touchstone on GitHub if you're looking for an example of what I'd consider idiomatic Carmine use (just search for the wcar calls). (Sorry, SO is preventing me from including another link).

Otherwise just pop me an email (or file a GitHub issue) if you have any other questions.

Answer 2

Don't worry, you're using it the correct way already.

The Redis request functions (such as the get and set that you're using above) are all routed through another function send-request! that relies on a dynamically bound *context* to provide the connection. Attempting to call any of these Redis commands without that context will fail with a "no context" error. The with-conn macro (used in wcar) sets that context and provides the connection.

The wcar macro is then just a thin wrapper around with-conn making the assumption that you will be using the same connection details for all Redis requests.

So far this is all very similar to how Tavis Rudd's redis-clojure works.

So, the question now is why does Carmine need multiple wcar's when Redis-Clojure only required a single with-server?

And the answer is, it doesn't. Apart from sometimes, when it does. Carmine's with-conn uses Redis's "Pipelining" to send multiple requests with the same connection and then package the responses together in a vector. The example from the README shows this in action.

(wcar (car/ping)
      (car/set "foo" "bar")
      (car/get "foo"))
=> ["PONG" "OK" "bar"]

Here you will see that ping, set and get are only concerned with sending the request, leaving the receiving of response up to wcar. This precludes asserts (or any result access) from inside of wcar and leads to the separation of requests and multiple wcar calls that you have.