Best way to manage a pool of resources in Clojure

Question

I have a a web service endpoint that uses a mutable resource from a Java library. That web service endpoint can receive multiple queries at the same time. (the endpoint is implemented using Ring/Compojure). Creating these resources is costly, so re-creating them for every web service call is really inefficient.

What I want to do is to create a pool of that resource that I populate when the web service starts. Then each time the endpoint is called, it takes a resource from the pool, use it for its processing, and then push it back into the pool and wait for the next call to happen.

I am wondering what would be the best way to do that in Clojure? Is there a "pool" Clojure library that could help me with that?

I naively tried to implement that using an vector in an atom where each item of the vector is that resource. However, it quickly learned that it could not really work that way.

Answer 1

This is based on Timothy Pratley's idea of using refs:

(def pool (ref ['a 'b 'c]))

(defn take' [pool]
  (dosync
    (let [[h & t] @pool]
      (ref-set pool (vec t))
      h)))

(defn put [pool x]
  (dosync
    (alter pool conj x)
    nil))

(take' pool)   ;; => 'a
(put pool 'a)  ;; => nil
(take' pool)   ;; => 'a
(take' pool)   ;; => 'b
(take' pool)   ;; => 'c

Maybe not the best way to attack this. But I like the simplicity of it.

Answer 2

To implement a pool you need to address 2 concerns:

1. Concurrency. Use locking https://clojuredocs.org/clojure.core/locking or a ref instead of an atom. Requests can be simultaneous, so you need to be careful that it is impossible for two consumers to receive the same resource.

2. Releasing resources. Consider using a pattern like (with-open ...) i.e. a macro that expands to a try-finally where the resource is released back to the open pool when you leave the block scope.

You might want to designate an 'error' status as well as 'available' or 'in use', where the resource may need to be released and recreated.

Answer 3

Have a look to this kul/pool. It uses Apache Commons Pool. I hope it's useful.