Thread safe file writing in Clojure

Question

I'm playing around with thread safe file writing and I can't figure out why this code isn't logging to the file (based on the agents section of: http://blakesmith.me/2012/05/25/understanding-clojure-concurrency-part-2.html)

The code is hitting all the right functions, but the file remains empty. I'm thinking it's because of the file close not happening in some fashion, but even deref-ing the futures to force the close to come last as a test doesn't seem to help.

(defn write-out [out msg]
  (.write out msg)
  out)

(defn log [logger msg]
  (send logger write-out msg))

(defn close [logger]
  (send logger #(.close %)))

(defn go []
  (let [ofile (agent (clojure.java.io/writer "/tmp/log.test.txt" :append true))]
    (dotimes [x 10]
      (future (log ofile (str "Log A " x "\n"))))
    (close ofile)
    (shutdown-agents)))

Minor bonus question: The post linked never really explains why I needed to return the file writer pointer at the end of write-out. I know you need it, but I'm not sure why.

Answer 1

Perhaps you need to call (.flush out) ?

Your function needs to return out so that the agent has that value for the next defn invocation it receives. The writing to the file is itself a side-effect.

The typical example use case for an agent is a counter that is "incremented" (well, replaced by a new incremented value) by multiple threads. However, in this case you are not incrementing a counter, instead you are writing to an out (OutputStream instance). So the out has to be returned so it can be passed to the next agent task.

Useful blog post I found: http://lethain.com/a-couple-of-clojure-agent-examples/

Answer 2

Note that even if you flush the file as suggested in another answer, this code's behavior is non-deterministic. For example, perhaps all 10 spawned futures take a while before being scheduled for the first time, so that before they get started the -main function has already queued a (close ofile) action on the logger: in that case, all attempts to write will fail. In fact, perhaps even shutdown-agents has been called before any of the futures start: then none of their send actions will have any effect at all!