How to download a file and unzip it from memory in clojure?

Question

I'm making a GET request using clj-http and the response is a zip file. The contents of this zip is always one CSV file. I want to save the CSV file to disk, but I can't figure out how.

If I have the file on disk, (fs/unzip filename destination) from the Raynes/fs library works great, but I can't figure out how I can coerce the response from clj-http into something this can read. If possible, I'd like to unzip the file directly without

The closest I've gotten (if this is even close) gets me to a BufferedInputStream, but I'm lost from there.

(require '[clj-http.client :as client])
(require '[clojure.java.io :as io])

(->
  (client/get "http://localhost:8000/blah.zip" {:as :byte-array})
  (:body)
  (io/input-stream))

Answer 1

(require '[clj-http.client :as httpc])
(import '[java.io File])


(defn download-unzip [url dir]
  (let [saveDir (File. dir)]
    (with-open [stream (-> (httpc/get url {:as :stream})
                           (:body)
                           (java.util.zip.ZipInputStream.))]
      (loop [entry (.getNextEntry stream)]
        (if entry
          (let [savePath (str dir File/separatorChar (.getName entry))
                saveFile (File. savePath)]
            (if (.isDirectory entry)
              (if-not (.exists saveFile)
                (.mkdirs saveFile))
              (let [parentDir (File. (.substring savePath 0 (.lastIndexOf savePath (int File/separatorChar))))]
                (if-not (.exists parentDir) (.mkdirs parentDir))
                (clojure.java.io/copy stream saveFile)))
            (recur (.getNextEntry stream))))))))

Answer 2

You can use the pure java java.util.zip.ZipInputStream or java.util.zip.GZIPInputStream. Depends how the content is zipped. This is the code that saves your file using java.util.zip.GZIPInputStream :

(->
  (client/get "http://localhost:8000/blah.zip" {:as :byte-array})
  (:body)
  (io/input-stream)
  (java.util.zip.GZIPInputStream.)
  (clojure.java.io/copy (clojure.java.io/file "/path/to/output/file")))

Using java.util.zip.ZipInputStream makes it only a bit more complicated :

(let [stream (->
                (client/get "http://localhost:8000/blah.zip" {:as :byte-array})
                (:body)
                (io/input-stream)
                (java.util.zip.ZipInputStream.))]
      (.getNextEntry stream)
      (clojure.java.io/copy stream (clojure.java.io/file "/path/to/output/file")))