Getting leiningen to cache packages

Question

In a clojurescript project I'd like leiningen to be less reliant on the internet connection during our CI builds. I was hoping to get it to cache packages on a network disc (using the :local-repo setting to create a "shared cache") and then add it as a repository in such way that it fetches from there first and only from clojars and other external sites when it can't find it in the "shared cache".

I read this, removed my ~/.m2 folder, and added the following to my project.clj:

:profiles {:local-cache
           {:local-repo "/shared/disc/clojars-cache"
            :repositories {"local" {:uri "file:///shared/disc/clojars-cache"
                                    :releases {:checksum :ignore}}}}}

The initial build with lein with-profile +local-cache cljsbuild does indeed populate the cache, but

1. My ~/.m2/repository folder is recreated and filled with stuff, though it seems to only be the clojure stuff needed by leiningen, and
2. after removing ~/.m2 subsequent rebuilds don't seem to use the local repository at all but instead download from clojars anyway.

Clearly I'm missing something... or maybe I'm going about this in the completely wrong way.

In short, how can I get leiningen to

1. create a cache of packages on a network disc, and
2. get it to prefer this cache as source of packages (over external sources like clojars)?

Answer 1

Leiningen already prefers to go to ~/.m2 by default. It will only go to Clojars if it doesn't already have a copy of the JAR that is requested stored locally in its ~/.m2. The exception to this rule is if you are specifying SNAPSHOT versions, where it will go out to the network to check if the SNAPSHOT version it has is the latest once a day (by default).

You can set the :offline? key as true if you don't want Leiningen to go to the network at all.

Answering your questions:

1. How can I get Leiningen to create a cache of packages on a network disk?

Leiningen already creates a cache of packages in ~/.m2. You could symlink that directory to your network disk, or use :local-repo as you are now, although it sounds like :local-repo isn't working for you?

2. How can I get Leiningen to prefer this cache over external sources?

Leiningen already does this. It sounds like :local-repo either isn't working, hasn't been configured correctly, or that directory isn't writable by Leiningen?

---

Stepping back to look at the wider problem, you're wanting to prevent unnecessary network traffic in your CI builds. Leiningen already caches every dependency by default. You haven't said which CI tool you're using, but they should all have the ability to cache the ~/.m2 folder between runs. Depending on the tool, you'll have to download your deps either once per project, or once per machine. I'd recommend sticking with that, rather than trying to share deps over the network, as that could lead to hard to debug test failures.

If that's not going to work for you, can you provide more details about your setup, and why you'd like Leiningen to be less reliant on the network in your CI builds?

---

UPDATE: After seeing that Gitlab CI is being used, it looks like you need to add a caching config?

cache:
  paths:
  - ~/.m2