How to use my own versions of Clojure libraries?

Question

Say I made a change to a Clojure library (eg. added a parameter to the request-token in clj-oauth) and want to use that changed library in my project. What's the best way to do this, short of compiling the new library as a JAR and copying that to my project lib?

I want to be able to tweak the library and my project at the same time (preferably in the REPL). If I were doing this in Ruby, I would download and 'require' the gem, then reopen that class in my own project source and add or override the methods as needed.

Answer 1

You can hack directly at the REPL. Suppose you've got incanter on your classpath. Start a REPL. The first thing we need to do is bring the incanter classes into it.

user> (require 'incanter.core)
nil

Now we can see the function incanter.core/matrix?

user> (incanter.core/matrix? 2)
false

We can look at the original source code:

user> (require 'clojure.repl)
nil
user> (clojure.repl/source incanter.core/matrix?)
(defn matrix?
  " Test if obj is 'derived' incanter.Matrix."
  ([obj] (is-matrix obj)))
nil 

Let's go and screw it up:

First change to the incanter.core namespace:

user> (in-ns 'incanter.core)
#<Namespace incanter.core>

Then we can redefine it, using the old source code as a crib:

incanter.core> (defn matrix? [obj] "hello")
#'incanter.core/matrix?

Unit test:

incanter.core> (matrix? 2)
"hello"

Switch back to the user namespace:

incanter.core> (in-ns 'user)
#<Namespace user>

Try it out:

user> (matrix? 2)
; Evaluation aborted.

There is no definition of user/matrix. We redefined it in the incanter.core namespace.

user> (incanter.core/matrix? 2)
"hello"

For experimenting at the repl, it's ok just to change source files and re-compile the single file (C-C C-k in emacs), or if you're in the right namespace, just re-evaluate the definition.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

Now, if we want to make our valuable change permanent and available to other projects, it depends on how everything is set up.

I use maven for dependency management, so it would be a question of modifying the source file, and then re-running the build process for the library to compile the new version and install it into the local maven repository.

With a maven project, that should be as simple as

$ mvn install

A note about version numbers:

If you do make permanent modifications and use dependency management to coordinate the differences, then you should change the version number of your library, from maybe 1.2.0 to 1.2.0-johnshack-SNAPSHOT, or something that is unlikely to collide with the real thing when you want to use an unperverted version in another project. You wouldn't want a modified version finding its way into projects where it isn't welcome.

Then you modify your own project files to make sure that you use the hacked version where you want to, and the next time you start your repl, it should pull in the last hack that you installed.

You will need to reinstall again every time you want your changes to make their way into the repository, but that's actually probably a good thing.

Unfortunately, (and it was at this point that I started to wish that I had chosen a different example) Incanter turns out to be a leiningen project which is split into sub-modules in an ad-hoc scripty sort of way, so we need to figure out how it expects to be installed. The figuring out turned out to be quite hard, although the answer is easy. Leiningen sets my hair on fire.

You can get incanter's source here:

$ git clone http://github.com/liebke/incanter.git

and the relevant source file is:

~/incanter/modules/incanter-core/src/incanter/core.clj

Modify it to break the matrix? function, and then it turns out that what you have to do is:

Change the version numbers in both the top level project.clj, and also in the submodule project.clj.

Then you run lein install in the incanter-core directory, and then again in the top-level directory, and you have to do it in that order. I don't quite understand why.

At the moment all this seems needlessly complicated. I'm (fairly) sure that it will settle down as the tools mature.

Answer 2

If you're using (or wouldn't mind using) cake, check out the subproject dependencies section of the README. I think it might be exactly what you're looking for.