I've been reading code and documentation to try to understand how class reloading works in clojure. According to many websites, such as http://tutorials.jenkov.com/java-reflection/dynamic-class-loading-reloading.html , whenever you load a class essentially you obtain the bytecode (via any data mechanism), convert the bytecode into an instance of class Class (via defineClass), and then resolve (link) the class via resolveClass. (Does defineClass implicitly call resolveClass?). Any given classloader is only allowed to link a class once. If it attempts to link an existing class, it does nothing. This creates a problem since you cannot link a newly instantiated class, therefore you have to create a new instance of a classloader everytime you reload a class.
Going back to clojure, I tried examining the paths to load classes.
In clojure, you can define new classes in multiple ways depending on what you want:
Anonymous Class: reify proxy
Named Class: deftype defrecord (which uses deftype under the hood) gen-class
Ultimately, those codes point to clojure/src/jvm/clojure/lang/DynamicClassLoader.java
where DynamicClassLoader/defineClass creates an instance with super's defineClass and then caches the instance. When you want to retrieve the class, clojure load with a call to forName which calls the classloader and DynamicClassLoader/findClass, which first looks in the cache before delegating to the super class (which is contrary to the way most normal classloaders work, where they delegate first, than try it themselves.) The important point of confusion is the following: forName is documented to link the class before it returns but this would imply you can not reload a class from the existing DynamicClassLoader and instead need to create a new DynamicClassLoader, however I don't see this in the code. I understand that proxy and reify define anonymous classes, so their names are different thus can be treated as if its a different class. However, for the named classes, this breaks down. In real clojure code, you can have references to the old version of the classes and references to the new version of the classes simultaneously, but attempts to create new class instances will be of the new version.
Please explain how clojure is able to reload classes without creating new instances of DynamicClassLoader, if I can understand the mechanism to reload classes, I would like to extend this reloading functionality to java's .class files I may create using javac.
Notes: This question refers to class RELOADING, not simply dynamic loading. Reloading means that I have already interned a class but want to intern a new updated version of that instance.
I want to reiterate, that its not clear how clojure is able to reload deftype defined classes. Calling deftype eventually leads to a call to clojure.lang.DynamicClassLoader/defineClass. Doing this again leads another call to defineClass, but doing this manually results in a Linkage Error. What is happening underneath here that allows clojure to do this with deftypes?
The Clojure compilation model preserves as much as possible the dynamic nature of Clojure, in spite of the code-reloading limitations of Java. Source and classfile pathing follows Java classpath conventions.
As of Clojure 1.8, the Clojure core library itself is compiled with direct linking. Classes generated by Clojure are highly dynamic. In particular, note that no method bodies or other implementation details are specified in gen-class - it specifies only a signature, and the class that it generates is only a stub.
The loader class is generated for each file referenced when a namespace is compiled, when its loader .class file is older than its source. A stand-alone gen-class facility is provided to create named classes for direct use as Java classes, with facilities for:
This indirection via the var is one of the ways that Clojure provides a dynamic runtime environment. However, it has long been observed that the majority of function invocations in a production environment are never redefined in this way, incurring unnecessary redirection.
Not all of these language features use the same technique.
The proxy
macro generates a class name based exclusively on the class and list of interfaces being inherited. The implementation of each method in this class delegates to a Clojure fn stored in the object instance. This allows Clojure to use the very same proxy class every time the same list of interfaces is inherited, whether the body of the macro is the same or not. No actual class reloading takes place.
For reify
, the method bodies are compiled directly into the class, so the trick proxy
uses won't work. Instead, a new class is generated when the form is compiled, so if you change the body of the form and reload it, you get a whole new class (with a new generated name). So again, no actual class reloading takes place.
With gen-class
you specify a name for the generated class, so neither of the techniques used for proxy
or reify
will work. A gen-class
macro contains only a sort of spec for a class, but none of the method bodies. The generated class, somewhat like proxy
, defers to Clojure functions for the method bodies. But because a name is tied to the spec, unlike proxy
it would not work to change the body of a gen-class
and reload it, so gen-class
is only available when compiling ahead-of-time (AOT compilation) and no reloading is allowed without restarting the JVM.
This is where real dynamic class reloading happens. I'm not deeply familiar with the internals of the JVM, but a little work with a debugger and the REPL makes one point clear: every time a class name needs to be resolved, such as when compiling code that uses the class or when the Class class's forName
method is called, Clojure's DynamicClassLoader/findClass
method is used. As you note this looks up the class name in in the DynamicClassLoader's cache, and this can be set to point to a new class by running deftype
again.
Note the caveats in the tutorial you mentioned about the reloaded class being a different class, despite having the same name, still apply to Clojure classes:
(deftype T [a b]) ; define an original class named T (def x (T. 1 2)) ; create an instance of the original class (deftype T [a b]) ; load a new class by the same name (cast T x) ; cast the old instance to the new class -- fails ; ClassCastException java.lang.Class.cast (Class.java:2990)
Each top-level form in a Clojure program gets a fresh DynamicClassLoader which is used for any new classes defined within that form. This will include not only classes defined via deftype
and defrecord
but also reify
and fn
. This means that the classloader for x
above is different than the new T
. Note the numbers after the @
s are different -- each gets its own classloader:
(.getClassLoader (class x)) ;=> #<DynamicClassLoader clojure.lang.DynamicClassLoader@337b4703> (.getClassLoader (class (T. 3 4))) ;=> #<DynamicClassLoader clojure.lang.DynamicClassLoader@451c0d60>
But as long as we don't define a new T
class, new instances will have the same class with the same classloader. Note the number after the @
here is the same as the second one above:
(.getClassLoader (class (T. 4 5))) ;=> #<DynamicClassLoader clojure.lang.DynamicClassLoader@451c0d60>
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With