How is the persistent JVMs feature implemented in cake?

Question

I'm trying to understand how cake implements its multiple JVM approach. At a high level, I thought that cake was working similar to nailgun, where there is a single JVM instance (one JVM process), and new "JVMs" for different projects were actually just clojure/jars evaluated in a new classloader (along with different jar dependencies), which in my eyes is not a new JVM instance. From What's the difference between Cake and Leiningen? however, there is an implication that there are multiple JVMs (one for cake, and * for the projects), not just a single JVM instance.

If there are new JVM instances created, where does the speedup come from? With my understanding, I would reason that starting a new JVM implies creating a new JVM process which incurs the same startup overhead as usual.

If there are not, how are native dependencies added on? From what I understand, the JVM only knows about native dependencies from command line arguments passed before runtime. The only way I know how to circumvent this is with a Sun/Oracle JVM implementation specific hack listed below.

 (let [clazz java.lang.ClassLoader
      field (.getDeclaredField clazz "sys_paths")] 
   (.setAccessible field true)
   (.set field clazz nil)
   (System/setProperty "java.library.path" (apply str (interpose ";" native-paths))))

Answer 1

Cake has a Ruby script that starts up and manages the JVMs. Ruby doesn't have the JVM overhead, so the Ruby script could create the JVMs and then when you execute commands, the Ruby script would delegate those commands to the JVMs.

The reason two JVMs was necessary was so that cake's dependencies (the cake JVM) were separate from the project's dependencies (the bake JVM). Some commands like cake repl run in the bake JVM to take advantage of the project's classpath.

However, in the most recent version, there is only a single JVM per project. This is possible using different classloaders in the same JVM. The relevant library used is classlojure.

Even with the two JVM versions, the JVMs were persistent, meaning they were only spawned once and then restarted only when absolutely necessary, like in the case of a changed classpath (when you add a new dependency or something similar). I'm not sure why you'd think this would mean incurring the JVM overhead every time a command is executed. The idea is that a lot of commands happen instantly rather than every command starting a JVM.

Answer 2

Raynes is correct. As of cake 0.6.0, there is one JVM per project. Cake runs in the main classloader and uses classlojure to load the project in a separate classloader and reload it when the classpath changes. We have discussed a global ~/.cake/config option to share a single JVM among all projects. It shouldn't be too hard to add this using classlojure. The main issue with this approach is how to keep cake task plugins separate. Perhaps the global cake project could run in the main classloader and each project could get two classloaders (one for cake and one for the project).

As for native dependencies, classlojure doesn't support adding them after the JVM starts up. A patch to add this functionality would be welcome as long as the native library path is local to a specific classloader and isn't shared among all classloaders in the same JVM.