Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Will running everything from RAM disk speed up scala compile time?

Scenario:

The machine I use for development have 32Gb of DDR3 RAM, i7 3770, SSD. The project is large, Scala compiles fast most of the time during incremental compilation but sometimes a single change leads to recompilation of hundreds of files, it then take some time to compile all and some good time for jrebel to reload all changed files.

Question:

Will putting everything on a RAMFS (Mac) make compile and jrebel reload significantly faster?

My plan was to put everything directly related to the project in a RAMFS partition ( .ivy, project source, .sbt, maybe even copy JDK. etc). I would create a script to do all that in the boot or manually, that won't be a problem. Also, I would setup file sync tasks, so, losing a change won't be a concern in case of a OS failure.

Updates:

  1. log says around 400 among java and scala sources are compiled after a clean.
  2. after changing a file in a core module, it recompiles 130 files in 50s.
  3. jrebel takes 72s to reload after #1 and 50s after #2
  4. adding -Drebel.check_class_hash=true made jrebel reload instantaneous after #2.

I am quite happy with these results, but still interested on how to make scala compilation even faster, since cpu usage gets at most 70% for just about 5 seconds in compilation process that takes 170s, overall cpu usage during the compilation is 20%.

UPDATE:

After putting JVM, source, .ivy2 and .sbt folders on RAMDISK, I noticed a small improvement on compile time only: from 132s to 122s ( after a clean). So, not worth the trouble.

NOTE:

That is excluding the dependency resolution, since I using this approach to avoid losing dependency resolution after a clean.

like image 504
Johnny Everson Avatar asked Nov 24 '12 18:11

Johnny Everson


1 Answers

I have no idea what speedup you can expect with a Mac, but I have seen speedups on Linux compiling the Scala compiler itself that are encouraging enough to try. My report (warning : quite Linux-specific) is there.

like image 192
Francois G Avatar answered Sep 18 '22 14:09

Francois G