Faster way to compile Factor Programs

Question

I really love the Factor language. But I find that compiling programs written in it is incredibly slow, and thus it's not feasible to create real projects with Factor.

For example, compiling the sample Calculator WebApp takes about 5 minutes on my laptop (i3 processor, 2GB RAM, running Fedora 15).

I've searched around but couldn't find a faster way to compile Factor programs than using the interpreter (the main factor binary executable).

It becomes ridiculous when you attempt to only use the interpreter for each run and not "deploy" your program to a native binary file (which doesn't even work on many programs). It means that every time I want to run the Calculator, for example, I have to wait a 5 minute cold-start duration.

I'd like to know whether this is a common issue, and whether there's a good way to tackle it.

Answer 1

I admit that before today, I had never heard of Factor. I took the opportunity to play with it. It is looking nice (reminds me of squeak-vm and lisp at the same time). I'll cut the smalltalk (pun very much intended) and jump to your question.

Analysis

It appears that the way Factor works, results in loading vocabularies being slow.

I compiled Factor on my 64 bit quadcore linux system (from git revision 60b1115452, Thu Oct 6). Putting everything on tmpfs the build dir takes 641Mb, of which 2x114Mb is in the factor.image and its backup (factor.image.fresh).

When strace-ing the calculator app loading, there is a huge list of factor files being loaded:

• 3175 factor files are touched.
• compilation of these takes roughly 30 seconds on my box
• the memory usage maxes out on just under 500Mb (virtual) and 300Mb reserved set

I'm strongly suspecting your box is low on memory, and might be getting very swappy
_{This would definitely explain compilation taking 5 minutes}

Can you confirm whether this is the case (likely if you are running some kind of shared host or VPS appliance). If you run a virtual machine, consider increasing the available system memory.

---

Saving Heap Images (snapshots)

I already mentioned the factor.image file (114Mb) before. It contains a 'precompiled' (bootstrapped, actually) heap image for the Factor VM. All operations in the VM (working on the UI listener or compiling factor files) affects the heap image.

To avoid having to recompile your source files time and time again, you can save the end-result into a custom heap image:

http://docs.factorcode.org/content/article-images.html

Images

To start Factor with a custom image, use the -i=image command line switch; see Command line switches for the VM.

One reason to save a custom image is if you find yourself loading the same libraries in every Factor session; some libraries take a little while to compile, so saving an image with those libraries loaded can save you a lot of time.

For example, to save an image with the web framework loaded,

USE: furnace
save

New images can be created from scratch: Bootstrapping new images

Deploying applications

Saving heap images results in files that will (typically) be bigger than the original bootstrap image.

The Application deployment tool creates stripped-down images containing just enough code to run a single application

The stand-alone application deployment tool, implemented in the tools.deploy vocabulary, compiles a vocabulary down to a native executable which runs the vocabulary's MAIN: hook. Deployed executables do not depend on Factor being installed, and do not expose any source code, and thus are suitable for delivering commercial end-user applications.

Most of the time, the words in the tools.deploy vocabulary should not be used directly; instead, use Application deployment UI tool.

You must explicitly specify major subsystems which are required, as well as the level of reflection support needed. This is done by modifying the deployment configuration prior to deployment.

Concluding

I expect you'll benefit from (in order of quickest win):

• increasing available RAM _{^{(only quick in virtual environments...)}}
• saving a heap image with

USE: db.sqlite
USE: furnace.alloy
USE: namespaces
USE: http.server
save

This step brought the compilation on my system **down from ~`30s` to `0.835s`**

• deploying the calculator webapp to a stripped heap image (refer to the source for deployment hints)

In short, thanks for bringing Factor to my attention, and I hope my findings will be of any help, Cheers