Lisp binary size

Question

As i am Interested in artificial intelligence, I recently decided to give Lisp a try. After compiling a very basic application with the common lisp compiler sbcl I noticed that the resulting binary was very big (around 43MB). I am interested of the reason for that. Is this common issue for (common) lisp and what is the technical background of this behavior?

Answer 1

There are several different architectures in Common Lisp implementations:

• interpreter
• byte code engine (CLISP is an example)
• compilation via the C compiler (ECL is an example)
• native code compiler (SBCL, LispWorks, Clozure CL)

Typically the Interpreter and the byte code engine use the smallest amount of memory. CLISP thus is very small. SBCL OTOH generates relatively large native code.

Second, there are several different ways to create applications:

1. saving an image
2. saving an optimized image
3. compiling to C code

Plus some more like compiling to DLLs.

SBCL basically does 1. It dumps the memory containing data and code and includes the runtime. Thus everything you have in the running system (documentation, source code links, argument lists, symbol names, debug infos, the compiler itself, ...) will be dumped into the image+runtime. Additionally SBCLs generated native code is large, there is potentially a lot of code information in the runtime memory and SBCL includes all its own functionality (including the compiler).

During development one often work(s/ed) with such unoptimized applications or images (with the external runtime) to save time to load code and data. I've used it myself with images which were larger than 100MB.

LispWorks for example does 1 and 2. It has an delivery process where you can selectively remove stuff (like documentation, some functionality like the compiler, source references, ...). This is also using a tree-shaker, which can remove unused functionality.

Optimizing an image could also mean to write it in some compressed way and decompressing it on start up. SBCL allows this for example.

Variant 3 was done in the past, but is currently not in use (other than in some specialized tools and apps). Thinlisp, Stella, CycL, ... are such delivery tools. In the past there was also a commercial vendor for such a tool (but this does not exist no more, IIRC the last owner of it is/was Oracle). Update: actually mocl, a recent Common Lisp application generator for iOS and Android does it. It takes a large subset of Common Lisp and compiles it to small, stand-alone mobile apps. For example on iOS it generates compact C code for Apple's provided C compiler.