Are Smalltalk bytecode optimizations worth the effort?

Question

Consider the following method in the Juicer class:

Juicer >> juiceOf: aString
    | fruit juice |
    fruit := self gather: aString.
    juice := self extractJuiceFrom: fruit.
    ^juice withoutSeeds

It generates the following bytecodes

25 self                     ; 1
26 pushTemp: 0              ; 2
27 send: gather:
28 popIntoTemp: 1           ; 3
29 self                     ; 4
30 pushTemp: 1              ; 5
31 send: extractJuiceFrom:
32 popIntoTemp: 2           ; 6 <-
33 pushTemp: 2              ; 7 <-
34 send: withoutSeeds
35 returnTop

Now note that 32 and 33 cancel out:

25 self                     ; 1
26 pushTemp: 0              ; 2
27 send: gather:
28 popIntoTemp: 1           ; 3 *
29 self                     ; 4 *
30 pushTemp: 1              ; 5 *
31 send: extractJuiceFrom:
32 storeIntoTemp: 2         ; 6 <-
33 send: withoutSeeds
34 returnTop

Next consider 28, 29 and 30. They insert self below the result of gather. The same stack configuration could have been achieved by pushing self before sending the first message:

25 self                     ; 1 <-
26 self                     ; 2
27 pushTemp: 0              ; 3
28 send: gather:
29 popIntoTemp: 1           ; 4 <-
30 pushTemp: 1              ; 5 <-
31 send: extractJuiceFrom:
32 storeIntoTemp: 2         ; 6
33 send: withoutSeeds
34 returnTop

Now cancel out 29 and 30

25 self                     ; 1
26 self                     ; 2
27 pushTemp: 0              ; 3
28 send: gather:
29 storeIntoTemp: 1         ; 4 <-
30 send: extractJuiceFrom:
31 storeIntoTemp: 2         ; 5
32 send: withoutSeeds
33 returnTop

Temporaries 1 and 2 are written but not read. So, except when debugging, they could be skipped leading to:

25 self                     ; 1
26 self                     ; 2
27 pushTemp: 0              ; 3
28 send: gather:
29 send: extractJuiceFrom:
30 send: withoutSeeds
31 returnTop

This last version, which saves 4 out 7 stack operations, corresponds to the less expressive and clear source:

Juicer >> juiceOf: aString
    ^(self extractJuiceFrom: (self gather: aString)) withoutSeeds

Note also that there are other possible optimizations that Pharo (I haven't checked Squeak) does not implement (e.g., jump chaining.) These optimizations would encourage the Smalltalk programmer to better express their intentions without having to pay the cost of additional computations.

My question is whether these improvements are an illusion or not. Concretely, are bytecode optimizations absent from Pharo/Squeak because they are known to have little relevance, or are they regarded as beneficial but haven't been addressed yet?

EDIT

An interesting advantage of using a register+stack architecture [cf. A Smalltalk Virtual Machine Architectural Model by Allen Wirfs-Brock and Pat Caudill] is that the additional space provided by registers makes it easier the manipulation of bytecodes for the sake of optimization. Of course, even though these kinds of optimizations are not as relevant as method inlining or polymorphic inline caches, as pointed out in the answer below, they shouldn't be disregarded, especially when combined with others implemented by the JIT compiler. Another interesting topic to analyze is whether destructive optimization (i.e., the one that requires de-optimization for supporting the debugger) is actually necessary or enough performance gains can be attained by non-destructive techniques.

Answer 1

The main annoyance when you start playing with such optimizations is debugger interface.

Historically and still currently in Squeak, the debugger is simulating the bytecode level and needs to map the bytecodes to corresponding Smalltalk instruction.

So I think the gain was too low for justifying complexification, or even worse degradation of debugging facility.

Pharo wants to change the debugger to operate at a higher level (Abstract Syntax Tree), but I don't know how they will end up at bytecode which is all the VM knows of.

IMO, this kind of optimization might better be implemented in the JIT compiler which transforms bytecode to machine native code.

EDIT

The greatest gains are in eliminating the sends themselves (by inlining) because they are much more expensive (x10) than the stack operations - there are 10 times more bytecodes executed per second than sends when you test 1 tinyBenchmarks (COG VM).

Interestingly, such optimizations could take place in the Smalltalk image, but only on hotspot detected by VM, as in the SISTA effort. See for example https://clementbera.wordpress.com/2014/01/22/the-sista-chronicles-iii-an-intermediate-representation-for-optimizations/

So, in the light of SISTA, the answer is rather: interesting, not yet addressed, but actively studied (and work in progress)!

All the machinery for de-optimizing when the method has to be debugged still is one of the difficult points as I understand it.

Answer 2

I think that a broader question is worth answering: are bytecodes worth the effort? Bytecodes were thought as a compact and portable representation of code that is close the target machine. As such, they are easy to interpret, but slow to execute.

Bytecodes do not excel in any of these games, and that usually makes them not the best choice if you want to either write an interpreter or a fast VM. On one hand, AST nodes far easier to interpret (only a few node types vs lots of different bytecodes). On the other hand, with the advent of JIT compilers, it became clear that running native code instead is not only possible but also much faster.

If you look at the most efficient VM implementations of JavaScript (which can be considered the most modern compilers of today) and also Java (HotSpot, Graal), you'll see they all use a tiered compilation scheme. Methods are initially interpreted from the AST, and only jitted when they become a hot spot.

At the hardest tiers of compilation there are no bytecodes. The key component in a compiler is its intermediate representation, and bytecodes do not fulfill the required properties. The most optimizable IRs are much more fine grained: they are in SSA form, and allow specific representation of registers and memory. This allows for much better code analysis and optimization.

Then again, if you are interested in portable code, there isn't anything more portable than the AST. Besides, it's easier and more practical to implement AST-based debuggers and profilers than bytecode-based ones. The only remaining problem is compactness, but in any case you can implement something like ast-codes (coded asts, similar to bytecodes but representing the tree)

On the other hand, if you want full speed, then you'll go for a JIT with a good IR and no bytecodes. I think that bytecodes don't fill many gaps in today VMs, but still remain mostly for backwards compatibility (also there are many examples of hardware archiqutectures that directly execute Java bytecodes).

There are also some cool experiments with the Cog VM related with bytecodes. But from what I understand they transform the bytecode into another IR for optimizing, then they convert back to bytecodes. I'm not sure if there's a technical gain in the last conversion besides reusing the original JIT architecture, or if there actually is any optimization at the bytecode level.