Debugging interpreter in VM when changing vm primitives

Question

Context

As a university project we want to change the the pharo vm to use an object-table and see what happens.

We use a pharo-vm clone from github and VMMaker. Building the VM works fine.

To get started we added a primitive that returns an incremented Integer:

InterpreterPrimitives>>primitiveIntegerIncrement
    "increments an integer"
    self pushInteger: self popInteger + 1 .

and modified StackInterpreter class>>initializePrimitiveTable accordingly

MaxPrimitiveIndex := 576.
"... and so on ..."
    (575 primitiveFail)
    (576 primitiveIntegerIncrement))

And it works.

Problem

When we make changes to the VM we want to test-run already in the SmalltalkImage so we do not need to compile and see it did not work.

Something like:

StackInterpreter test: '1 inc'

And then I can debug if the primitive is wrong or an error occurs. Of course there needs to be done much more but how can I achieve this?

What we tried

1. category VMMaker-InterpreterSimulation class StackInterpreterSimulator. Trying the code in the comments

   DoIt
       ^ (StackInterpreterSimulator new openOn: Smalltalk imageName) test 
   

   errors:

       displayForm := 'Display has not yet been installed' asDisplayText form.
   

   the ByteString does not understand asDisplayText

2. (CogVMSimulator new openOn: Smalltalk imageName) test 
   (InterpreterSimulator new openOn: Smalltalk imageName) test
   

   error:

       PrimitiveFailed: primitive #basicNew: in Array class failed
   

I also found this screen cast but it only debugs the VM from outside using gbd: http://vimeo.com/22485382#

Our project is hosted here: http://smalltalkhub.com/#!/~kirstin/PharoObjectTable

Current Status

We started implementing an object table. The lookup of attributes can go throught the object table. Full object table support and no usage of direct pointes is very tricky since pointers are expected everywhere. So we use pointers into the object table to identify when a lookup should go through the OT. We also found all object creation primitives and add new objects to the table.

Answer 1

How long is your project and how many people are you ? To me what you try to do is quite some work. Do you have good knowledge about low level behavior ?

To answer your question, the main problem here is that the cog simulator is not maintained in the pharo vm fork. This is because no one in the pharo crew use the simulator. We only use external debugging from gdb. In fact the pharo folks work mostly on VM plugins, the core of the VM is mainly maintained and developed by Eliot Miranda which works on Squeak. Therefore we report to him when there's a bug in the VM core.

For your project you would have to split it in at least 2 steps:

step 1: make the object table work with the stack VM

step 2: make the JIT work with your object table

Note that for step 2 I would recommend not to change the way an object access its header, therefore having a VW-like object table where you have the fixed size header on the one in the the object table, and the fields of the objects (and maybe header extensions) in the heap.

So use the StackVMSimulator and build the StackVM first. When everything will work (including context), you can think about hacking the JIT. Recently Guillermo Polito ported the Stack VM to the build process (see PharoSVMBuilder instead of PharoVMBuilder), a guy reported problems with this builder but you could hack it a bit to make it work.

Now to make the simulator work on Pharo 2.0 (which is the Pharo version of the generator image you have), you have to open the monticello browser and merge from Eliot's branch the Cog package (repo MCHttpRepository location: 'http: //source. squeak. org/VMMaker'), but not the latest Cog, the one at around the same date as the current VMMaker package of pharo-vm because the latest Cog and VMMaker of Eliot's branch are not stable.

The alternative being to start from Eliot's build image and merge things from the pharo branch. Here are infos about how to build the squeak development image (http://www.mirandabanda.org/cogblog/build-image/).

Then Eliot gave me this script once:

| cos |
cos := CogVMSimulator newWithOptions: #(Cogit SistaStackToRegisterMappingCogit).
cos desiredNumStackPages: 8.
cos openOn: 'my/favourite.image'.
cos openAsMorph; toggleTranscript; halt; run

You don't need the SistaStackToRegisterMappingCogit option. I guess some similar script with the StackVMSimulator should work.

Lastly there are some documentation about the simulator but it is only for the CogSimulator (these documentations expects you already knows how the StackSimulator works, and just give you hints about how to use it with the JIT): http://www.mirandabanda.org/cogblog/2008/12/12/simulate-out-of-the-bochs/ and in one of the video named "Cog VM (part x)", x being from 1 to 6, Eliot shows how he uses the simulator to disassemble x86, print the stack and inspect the heap.

Another tip, ask your questions on the pharo mailing list (pharo users or pharo dev), because here no one may notice your question (fortunately someone pointed me out your question this time).

And tell on the pharo mailing list if you managed to run the simulator in Pharo 2.0, some people (as me) are very interested in it. I was planning to do it at some point.

Good luck ! Nice project anyway.