How to dynamically generate mathematica code?

Question

I want to make a mini programming language in mma. From a textfile to Module(s) in a Package. Ideally I should be able to generate the package and modules from within Mathematica by functions in another package.

Question: Is this possible? I am looking for a reference or an example to get this started.

EDIT: For example:

Imagine a memory bank with n integer typed registers.

The instructions are:

1 Z(n)

2 C(m,n)

3 J(m,n,q)

4 S(n)

Each line has an address. First line 1, second 2, etc. Z(n) store 0 in register n. C(m,n) store value of register m in register n. J(m,n,q) if value register m equals value of register n then jump to line with address q. S(n) add 1 to value in register n.

Then given two working programs P and Q I want to generate the concatenated program P+Q.

Then given two working programs P and Q I want to generate the substitution Q after P.

Finally I want to start experimenting with recursion... the purpose of this 'mini-project'.

Answer 1

Your question has several parts. First, if you want to use some non-mma syntax for your language, you need to make a parser from your language to mma expression (AST of your code). I will leave this one out (since this is a separate topic) and assume that you are willing to use mma syntax or have means to transfer your program to some mma expression.

Regarding the mma code generation, Mathematica is very well suited for it since it embraces the code-is-data paradigm. The hardest part here is the evaluation control - we want to make sure that none of our generated code pieces evaluates during the code-generation process. The standard techniques of evaluation control can be successfully used for that, but this will generally make things rather complicated. I will illustrate one technique of mma code generation, which is not the best/most powerful one, but the easiest.

Consider a toy language created by these definitions:

SetAttributes[testSet, HoldFirst];
SetAttributes[testIf, HoldRest];
SetAttributes[testVar, HoldAll];
SetAttributes[module, HoldAll];
SetAttributes[{package, inContext}, HoldRest];
testPlus[x_, y_] := Plus[x, y];
testTimes[x_, y_] := Times[x, y];
testDivide[x_, y_] := If[y == 0, Inf, Times[x, Power[y, -1]]];
testPower[x_, y_] := If[x == 0 && y < 0, Inf, Power[x, y]];
testSet[HoldPattern[testVar[x_]], expr_] := Set[x, expr];
testVar[x_] := If[ValueQ[x], x, Throw[$Failed, {"varundef", x}]];
testIf[cond_, expr_] := If[cond, expr];
testIf[cond_, expr_, else_] := If[cond, expr, else];
module[{vars__}, body_] := Module[{vars}, body];
package[name_, code_] := (BeginPackage[name]; code; EndPackage[]);
inContext[name_, code_] := (Begin[name]; code; End[]);

Here is a small code snippet in this new language (wrapped in Hold):

cd = 
Hold[module[{a}, testSet[testVar[a],
  testPlus[testTimes[testTimes[testPlus[1, 2],
    testPower[testPlus[3, 4], -1]], testPlus[5, 6]], -7]]; testVar[a]]]

It corresponds to this mma code:

Module[{a},a = (1 + 2)/(3 + 4)*(5 + 6) - 7; a]

Our code-generator is based on a very simple idea - we will repeatedly apply local rules to our held code. The local rules will be extracted from the definitions of our functions, like so:

ClearAll[expansionRules];
expansionRules[heads : {__Symbol}] := Flatten[DownValues /@ heads]

We need to supply a list of heads for our language. I will do that manually, but it is easy to automate, by creating custom assignment operators.

allHeadsToExpand[] := {testIf, testVar, testPlus, testTimes, testDivide, 
      testPower, testSet, testIf,module,package, inContext}

Now, we generate our code:

In[195]:= expanded = cd//.expansionRules[allHeadsToExpand[]]

Out[195]= 
 Hold[Module[{a}, 
    a = ((1 + 2) If[3 + 4 == 0 && -1 < 0, Inf, 1/(3 + 4)]) (5 + 6) - 7; 
    If[ValueQ[a], a, Throw[$Failed, {"varundef", a}]]]]

To execute it, you can simply use ReleaseHold:

In[197]:= ReleaseHold[expanded]

Out[197]= -(16/7)

The advantage of our construction is that we can also execute our AST directly:

In[198]:= ReleaseHold[cd]

Out[198]= -(16/7)

To save this to a package, you can simply use Put command. It is also easy to extend the language in any way you want. Of course, the way the code in this language looks is not pretty, since it is essentially the AST expressed as mma expression. To make it prettier, you'd need to introduce your own syntax and write a parser from it to mma AST, but that is another story.

EDIT

Regarding automating of code-generation and saving the generated code into a package: here are a couple of utilities to do that.

Clear[generateCode];
generateCode[code_Hold] :=
  code //. expansionRules[allHeadsToExpand[]] //.
   HoldPattern[
      CompoundExpression[left___, CompoundExpression[middle___], right___]] :> 
       (left; middle; right);

Clear[formatCode];
formatCode[code_Hold] :=
  StringReplace[Function[Null, ToString[Unevaluated[#], InputForm], HoldAll] @@ 
     code, ";" :> ";\n"];

Clear[saveCode];
saveCode[file_, generatedCode_] :=
 With[{result = BinaryWrite[file, formatCode@generatedCode]},
   Close[file];
   result];

Here is the same example but placed in a package:

cdp = Hold[
   package["myPackage`",
     inContext["`Private`",
       module[{a}, 
         testSet[testVar[a],
           testPlus[testTimes[testTimes[testPlus[1, 2],
            testPower[testPlus[3, 4], -1]], testPlus[5, 6]], -7]]; 
         testVar[a]]]]]

We generate and save the code as follows:

In[101]:= file = FileNameJoin[{"C:","Temp","myPackage.m"}]
Out[101]= C:\Temp\myPackage.m

In[106]:= saved =saveCode[file,generateCode[cdp]]
Out[106]= C:\Temp\myPackage.m

We can Import it to test:

In[107]:= Import[file,"Text"]

Out[107]= 
BeginPackage["myPackage`"];
 Begin["`Private`"];
 Module[{a}, a = ((1 + 2)*If[3 + 4 == 0 && -1 < 0, Inf, (3 + 4)^(-1)])*(5 + 6) - 7;
  If[ValueQ[a], a, Throw[$Failed, {"varundef", a}]]];
 End[];
 EndPackage[]

EDIT 2

Regarding the way the code in your language will look, you can make this prettier without going all the way to create your own parser, by using the Notation package to alter the way you can input code and Format/FormatValues to control how it is rendered by the FrontEnd.

Answer 2

It is tangential to the question, but you may find important utility in the setting CellEvaluationFunction as described in a post by WReach.