Why would you use a proto rule in a Perl 6 grammar?

Question

Here are two grammars. One uses a proto token and one doesn't. They both get the same done. These are basically the examples in S05 under "Variable (non-)interpolation". In this simple example, they are both able to do the same things.

Which situations justify all the extra typing? The proto tokens have distinct methods in the action class, and maybe there's a small benefit there. However, you have to type some extra stuff to get that benefit.

Is there some feature of proto that makes other parts of the grammar easier?

grammar NoProto {
    token variable   { <sigil> <identifier> }

    token identifier { <ident>+  }
    token sigil      { < $ @ % & :: > }
    }

grammar YesProto {
          token variable      { <sigil> <identifier> }

          token identifier    { <ident>+ }
    proto token sigil         {    *     }
          token sigil:sym<$>  {  <sym>   }
          token sigil:sym<@>  {  <sym>   }
          token sigil:sym<%>  {  <sym>   }
          token sigil:sym<&>  {  <sym>   }
          token sigil:sym<::> {  <sym>   }
    }

class Proto::Actions {
    method variable ($/) {
        say "found variable: " ~ $/;
        }
    method identifier ($/) {
        say "found identifier: " ~ $/;
        }
    method sigil ($/) {
        say "found sigil: " ~ $/;
        }
    method sigil:sym<$> ($/) {
        say "found sym sigil: " ~ $/;
        }
    }

my $variable = '$butterfuly';

say "------No proto parsing";
my $no_proto_match = NoProto.parse(
    $variable,
    :rule<variable>,
    :actions(Proto::Actions),
    );

say "------Yes proto parsing";
my $yes_proto_match = YesProto.parse(
    $variable,
    :rule<variable>,
    :actions(Proto::Actions),
    );

The output shows that proto calls a different method in the action class:

------No proto parsing
found sigil: $
found identifier: butterfuly
found variable: $butterfuly
------Yes proto parsing
found sym sigil: $
found identifier: butterfuly
found variable: $butterfuly

Answer 1

Technically, a proto will be made for you if you don't specify it yourself. It basically creates the multi-method dispatch handler for that particular token (just as it does with sub and method). Which you usually don't need to care about.

Why would you specify a proto? I can think of a number of reasons:

1. because you want the tokens to share some traits
2. because you want to execute some code before or after the dispatch

Yes, the { * } may contain executable code. The bare Whatever indicates the dispatching to the appropriate candidate. Showing this in a simpler situation with sub:

proto a(|) { say "before"; {*}; say "after" }
multi a(Int) { say "Int" }
multi a(Str) { say "Str" }
a 42; a "42"

shows:

before
Int
after
before
Str
after

Hope this helps :-)

Answer 2

One benefit of splitting your alternatives into a proto and multis is that you can extend it more reliably. You can add multis to the existing proto in a grammar that inherits from the grammar that declares the proto, and you don't need to list all the possible alternatives (which you'd have to do in case a of a single rule).

This means that you can even have multiple independent extensions to the same grammar, for example by mixing in several rules that supply multis for different symbols to match.

This is basically the mechanism that Perl 6 itself uses when you define a custom operator: there are rules for matching the different kinds of operators (like infix, prefix, postfix, ...), and declaring a new operator derives a new grammar from the currently active one, with a multi candidate added for the new operator. A script can import operators from several modules that don't know of each other thanks to the extensibility of proto token mechanism.