What scope does ":my $foo" have and what is it used for?

Question

With a regex, token or rule, its possible to define a variable like so;

token directive {
    :my $foo = "in command";
    <command> <subject> <value>?
}

There is nothing about it in the language documentation here, and very little in S05 - Regexes and Rules, to quote;

Any grammar regex is really just a kind of method, and you may declare variables in such a routine using a colon followed by any scope declarator parsed by the Perl 6 grammar, including my, our, state, and constant. (As quasi declarators, temp and let are also recognized.) A single statement (up through a terminating semicolon or line-final closing brace) is parsed as normal Perl 6 code:

token prove-nondeterministic-parsing {
    :my $threshold = rand;
    'maybe' \s+ <it($threshold)>
}

I get that regexen within grammars are very similar to methods in classes; I get that you can start a block anywhere within a rule and if parsing successfully gets to that point, the block will be executed - but I don't understand what on earth this thing is for.

Can someone clearly define what it's scope is; explain what need it fulfills and give the typical use case?

Answer 1

What scope does :my $foo; have?

:my $foo ...; has the lexical scope of the rule/token/regex in which it appears.

(And :my $*foo ...; -- note the extra * signifying a dynamic variable -- has both the lexical and dynamic scope of the rule/token/regex in which it appears.)

What this is used for

Here's what happens without this construct:

regex scope-too-small {    # Opening `{` opens a regex lexical scope.
    { my $foo = / bar / }  # Block with its own inner lexical scope.
    $foo                   # ERROR: Variable '$foo' is not declared
}

grammar scope-too-large {  # Opening `{` opens lexical scope for gramamr.
    my $foo = / bar / ;
    regex r1   { ... }     # `$foo` is recognized inside `r1`...
    ...
    regex r999 { ... }     # ...but also inside r999
}

So the : ... ; syntax is used to get exactly the desired scope -- neither too broad nor too narrow.

Typical use cases

This feature is typically used in large or complex grammars to avoid lax scoping (which breeds bugs).

For a suitable example of precise lexical only scoping see the declaration and use of @extra_tweaks in token babble as defined in a current snapshot of Rakudo's Grammar.nqp source code.

P6 supports action objects. These are classes with methods corresponding one-to-one with the rules in a grammar. Whenever a rule matches, it calls its corresponding action method. Dynamic variables provide precisely the right scoping for declaring variables that are scoped to the block (method, rule, etc.) they're declared in both lexically and dynamically -- which latter means they're available in the corresponding action method too. For an example of this, see the declaration of @*nibbles in Rakudo's Grammar module and its use in Rakudo's Actions module.