Do Perl 6 programs have to compile to read embedded docs?

Question

Perl 6 Plain-Old-Documentation (perhaps Fancy-New-Documentation) has some features that allow it to construct documentation for things it sees, and the documentation shows up in the $=pod variable at runtime.

However, I was surprised when I couldn't read the docs when I'd made an error in the program text. Here I've left out a statement separator between two statements:

use v6;

BEGIN { put "BEGIN" }
INIT  { put "INIT" }
CHECK { put "CHECK" }

"foo" "bar";

DOC INIT  { put "DOC INIT" }
DOC BEGIN { put "DOC BEGIN" }
DOC CHECK { put "DOC CHECK" }

=begin pod

=head1 This is a title

This is a bit of pod

=end pod

When I run it with the --doc switch, the program syntax matters (and BEGIN runs):

$ perl6 --doc doc.p6
BEGIN
===SORRY!=== Error while compiling ...
Two terms in a row
------>     "foo"⏏ "bar";
    expecting any of:
        infix
        infix stopper
        statement end
        statement modifier
        statement modifier loop

When I fix it, I get some warnings (so, perl6 is compiling) and the compilation-time phasers run:

BEGIN
DOC BEGIN
DOC CHECK
CHECK
WARNINGS for /Users/brian/Desktop/doc.p6:
Useless use of constant string "bar" in sink context (line 9)
Useless use of constant string "foo" in sink context (line 9)
INIT
DOC INIT
This is a title

This is a bit of pod

We already know this is a bit dangerous in Perl 5. A perl -c and a BEGIN block can run code. See How to check if a Perl script doesn't have any compilation errors?. I don't think this is any more dangerous than something we already know, but now it's happening at a time when I'm not explicitly asking something to compile program statements.

I haven't delved into the details of Perl 6 pod and why this might be necessary outside of declarator blocks and .WHY (a cool feature), but it seems like this can lead to trouble. Is there perhaps an external program that might extract the Pod? Or a way to do without the declarators unless the program will run?

Answer 1

Yes, the whole file has to be parsed, which in turn requires running BEGIN and use statements and such.

The Perl 6 language is designed for one-pass parsing from top to bottom, so that at any given point the parser understands what it is parsing based on what it has parsed so far.

Consider code like the following:

say "

=begin pod

Not POD, just a string!

";

If you'd just grep the file for POD statements without parsing all of it, it would misinterpret this piece of code.

I.e. you can't parse only the POD parts without parsing the normal Perl 6 code parts, because without parsing it all from top to bottom you can't know which is which.

---

PS: In theory, the Perl 6 designers could have accommodated POD-only parsing, by making it illegal for normal Perl 6 code to contain lines that look like they start a POD block. In this scenario, the above code snippet would be a syntax error when the whole file is parsed, because starting a line inside a string literal with =begin pod would be disallowed, so the --pod switch could rely on all lines that begin with =begin foo actually starting a POD block.

Such a restriction probably wouldn't be a major burden for normal Perl 6 code (after all, who needs to write =begin pod at the start of a line in a multi-line string literal), but note that one of the reasons for the one-pass top-to-bottom parsing architecture is to facilitate language extensibility via slangs.
E.g. CPAN modules could add support for users writing a single subroutine (or other lexical scope) in another language or DSL. (Implementing such modules isn't actually possible yet without hacking into Rakudo internals via NQP, but once the macro/slang design is complete, it will be).
The burden for disallowing lines that look like they start a POD block, would then be passed on to all those slang parsers.

You could always submit a feature request for Larry and the other Perl 6 designers to consider this, though.