Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

raku grammar problem when trying to define a grammar for markdown

Tags:

raku

I'm using the following code to parse a subset of Markdown text:

#!/usr/bin/perl6

use v6;

my @tests = '',         #should print OK
            "\n\n\n",   #should print OK
            'alpha',    #should print OK
            '1234',     #should print OK
            'alpha123', #should print OK
            'a simple test without a heading and whitespaces and 123 numbers',                                                  #should print OK
            'a simple test without a heading with punctuation, whitespaces and 123 numbers.',                                   #should print OK
            'a simple test without a heading with punctuation, whitespaces, 123 numbers, quotes\' ", braces (){}<>[], \/.',     #should print OK
            'a simple test with a # in the middle',                                                                             #should print OK
            "#heading",         #should print FAILED
            "#heading ",        #should print FAILED
            "#heading\n",       #should print OK
            "#heading \n",      #should print OK
            "#heading\nand a text",                     #should print OK
            "#heading\nand\na\ntext\nwith\nnewlines",   #should print OK
            "#heading1\nand text\n#heading2\nand text"  #should print OK
            ;

grammar markdown_ltd {
    rule TOP {
        [ <text> | <headingandtext>+ ]
    }
    rule text {
        <textline>*<lasttextline>?
    }
    rule textline {
        <textlinecontent>? [\n]
    }
    rule lasttextline {
        <textlinecontent>
    }
    rule textlinecontent {
        <[\N]-[\#]> [\N]*
    }
    rule headingandtext {
        <heading> [\n] <text>
    }
    rule heading {
        [\#] [\N]+
    }
}
for @tests -> $test {
    my $result = markdown_ltd.parse($test);
    say "\nTesting '" ~ $test ~ "' " ~ ($result ?? 'OK' !! 'FAILED');
}

The purpose pf this grammar is to detect headings and regular text. However, there is some error in the code such that the included tests do not lead to the expected behavior. When I run the script, I get the following output:

Testing '' OK

Testing '


' OK

Testing 'alpha' FAILED

Testing '1234' FAILED

Testing 'alpha123' FAILED

Testing 'a simple test without a heading and whitespaces and 123 numbers' OK

Testing 'a simple test without a heading with punctuation, whitespaces and 123 numbers.' OK

Testing 'a simple test without a heading with punctuation, whitespaces, 123 numbers, quotes' ", braces (){}<>[], \/.' OK

Testing 'a simple test with a # in the middle' OK

Testing '#heading' FAILED

Testing '#heading ' FAILED

Testing '#heading
' FAILED

Testing '#heading 
' FAILED

Testing '#heading
and a text' FAILED

Testing '#heading
and
a
text
with
newlines' FAILED

Testing '#heading1
and text
#heading2
and text' FAILED

Which differs from the output I expected.

like image 352
byteunit Avatar asked Feb 24 '21 09:02

byteunit


1 Answers

Thanks @donaldh for pointing me to the right direction! By changing all occurences of rule to token fixed the issue!

like image 134
byteunit Avatar answered Sep 23 '22 02:09

byteunit