wrong line number in parse Exception

Question

I have a simple language defined in pyparsing. The parsing is working good, but the problem is with the error messages. They are showing wrong line number. I am showing major part of the code here

communications = Group( Suppress(CaselessLiteral("communications")) + op + ZeroOrMore(communicationList) + cl + semicolon)

language = Suppress(CaselessLiteral("language")) + (CaselessLiteral("cpp")|CaselessLiteral("python")) + semicolon

componentContents = communications.setResultsName('communications') & language.setResultsName('language') & gui.setResultsName('gui') & options.setResultsName('options')

component = Suppress(CaselessLiteral("component")) + identifier.setResultsName("name") + op + componentContents.setResultsName("properties") + cl + semicolon

CDSL = idslImports.setResultsName("imports") + component.setResultsName("component")

It is reporting correct line number only till component, but for any errors inside the component (ie in componentContents) its just saying the line number of where component starts. For example, this is a example of text to be parsed

import "/robocomp/interfaces/IDSLs/Test.idsl";

Component publish
{
    Communications
    {
        requires test;
        implements test;
    };
    language python;
};

here if i miss the semicolon after python; or after test. it would say (line:4, col:1) ie at {.

Answer 1

This behavior is characteristic of pyparsing, not a bug, and takes some extra care to work with (or work around).

When pyparsing fails to match somewhere in a complex expression it will unwind its parsing stack back to its last fully complete expression alternative. You know that after having matched "component" that anything after that should be an error within the component definition, but pyparsing does not. So when a failure occurs after the opening keyword, then pyparsing will backup and report that the keyword expression, including the keyword, could not be matched.

When you have a grammar of commands like this, the keywords are often unambiguous. For instance, after matching 'component', anything that is not an identifier followed by an argument list in parentheses is going to be an error. You can indicate that pyparsing should not back up past 'component' by replacing the '+' operator with '-' operator.

Looking at your grammar, I'll back up and write a short BNF (always good practice):

communications ::= 'communications' '(' communicationList* ')' ';'
language       ::= 'language' ('cpp' | 'python') ';'
componentContents ::= communications | language | gui | options
component      ::= 'component' identifier '(' component_contents+ ')' ';'
CDSL           ::= idslImports component

When there are keywords in a grammar, I always recommend using Keyword or CaselessKeyword, not Literal or CaselessLiteral. The Literal classes do not enforce word boundaries, so if I used Literal("no") as part of a grammar, it could match the leading 'no' of 'not' or 'none' or 'nothing', etc.

Here is how I would approach this BNF. (I'll use the shortcut version of setResultsName, which I find to leave the grammar itself to be clearer.):

LBRACE,RBRACE,SEMI = map(Suppress, "{};")
identifier = pyparsing_common.identifier

# keywords - extend as needed
(IMPORT, COMMUNICATIONS, LANGUAGE, COMPONENT, CPP, 
 PYTHON, REQUIRES, IMPLEMENTS) = map(CaselessKeyword, """
    IMPORT COMMUNICATIONS LANGUAGE COMPONENT CPP PYTHON 
    REQUIRES IMPLEMENTS""".split())

# keyword-leading expressions, use '-' operator to prevent backtracking once significant keyword is parsed
communicationItem = Group((REQUIRES | IMPLEMENTS) - identifier + SEMI)
communications = Group( COMMUNICATIONS.suppress() - LBRACE + ZeroOrMore(communicationItem) + RBRACE + SEMI)
language = Group(LANGUAGE.suppress() - (CPP | PYTHON) + SEMI)

componentContents = communications('communications') & language('language') & gui('gui') & options('options')
component = Group(COMPONENT - identifier("name") + Group(LBRACE + componentContents + RBRACE)("properties") + SEMI)

CDSL = idslImports("imports") + component("component")

Parsing your sample component with:

sample = """\
Component publish
{
    Communications
    {
        requires test;
        implements test;
    };
    language python;
};
"""

component.runTests([sample])

gives:

[['COMPONENT', 'publish', [[['REQUIRES', 'test'], ['IMPLEMENTS', 'test']], ['PYTHON']]]]
[0]:
  ['COMPONENT', 'publish', [[['REQUIRES', 'test'], ['IMPLEMENTS', 'test']], ['PYTHON']]]
  - name: 'publish'
  - properties: [[['REQUIRES', 'test'], ['IMPLEMENTS', 'test']], ['PYTHON']]
    - communications: [['REQUIRES', 'test'], ['IMPLEMENTS', 'test']]
      [0]:
        ['REQUIRES', 'test']
      [1]:
        ['IMPLEMENTS', 'test']
    - language: ['PYTHON']

(BTW, I like your use of '&' operator for unordered matching of the different contents with pyparsing's Each class - I think this makes for a friendlier and more robust parser. It turns out that Each has a slight conflict with '-' operator, I'll have to fix this in the next release.)