Bison error output

Question

I'm using Bison and I've generated a quite complex grammar. The trouble is that my first test case is failing- but Bison will only say "syntax error". Is there any way to ask Bison to output the rule that failed to match and the token that is a problem? I used

%define parse.trace true

but still only get syntax error as the output.

Answer 1

None of the Yacc-based parsers does anything much better than 'syntax error' when there is a problem; it is largely up to you to improve on that.

There are a couple of things you can do fairly readily.

One is to instrument your lexical analyzer so it prints out the tokens it finds as it returns them to the parser proper. This tells you which token the grammar is failing on, and what tokens were supplied beforehand.

The other is to compile with Yacc debug enabled, and then turn it on. This requires -DYYDEBUG=1 and setting the variable yydebug to a non-zero value (conventionally 1). The first step compiles the extra information into the grammar; the second step enables the output.

From the Bison 2.4.3 manual:

§8.2 Tracing Your Parser

If a Bison grammar compiles properly but doesn’t do what you want when it runs, the yydebug parser-trace feature can help you figure out why.

There are several means to enable compilation of trace facilities:

• the macro YYDEBUG

  Define the macro YYDEBUG to a nonzero value when you compile the parser. This is compliant with POSIX Yacc. You could use -DYYDEBUG=1 as a compiler option or you could put #define YYDEBUG 1 in the prologue of the grammar file (see Section 3.1.1 [The Prologue], page 47).

• the option -t, --debug

  Use the -t option when you run Bison (see Chapter 9 [Invoking Bison], page 117). This is POSIX compliant too.

• the directive %debug

  Add the %debug directive (see Section 3.7.12 [Bison Declaration Summary], page 72). This is a Bison extension, which will prove useful when Bison will output parsers for languages that don’t use a preprocessor. Unless POSIX and Yacc portability matter to you, this is the preferred solution.

We suggest that you always enable the debug option so that debugging is always possible.

The trace facility outputs messages with macro calls of the form YYFPRINTF (stderr, format, args) where format and args are the usual printf format and variadic arguments. If you define YYDEBUG to a nonzero value but do not define YYFPRINTF, <stdio.h> is automatically included and YYFPRINTF is defined to fprintf.

Once you have compiled the program with trace facilities, the way to request a trace is to store a nonzero value in the variable yydebug. You can do this by making the C code do it (in main, perhaps), or you can alter the value with a C debugger.

Each step taken by the parser when yydebug is nonzero produces a line or two of trace information, written on stderr. The trace messages tell you these things:

• Each time the parser calls yylex, what kind of token was read.
• Each time a token is shifted, the depth and complete contents of the state stack (see Section 5.5 [Parser States], page 95).
• Each time a rule is reduced, which rule it is, and the complete contents of the state stack afterward.

Answer 2

Use the %error-verbose directive to help you out. It will give you a hint of which tokens the parser was expecting. Note that there may be more than one potential rule that has failed to match.

Answer 3

If you set the global variable yydebug to a nonzero value, bison will output debugging information as it's running that contains information about what the parse stack looks like, what state it's in, what rules it's using, etc. It's how I usually go about debugging these sorts of errors.

Answer 4

There is a confusion here.

"%define parse.trace" is exactly the same as "%debug": it instruments the generated parser for runtime traces, showing the parser at work (but you still have to set the yydebug variable, see the documentation kindly reproduced by Jonathan Leffler).

For error messages to be more accurate (on this regard Jonathan is wrong), Bison can do better, thanks to "%define error-verbose", as reported by Alek.