How to prevent table regeneration in PLY

Question

I am using PLY in a command line application that I package as a Python egg to be installed via pip. Everytime I run my script from the command line, I see the following message:

"Generating LALR tables"

Additionally, parser.out and parsetab.py files are written to the directory from which the script is invoked. Is there some way to ship these files with the application so that it does not regenerate the tables each and every time?

Answer 1

use

yacc.yacc(debug=0, write_tables=0)

Answer 2

You want to use optimized mode, by calling lex as:

lexer = lex.lex(optimize=1)

.

It's worth emphasising (from the same link):

On subsequent executions, lextab.py will simply be imported to build the lexer. This approach substantially improves the startup time of the lexer and it works in Python's optimized mode.

When running in optimized mode, it is important to note that lex disables most error checking. Thus, this is really only recommended if you're sure everything is working correctly and you're ready to start releasing production code.

Since this is production code, this sounds like exactly what you want.

.

In looking into this issue, I came across the miscellaneous Yacc notes:

Since the generation of the LALR tables is relatively expensive, previously generated tables are cached and reused if possible. The decision to regenerate the tables is determined by taking an MD5 checksum of all grammar rules and precedence rules. Only in the event of a mismatch are the tables regenerated.

And looking deeper into the yacc function inside yacc.py, we see that optimise ignores this mismatch in the following snippet:

if optimize or (read_signature == signature):
    try:
        lr.bind_callables(pinfo.pdict)
        parser = LRParser(lr,pinfo.error_func)
        parse = parser.parse
        return parser

where signature is compared to checksum stored in parsetab.py (as _lr_signature).