ANTLR4 and the Python target

Question

I'm having issues getting going with a Python target in ANTLR4. There seems to be very few examples available and going to the corresponding Java code doesn't seem relevant.

I'm using the standard Hello.g4 grammar:

// Define a grammar called Hello
grammar Hello;
r  : 'hello' ID ;         // match keyword hello followed by an identifier
ID : [a-z]+ ;             // match lower-case identifiers
WS : [ \t\r\n]+ -> skip ; // skip spaces, tabs, newlines

The example (built from the standard Hello.g4 example):

input_ = antlr4.FileStream(_FILENAME)
lexer = HelloLexer.HelloLexer(input_)
stream = antlr4.CommonTokenStream(lexer)
parser = HelloParser.HelloParser(stream)

rule_name = 'r'
tree = getattr(parser, rule_name)()

I also wrote a listener. To assert/verify that this is correct, I'll repeat it here:

class HelloListener(antlr4.ParseTreeListener):
    def enterR(self, ctx):
        print("enterR")

    def exitR(self, ctx):
        print("exitR")

    def enterId(self, ctx):
        print("enterId")

    def exitId(self, ctx):
        print("exitId")

So, first, I can't guarantee that the string I'm giving it is valid because I'm not getting any screen output. How do I tell from the tree object if anything was matched? How do I extract the matching rules/tokens?

A Python example would be great, if possible.

Answer 1

I hear you, having the same issues right now. The Python documentation for v4 is useless and v3 differs to much to be usable. I'm thinking about switching back to Java to implement my stuff.

Regarding your code: I think your own custom listener has to inherit from the generated HelloListener. You can do the printing there.

Also try parsing invalid input to see if the parser starts at all. I'm not sure about the line with getattr(parser, rule_name)() though. I followed the steps in the (unfortunately very short) documentation for the Antlr4 Python target: https://theantlrguy.atlassian.net/wiki/display/ANTLR4/Python+Target

You can also find some documentation about the listener stuff there. Hope it helps.

Answer 2

This question seems to be old, but I also had the same problems and found out how to deal with it. When using strings in python, you have to use the function antlr4.InputStream as pointed out here

So, in the end, you could get a working example with this sort of code (based on Alan's answer and an example from dzone)

from antlr4 import *
from grammar.HelloListener import HelloListener
from grammar.HelloLexer import HelloLexer
from grammar.HelloParser import HelloParser
import sys

class HelloPrintListener(HelloListener):
   def enterHi(self, ctx):
      print("Hello: %s" % ctx.ID())

def main():
   giveMeInput = input ("say hello XXXX\n")
   print("giveMeInput is {0}".format(giveMeInput))

   # https://www.programcreek.com/python/example/93166/antlr4.InputStream
   # https://groups.google.com/forum/#!msg/antlr-discussion/-9VJ5H9NcDs/OukVNCTQCAAJ
   i_stream = InputStream(giveMeInput)

   lexer = HelloLexer(i_stream)
   t_stream = CommonTokenStream(lexer)
   parser = HelloParser(t_stream)
   tree = parser.hi()
   printer = HelloPrintListener()
   walker = ParseTreeWalker()
   walker.walk(printer, tree)

if __name__ == '__main__':
   main()