Antlr4 - Is there a simple example of using the ParseTree Walker?

Question

Antlr4 has a new class ParseTreeWalker. But how do I use it? I am looking for a minimal working example. My grammar file is 'gram.g4' and I want to parse a file 'program.txt'

Here is my code so far. (This assumes ANTLR has run my grammar file and created all of the gramBaseListener, gramLexer, etc etc):

import org.antlr.v4.runtime.*;
import org.antlr.v4.runtime.tree.*;
import static org.antlr.v4.runtime.CharStreams.fromFileName;

public class launch{
public static void main(String[] args) {

    CharStream cs = fromFileName("gram.g4");  //load the file
    gramLexer lexer = new gramLexer(cs);  //instantiate a lexer
    CommonTokenStream tokens = new CommonTokenStream(lexer); //scan stream for tokens
    gramParser parser = new gramParser(tokens);  //parse the tokens

    // Now what??  How do I connect the above with the below? 

    ParseTreeWalker walker = new ParseTreeWalker();  // how do I use this to parse program.txt??
}}

I am using java but I assume it is similar in other languages.

The ANTLR documentation (http://www.antlr.org/api/Java/index.html) is short on examples. There are many tutorials on the internet but they are mostly for ANTLR version 3. The few using version 4 don't work or are outdated (for example, there is no parser.init() function, and classes like ANTLRInputStream are depreciated)

Thanks in advance for anyone who can help.

Answer 1

For each of your parser rules in your grammar the generated parser will have a corresponding method with that name. Calling that method will start parsing at that rule.

Therefore if your "root-rule" is named start then you'd start parsing via gramParser.start() which returns a ParseTree. This tree can then be fed into the ParseTreeWalker alongside with the listener you want to be using.

All in all it could look something like this (EDITED BY OP):

import org.antlr.v4.runtime.*;
import org.antlr.v4.runtime.tree.*;
import static org.antlr.v4.runtime.CharStreams.fromFileName;

public class launch{
public static void main(String[] args) {

    CharStream cs = fromFileName("program.txt");  //load the file
    gramLexer lexer = new gramLexer(cs);  //instantiate a lexer
    CommonTokenStream tokens = new CommonTokenStream(lexer); //scan stream for tokens
    gramParser parser = new gramParser(tokens);  //parse the tokens

    ParseTree tree = parser.start(); // parse the content and get the tree
    Mylistener listener = new Mylistener();

    ParseTreeWalker walker = new ParseTreeWalker();
    walker.walk(listener,tree);
}}

************ NEW FILE Mylistener.java ************

public class Mylistener extends gramBaseListener {
        @Override public void enterEveryRule(ParserRuleContext ctx) {  //see gramBaseListener for allowed functions
            System.out.println("rule entered: " + ctx.getText());      //code that executes per rule
        }
    }

Of course you have to replace <listener> with your implementation of BaseListener

And just one small sidenode: In Java it is convention to start classnames with capital letters and I'd advise you to stick to that in order for making the code more readable for other people.

Answer 2

This example should work with ANTLR 4.8.

Below the example you can find references to setup your Java env, API and Listeners.

public class Launch {
    public static void main(String[] args) {
        InputStream inputStream = null;
        MyprogramLexer programLexer = null;
        try {
            File file = new File("/program.txt");
            inputStream = new FileInputStream(file);
            programLexer = new MyprogramLexer(CharStreams.fromStream(inputStream)); // read your program input and create lexer instance
        } finally {
            if (inputStream != null) {
                inputStream.close();
            }
        }
        
        /* assuming a basic grammar:
            myProgramStart: TOKEN1 otherRule TOKEN2 ';' | TOKENX finalRule ';'
            ...
        */
        CommonTokenStream tokens = new CommonTokenStream(programLexer); // get tokens
        MyParser parser = new MyParser(tokens);
        MyProgramListener listener = new MyProgramListener();   // your custom extension from BaseListener
        parser.addParseListener(listener);
        parser.myProgramStart().enterRule(listener);    // myProgramStart is your grammar rule to parse

        // what we had built?
        MyProgram myProgramInstance = listener.getMyProgram();    // in your listener implementation populate a MyProgram instance
        System.out.println(myProgramInstance.toString());
    }
}

References:

• https://www.antlr.org/api/Java/
• https://tomassetti.me/antlr-mega-tutorial/#java-setup
• https://riptutorial.com/antlr/example/16571/listener-events-using-labels