How to use ANTLR v4 for syntax highlighting?

Question

I've built a grammar for a DSL and I'd like to display some elements (table names) in some colors. I output HTML from Java.

columnIdentifier :
      columnName=Identifier
    | tableName=Identifier '.' columnName=Identifier
    ;

Identifier : Letter LetterOrDigit* ;
fragment Letter : [a-zA-Z_];
fragment LetterOrDigit : [a-zA-Z0-9_];

WS  :  [ \t\r\n\u000C]+ -> skip ;

I thought about using an AbstractParseTreeVisitor to return all elements as-is, except those I want to highlight which would be returned as <span class="tablename" >theoriginaltext</span>. Is it a good approach?

Note the whitespaces are dismissed before they are sent to the parser, correct? So if I rebuilt the output using an AbstractParseTreeVisitor, I can't rebuild the spaces.

I assume there's a canonical way of doing syntax highlighting with ANTLR4. It's difficult to find information about this because searches often return results about highlighting the Antlr4 files in Eclipse/IDEA.

Answer 1

The Definitive ANTLR4 Reference contains the answer at paragraph 12.1:

Instead of skipping whitespace, send it to a hidden channel:

WS  :  [ \t\r\n\u000C]+ -> channel(HIDDEN)
    ;

Then the whitespace is still ignored in the context of the grammar (which is what we want) and getTranslatedText() will successfully return all text including whitespace. Use a listener such as:

public static class HtmlHighlighterListener extends MyDSLBaseListener {
    private final CommonTokenStream tokens;
    private final TokenStreamRewriter rewriter;

    public HtmlHighlighterListener(CommonTokenStream tokens) {
        this.tokens = tokens;
        this.rewriter = new TokenStreamRewriter(tokens);
    }

    ... Place here the overrides of "enterEveryRule" and "exitEveryRule"

    public String getTranslatedText() {
        return rewriter.getText();
    }
}

ParseTreeWalker walker = new ParseTreeWalker();
HtmlHighlighterListener listener = new HtmlHighlighterListener(tokens);
walker.walk(listener, tree);
return listener.getTranslatedText();

Then you can override "enterEveryRule" and "exitEveryRule" to add HTML tags for coloration.