Gradle can't find Antlr token file

Question

I created a file MyLexer.g4 inside myproject/src/main/antlr/com/mypackage like:

lexer grammar MyLexer;

DIGIT : '0' .. '9' ;

...

WS  : [ \t\r\n]+ -> skip ;

and then trying to write parser in MyParser.g4 in the same directory:

grammar MyParser;

options
   { tokenVocab = MyLexer; }

SHORT_YEAR: DIGIT DIGIT;

unfortunately, when I run gradle task of generateGrammarSource, the following error occurs:

error(160): com\mypackage\MyParser.g4:4:18: cannot find tokens file MYPROJECT\build\generated-src\antlr\main\MyLexer.tokens

I.e. file is sought in incorrect place.

Actual file is created inside MYPROJECT\build\generated-src\antlr\main\com\mypackage\MyLexer.tokens

Answer 1

As Steven Spungin said, you need to put your ANTLR source files in the directory src/main/antlr/ and not in a subdirectory of that directory.

You do not need to add the @header to your ANTLR source files. Use the following instead.

In your build.gradle you should have (modified for your version of ANTLR):

apply plugin: 'antlr'

dependencies {
    antlr "org.antlr:antlr4:4.7.1"
}

generateGrammarSource {
    arguments += ['-package', 'com.mypackage']
    outputDirectory = new File(buildDir.toString() + "/generated-src/antlr/main/com/mypackage/")
}

Answer 2

When generating your parser in a package by using:

@header {package org.acme.my.package;}

and declaring tokenVocab in your parser

options {tokenVocab = MyLanguage;}

The MyLanguageLexer.g4 and MyLanguageParser.g4 files must NOT BE in a package directory. due to a bug of sorts.

So this means /src/main/antlr/MyLanguageParser.g4 and not /src/main/antlr/com/acme/my/package/MyLanguageParser.g4.

The java files end up in the wrong directory in build/generated-src/antlr, but somehow make it to the correct directory in build/classes/java/main. And the .tokens file ends up where antlr expects it.

---

Keep in mind this will confuse your IDE; I had to add the compiled classes back to my compileClasspath to avoid visual class-not-found issues.

dependencies {
    testCompile fileTree('build/classes/java/main')
}