[antlr-interest] ANTLR4, Left Recursion and imported grammars

[Rene Treffer]

Hi *,

I get really strange behavior if I try to split a parser with left
recursive rules into multiple files...

I've reduced the problem to a rather primitive Example:

- MainLexer.g

lexer grammar MainLexer;

NUMBER: [0-9]+ ;
ADD: '+' ;
MUL: '*' ;

- MainParser.g

parser grammar MainParser;

options {
    tokenVocab = MainLexer;
}

import MainExpression;

- MainExpression.g

parser grammar MainExpression;

expr :
    expr MUL expr
  | expr ADD expr
  | NUMBER
;

- Running antlr4

java -jar ~/bin/antlr-4.0b4-complete.jar MainLexer.g MainParser.g
Exception in thread "main" java.util.NoSuchElementException: token
index 27 out of range 0..24
        at
org.antlr.runtime.BufferedTokenStream.get(BufferedTokenStream.java:143)
        at
org.antlr.v4.analysis.LeftRecursiveRuleAnalyzer.stripAltLabel(LeftRecursiveRuleAnalyzer.java:363)
        at
org.antlr.v4.analysis.LeftRecursiveRuleAnalyzer.otherAlt(LeftRecursiveRuleAnalyzer.java:230)
        at
org.antlr.v4.parse.LeftRecursiveRuleWalker.outerAlternative(LeftRecursiveRuleWalker.java:787)
        at
org.antlr.v4.parse.LeftRecursiveRuleWalker.ruleBlock(LeftRecursiveRuleWalker.java:583)
        at
org.antlr.v4.parse.LeftRecursiveRuleWalker.rec_rule(LeftRecursiveRuleWalker.java:352)
        at
org.antlr.v4.analysis.LeftRecursiveRuleTransformer.translateLeftRecursiveRule(LeftRecursiveRuleTransformer.java:123)
        at
org.antlr.v4.analysis.LeftRecursiveRuleTransformer.translateLeftRecursiveRules(LeftRecursiveRuleTransformer.java:89)
        at
org.antlr.v4.semantics.SemanticPipeline.process(SemanticPipeline.java:93)
        at org.antlr.v4.Tool.processNonCombinedGrammar(Tool.java:393)
        at org.antlr.v4.Tool.process(Tool.java:381)
        at org.antlr.v4.Tool.processGrammarsOnCommandLine(Tool.java:345)
        at org.antlr.v4.Tool.main(Tool.java:192)

Moving the expr rule to MainParser works. Is this a Bug? Can I somehow
solve this problem?

Regards,
  Rene Treffer