[antlr-interest] Fwd: Rule precedence works differently when using a predicate?

[Bart Kiers]

Just a little bump, in case it got buried under some of the newer posts.
And in case my previous grammar wasn't entirely clear, the following
grammar:

grammar T;

@lexer::members {
  private boolean test() {
    return true;
  }
}

parse
  :  KEY EOF
  ;

KEY
  :  'key'
  ;

ANY
  :  ({test()}?=> . )+
  ;


with the test class:

import org.antlr.runtime.*;

public class Main {
  public static void main(String[] args) throws Exception {
    TLexer lexer = new TLexer(new ANTLRStringStream("key"));
    TParser parser = new TParser(new CommonTokenStream(lexer));
    parser.parse();
  }
}


Produces the following error:

line 1:0 mismatched input 'key' expecting KEY


In other words, 'key' is being tokenized as ANY instead of KEY.
Is this expected behavior or a bug? And if it's expected behavior, could
someone point me to the documentation (book) or wiki-link that explains
this?

Cheers & regards,

Bart.

-------------------

From: Bart Kiers <bkiers at gmail.com>
Date: Mon, Oct 24, 2011 at 11:46 AM
Subject: Rule precedence works differently when using a predicate?
To: "antlr-interest at antlr.org interest" <antlr-interest at antlr.org>


Hi all,

As I understand it, ANTLR's lexer matches rules from top to bottom in the .g
grammar file and when two rules match the same number of characters, the one
that is defined first has precedence over the later one(s).

However, take the following grammar:

grammar T;

@lexer::members {
  private boolean test() {
    return true;
  }
}

parse
  :  (t=. {System.out.println(tokenNames[$t.type] + " :: " +  $t.text);})*
EOF
  ;

KEY
  :  'key'
  ;

ANY
  :  ({test()}?=> . )+
  ;


And the test class:"

import org.antlr.runtime.*;


public class Main {
  public static void main(String[] args) throws Exception {
    TLexer lexer = new TLexer(new ANTLRStringStream("key"));
    TParser parser = new TParser(new CommonTokenStream(lexer));
    parser.parse();
  }
}


I'd expected "KEY :: key" to be printed to the console, however, "ANY :: key"
is printed instead. So the last rule is matched, while the KEY rule also
matches the same input and is defined before ANY. Why?

Kind regards,

Bart.