[antlr-interest] RFE: bring back (at least the global) testLiterals option!

Terence Parr parrt at cs.usfca.edu
Mon Dec 31 11:47:07 PST 2007


On Dec 28, 2007, at 2:14 PM, J Chapman Flack wrote:

>
> Hi,
>
> I'm thinking of using the "implicit strategy" for keyword lexing
> described in
> http://www.antlr.org/wiki/pages/viewpage.action?pageId=1802308
> because it closely matches how the reference implementation of
> this language does it, and it was very easy in ANTLR 2.7.7 using
>
> tokens {
>  KWD1 = 'kwd1';
>  KWD2 = 'kwd2';
>  ...
>  KWD252 = 'kwd252';
> }
>
> with global options { testLiterals=false;  
> caseSensitiveLiterals=false; }
>
> and rule options { testLiterals=true; } on the rule for identifiers.
> The result was clear, correct, and compact.
>
> To do the same thing in ANTLR 3, I have to supply my own
> CheckKeywordsTable() that implements the intended matching rules,
> and use it explicitly in the identifier rule.  That's no big deal.
>
> What's unpleasant is that ANTLR hasn't preserved my nice, clear
> tokens { } block in any form I can use for the purpose. An
> array of the keyword strings would offer a way for Java code
> to recover and use them.

I looked in the parser generated from your above list and see:

     public static final String[] tokenNames = new String[] {
         "<invalid>", "<EOR>", "<DOWN>", "<UP>", "KWD1", "KWD2",  
"KWD252"
     };

Would that work for you?

Ter


More information about the antlr-interest mailing list