[antlr-interest] How to swich the recognition scope in Lexer
Micheal J
open.zone at virgin.net
Wed Jun 20 05:10:45 PDT 2007
> Silvester Pozarnik wrote this on [20 June 2007 13:00]:
>
> In the antlr 2.7.7 you could change the behaviour of Lexer so
> that tokens are recognized az literals in special cases by
> overriding the
> testLiteralsTable() method in CharScanner class. How to the
> same in antlr 3.0 if you have a grammar as:
>
> grammar test;
> tokens {
> MYTOKEN = 'mytoken';
> }
> mygrammar:
> {
> MYTOKEN LPAREN IDENTIFIER RPAREN
> }
>
> LPAREN : '(' ;
> RPAREN : ')' ;
> IDENTIFIER
> : ('a'..'z' | 'A'..'Z' | '\u0080'..'\ufffe') (
> Letter | Digit)*;
>
> fragment Letter
> : 'a'..'z' | 'A'..'Z' | '_' |'-' | '\u0080'..'\ufffe';
>
> fragment Digit
> : '0'..'9';
>
> So that the input "mytoken(mytoken)" is a valid. The first
> 'mytoken' should be recognized as MYTOKEN, but the second
> 'mytoken' has to be recognized as an IDENTIFIER. Is there a
> way to achieve this?
Not to my knowledge (and this applies to V2.x too). Is suspect you need to
change your 'mygrammar' rule:
mygrammar : MYTOKEN LPAREN (MYTOKEN|IDENTIFIER) RPAREN
Micheal
-----------------------
The best way to contact me is via the list/forum. My time is very limited.
More information about the antlr-interest
mailing list