[antlr-interest] behaviour of lexer

michael micha-1 at fantasymail.de
Thu Mar 4 01:14:17 PST 2010


I have changed the AT rule a little bit and it's working now (I think):
In the AT rule the type of the token is changed dependent on the following 
text ("int("), note the empty fragment INTTOKEN:

lexer grammar TestLexer;

fragment DIGIT  : ('0'..'9');
fragment ALPHA  : ('a'..'z'|'A'..'Z'|'_');
fragment INTTOKEN :	 ;

OB              : '(';
AT              : ('@' 'int' OB) => '@int(' { $type=INTTOKEN;}
	       | '@' ;

NAME            : ALPHA (ALPHA | DIGIT)*;

WS:    (' ' |'\t' |'\n' |'\r' )+ {skip();} ;




and the parser:
parser grammar TestParser;


options {
	tokenVocab = TestLexer;
	
}

start: go+ EOF;


go: OB  { System.err.println("OB!"); }
| INTTOKEN {System.err.println("INTTOKEN"); }
| AT {System.err.println("AT"); }
| NAME {System.err.println("NAME" + $NAME); }
;


the output for  this input:
@ name @foo  @integer @int( @int4

is:

AT
NAME[@1,2:5='name',<9>,1:2]
AT
NAME[@3,8:10='foo',<9>,1:8]
AT
NAME[@5,14:20='integer',<9>,1:14]
INTTOKEN
AT
NAME[@8,29:32='int4',<9>,1:29]


cheers,
 Michael




More information about the antlr-interest mailing list