[antlr-interest] Guidance to lexer from parser

Michael LeBlanc Mike at LeBlanc-Engineering.com
Wed Oct 24 11:04:36 PDT 2007


I want to extend a language with a form to modify the token type of  
an identifier.  For example,

	DeclareSpecial(ABC);

will change ABC's type from IDENT to SPECIAL.  It just adds an entry  
to a HashTab with key "ABC".  The lexer rule for IDENT contains an  
action to consult the table to see if its result is SPECIAL instead  
of IDENT and invokes setType(SPECIAL).  It all works, except ...  
Suppose the next line is

	ABC

The lexer returns IDENT because it has already consumed that token  
into a buffer before the specialness was asserted.  Is there a way to  
rewind the token buffer and rescan?



More information about the antlr-interest mailing list