[antlr-interest] Howto modify token creation?
David-Sarah Hopwood
david-sarah at jacaranda.org
Thu Sep 24 00:09:42 PDT 2009
Indhu Bharathi wrote:
> You can do something like
>
> ID : LETTER (LETTER|DIGIT)*
> {
> String text = getText();
> Integer tknType;
> if( (tknType=table.get(text))!=null ) {
> $type = tknType;
> }
> }
>
> The table can be passed to the lexer using some member function. But I don't
> know any clean way how to make sure ANTLR lexer doesn't assign the same int
> to some other token.
YourParser.tokenNames.length should be the first unused token number.
(Obviously, this is relying on an implementation detail, but probably a
fairly stable one.)
Note that if you use token numbers >= tokenNames.length, you should override
getErrorMessage in the parser, so that it doesn't throw an
ArrayIndexOutOfBoundsException when constructing an error message involving
one of these tokens. For example:
@parser::members {
public String getErrorMessage(RecognitionException e, String[] names) {
String[] dynamicNames = /* array also containing dynamic token names */;
return super.getErrorMessage(e, dynamicNames);
}
}
--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com
More information about the antlr-interest
mailing list