[antlr-interest] Dealing with tokens and keyword abbreviations

Brisard, Fred D Fred.Brisard at ca.com
Tue Aug 19 05:57:03 PDT 2008


I am trying to implement detection of abbreviated keywords using the
technique described in the wiki page
http://www.antlr.org/wiki/pages/viewpage.action?pageId=1802308

It appears to be a perfect solution for me except that when I output the
token to the AST I get the token value (the actual characters associated
with the token);  I would prefer to get the token id (type) value.

So if I use the technique described in the wiki I get the following...

tokens
{
KeyUpload
KeyDownload
}

I use a method to map the keywords upload and download to KeyUpload and
KeyDownload respectively.  The method also handles abbreviations and
ignores case so that u, U, up, Up, UP all map to KeyUpload.

When I output the KeyUpload token to the AST using something like
^(KeyUpload), I get the original source value for the keyword.  I would
prefer to have the type KeyUpload set in the AST since I've already
identified that any of the valid abbreviations map to that token.

I was using a predicate rule technique to accomplish the same
previously, but this technique is cleaner.  Using predicate rules the
token type was stored in the AST instead of the token value.

Thanks for any help.

Fred


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20080819/fb9fa652/attachment.html 


More information about the antlr-interest mailing list