[antlr-interest] Returning Multiple Tokens

Jason jasonriz at yahoo.com
Mon Jul 5 14:20:42 PDT 2004


Hello,

There are certain circumstances under which I'd like
to return more than a single token to the parser.  I
can accomplish this by setting a flag when such a
condition occurs and manually inserting something like
the following (in psuedocode) at the beginning of
nextToken( ) in the generated lexer:

if (tokensQueud)
{
  Token token = first token on list;
  if (list is empty)
  {
    tokensQueud = false;
  }
  return token;
}

Obviously though I'd rather not have to hack the
generated lexer.  Is there any other way to accomplish
this?  Is there some reason I should avoid this
approach and restructure the rules in my lexer grammar
to avoid the situtation?  Thanks in advance for any
replies.

-jason


		
__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - Send 10MB messages!
http://promotions.yahoo.com/new_mail 


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list