[antlr-interest] Getting token extents for grammar rules

[Mark Lentczner]

Paul -

You can't call giveToken() to each lexer rule: This will execute it 
when the token is lex'd which, as you surmise, due to look ahead could 
be well before it is used.  Think of the lexer and parser as 
asynchronous threads (even though they are not).  They are loosely 
coupled.

What you want is to call giveToken() when the parser calls consume().  
This will happen exactlly between your calls to enterRule()/leaveRule() 
and will happen exactly and only when the parser really uses the token.

Since consume() is a virtual method of Parser, just override it in your 
parser sub-class.  So, in your override you'll call:

	void consume()
	{
	    giveToken(LA(1));
	    Parser::consume();
	}

(All examples C++, but I'm sure the Java is very similar.)

	- Mark

Mark Lentczner
markl at wheatfarm.org
http://www.wheatfarm.org/



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/