Why I have to use explicitly $setType()?

ycmichelid michel_cedric at hotmail.com
Tue Feb 12 01:42:11 PST 2002


Thank you for the advice.
In fact, my real grammar is much more sophisticated than the
simplified one I reproduced here.
I want to parse an XML-like language whith tags, where the syntax and
semantics is changing between tags.
So, I need to switch between multiple lexers using a
TokenStreamSelector. The problem with literals within "tokens" section
in the parser is that I can't write java code behind in order to
switch to a different lexer. That's why I need to write rules in
lexers to recognize my tags/keywords.
When I do that normally, tokens are not recognized properly and I get
an exception even for the first token.
The only (crazy) way I found to make it work is to use $setType() for
each token recognized by the lexer. When I remove these $setType(),
nothing works. It's a bit strange but it works.


--- In antlr-interest at y..., mzukowski at b... wrote:
> ALERT really should be a keyword and not a lexer rule.  FUNCNAME
should have
> the option testLiterals set to true.  You should not have to set
types
> explicitly as you have done below, by default they are the same type
as the
> rule.
> 
> Read the generated code, it's always the key to understanding antlr.
> 
> Monty
> 
> > -----Original Message-----
> > From: ycmichelid [mailto:michel_cedric at h...]
> > Sent: Monday, February 11, 2002 6:08 AM
> > To: antlr-interest at y...
> > Subject: [antlr-interest] Why I have to use explicitly $setType()?
> > 
> > 
> > My problem
(http://groups.yahoo.com/group/antlr-interest/message/5028)
> > seems to be also a conflict between keyword and identifier.
> > 
> > I found a solution to make my tokens being recognized normally.
> > I have to explicitly use the function $setType().
> > 
> > In my case , rules in my lexer become (rule to skip whitespace
> > ignored):
> > 
> > ALERT: "Alert" {$setType(TestTokenTypes.ALERT);};
> > OPENPAR : "(" {$setType(TestTokenTypes.OPENPAR);};
> > FUNCNAME : ('A'..'Z')( 'a'..'z' | 'A'..'Z' | '0'..'9' | '_' )+
> >            {$setType(TestTokenTypes.FUNCNAME);}
> > ;
> > 
> > So, it works now.
> > But why do i have to set the type of the tokens manually?
> > 
> > 
> > 
> >  
> > 
> > Your use of Yahoo! Groups is subject to 
> http://docs.yahoo.com/info/terms/



More information about the antlr-interest mailing list