Why I have to use explicitly $setType()?
ycmichelid
michel_cedric at hotmail.com
Tue Feb 12 01:42:11 PST 2002
Thank you for the advice.
In fact, my real grammar is much more sophisticated than the
simplified one I reproduced here.
I want to parse an XML-like language whith tags, where the syntax and
semantics is changing between tags.
So, I need to switch between multiple lexers using a
TokenStreamSelector. The problem with literals within "tokens" section
in the parser is that I can't write java code behind in order to
switch to a different lexer. That's why I need to write rules in
lexers to recognize my tags/keywords.
When I do that normally, tokens are not recognized properly and I get
an exception even for the first token.
The only (crazy) way I found to make it work is to use $setType() for
each token recognized by the lexer. When I remove these $setType(),
nothing works. It's a bit strange but it works.
--- In antlr-interest at y..., mzukowski at b... wrote:
> ALERT really should be a keyword and not a lexer rule. FUNCNAME
should have
> the option testLiterals set to true. You should not have to set
types
> explicitly as you have done below, by default they are the same type
as the
> rule.
>
> Read the generated code, it's always the key to understanding antlr.
>
> Monty
>
> > -----Original Message-----
> > From: ycmichelid [mailto:michel_cedric at h...]
> > Sent: Monday, February 11, 2002 6:08 AM
> > To: antlr-interest at y...
> > Subject: [antlr-interest] Why I have to use explicitly $setType()?
> >
> >
> > My problem
(http://groups.yahoo.com/group/antlr-interest/message/5028)
> > seems to be also a conflict between keyword and identifier.
> >
> > I found a solution to make my tokens being recognized normally.
> > I have to explicitly use the function $setType().
> >
> > In my case , rules in my lexer become (rule to skip whitespace
> > ignored):
> >
> > ALERT: "Alert" {$setType(TestTokenTypes.ALERT);};
> > OPENPAR : "(" {$setType(TestTokenTypes.OPENPAR);};
> > FUNCNAME : ('A'..'Z')( 'a'..'z' | 'A'..'Z' | '0'..'9' | '_' )+
> > {$setType(TestTokenTypes.FUNCNAME);}
> > ;
> >
> > So, it works now.
> > But why do i have to set the type of the tokens manually?
> >
> >
> >
> >
> >
> > Your use of Yahoo! Groups is subject to
> http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list