[antlr-interest] Problem with $type in final ANTLR 3.0

Jose San Leandro jose.sanleandro at ventura24.es
Thu Jun 21 22:24:40 PDT 2007


Thank you for your advice.
It used such construct to keep the lookahead at 1, and hand-code the 
dissambiguations.
Anyway, I actually tended to think the purpose of "fragment" rules in lexers 
where just to define which can be directly seen by the parser and which 
don't.
I'll follow your recommendation from now on. Besides that, I encounter this 
issue only on released ANTLR 3.0, not in ANTLR 3.0b6. Maybe ANTLR 3.0b6 just 
forgot to throw warnings for some "habits" that now in ANTLR 3.0 are 
discarded?

Again, thank you very much.
Jose.

On Thursday 21 June 2007 17:04, Jim Idle wrote:
> I see what You are trying to construct I think, but this is not the way
> to go about it as best I can tell without your full grammar.
>
> It seesm that you want a placeholding token that recognizes the keywords
> that begin with @ and sets them to their particular type. I presume that
> you have some reason for not just listing them as:
>
> STATIC  : '@static' ;
> ISA	  : '@isa' ;
> ISATYPE : '@isatype' ;
> AT  : '@' ;
>
> When you have a lot of things like this you can end up with more code
> than you expect I have found. So, I tend to do this:
>
> tokens
> {
> 	STATIC;
> 	ISA;
> 	ISATYPE;
> }
>
> AT : '@'
> 	(
> 		  'static' 	{ $type = STATIC; }
>
> 		| 'isa'	{ $type = ISA;	}
> 		| 'isatype'	{ $type = ISATYPE;}
> 		|		{ $type = AT; // Not needed but
>
> documents the rule }
> 	)
> 	;
>
> This tends to generate simpler code and gives you the token types you
> want back at the parser :-).
>
> The reason that your first attempt does nto work is because there is no
> token for a fragment rule per se, hence the code generated by the $type
> has not token being constructed that it can affect the type of so you
> get this error. In the second instance, you are trying to use predicates
> from a rule that is not a fragment to enter into a non fragment rule.
>
> You can pick up things like the text associated with a fragment rule
> like this:
>
> FRED : '"' bodytext=STRING_GUTS '"'
> 		{ System.out.println("text is " + $bodytext.text); }
> 	;
>
> fragment
> STRING_GUTS :	( EscapeSequence | ~('\\'|'"') )* ;
>
>
> fragment
> EscapeSequence
>
>     :   '\\' ('b'|'t'|'n'|'f'|'r'|'\"'|'\''|'\\')
>
> 	;
>
> Which can tends to fool you into thinking there is an actual token
> emitted by the fragment. You can of course use it to construct your own
> token, which you can emit() in the lexer action.
>
> Hope that helps,
>
> Jim
>
> > -----Original Message-----
> > From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> > bounces at antlr.org] On Behalf Of Jose San Leandro
> > Sent: Thursday, June 21, 2007 3:18 AM
> > To: antlr-interest at antlr.org
> > Subject: [antlr-interest] Problem with $type in final ANTLR 3.0
> >
> > Hi,
> >
> > I've been using a grammar which used $type keyword (the replacement of
> > ANTLR2
> > setType()[1]) to guide the lexer and define explicitly the type of the
> > matched token:
> >
> > fragment AT
> >
> >     :  (  ('@static')    => STATIC {$type = STATIC;}
> >     :
> >         | ('@isa')       => ISA {$type = ISA;}
> >         | ('@isatype')   => ISATYPE  {$type = ISATYPE;}
> >         | '@')
> >
> >     ;
> >
> > With ANTLR 3.0 (Relased May 17), I get compilation errors in the
> > generated
> > lexer:
> >
> > [..]
> >             switch (alt1) {
> >                 case 1 :
> >                     // PerComment.g:575:11: ( '@static' )=> STATIC
> >                     {
> >                     mSTATIC(); if (failed) return ;
> >                     if ( backtracking==0 ) {
> >                       _type = STATIC;
> >                     }
> > [..]
> >
> > The _type variable is not defined.
> >
> > This issue has a different flavour on non-"fragment" rules:
> >
> > [..]
> >             this.type = _type;
> >         }
> >         finally {
> >         }
> >     }
> >     // $ANTLR end AT
> >
> > The compilation stops since type is not a declared attribute.
> >
> > Do you have any suggestions?
> >
> > Kind regards,
> > Jose.


More information about the antlr-interest mailing list