[antlr-interest] Problem with $type in final ANTLR 3.0

[Jim Idle]

I see what You are trying to construct I think, but this is not the way
to go about it as best I can tell without your full grammar.

It seesm that you want a placeholding token that recognizes the keywords
that begin with @ and sets them to their particular type. I presume that
you have some reason for not just listing them as:

STATIC  : '@static' ;
ISA	  : '@isa' ;
ISATYPE : '@isatype' ;
AT  : '@' ;

When you have a lot of things like this you can end up with more code
than you expect I have found. So, I tend to do this:

tokens
{
	STATIC;
	ISA;
	ISATYPE;
}

AT : '@'
	(
		  'static' 	{ $type = STATIC; }
		| 'isa'	{ $type = ISA;	}
		| 'isatype'	{ $type = ISATYPE;}
		|		{ $type = AT; // Not needed but
documents the rule }
	)
	;

This tends to generate simpler code and gives you the token types you
want back at the parser :-). 

The reason that your first attempt does nto work is because there is no
token for a fragment rule per se, hence the code generated by the $type
has not token being constructed that it can affect the type of so you
get this error. In the second instance, you are trying to use predicates
from a rule that is not a fragment to enter into a non fragment rule.

You can pick up things like the text associated with a fragment rule
like this:

FRED : '"' bodytext=STRING_GUTS '"'
		{ System.out.println("text is " + $bodytext.text); }
	;

fragment
STRING_GUTS :	( EscapeSequence | ~('\\'|'"') )* ;


fragment
EscapeSequence
    :   '\\' ('b'|'t'|'n'|'f'|'r'|'\"'|'\''|'\\')
	;

Which can tends to fool you into thinking there is an actual token
emitted by the fragment. You can of course use it to construct your own
token, which you can emit() in the lexer action.

Hope that helps,

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Jose San Leandro
> Sent: Thursday, June 21, 2007 3:18 AM
> To: antlr-interest at antlr.org
> Subject: [antlr-interest] Problem with $type in final ANTLR 3.0
> 
> Hi,
> 
> I've been using a grammar which used $type keyword (the replacement of
> ANTLR2
> setType()[1]) to guide the lexer and define explicitly the type of the
> matched token:
> 
> fragment AT
>     :  (  ('@static')    => STATIC {$type = STATIC;}
>         | ('@isa')       => ISA {$type = ISA;}
>         | ('@isatype')   => ISATYPE  {$type = ISATYPE;}
>         | '@')
>     ;
> 
> With ANTLR 3.0 (Relased May 17), I get compilation errors in the
> generated
> lexer:
> 
> [..]
>             switch (alt1) {
>                 case 1 :
>                     // PerComment.g:575:11: ( '@static' )=> STATIC
>                     {
>                     mSTATIC(); if (failed) return ;
>                     if ( backtracking==0 ) {
>                       _type = STATIC;
>                     }
> [..]
> 
> The _type variable is not defined.
> 
> This issue has a different flavour on non-"fragment" rules:
> 
> [..]
>             this.type = _type;
>         }
>         finally {
>         }
>     }
>     // $ANTLR end AT
> 
> The compilation stops since type is not a declared attribute.
> 
> Do you have any suggestions?
> 
> Kind regards,
> Jose.