[antlr-interest] (no subject)

[Gavin Lambert]

At 03:14 2/11/2008, =?ISO-8859-2?Q?Imre_Andr=E1s?= wrote:
>This compiles ok (I'm still wondering about the $type and token 
>magic), but there is a problem with processing A ::= SEQUENCE { 
>a, SEQUENCE, c} as input. A MismatchedTokenException occurs 
>instead of treating SEQUENCE as an ID.
[...]
>tokens { SQ; }
[...]
>SEQUENCE : 'SEQUENCE';
[...]
>ID: ( SEQUENCE )=> SEQUENCE {$type = SQ;} | ('a'..'z'|'A'..'Z')+ 
>;

I missed the earlier part of this thread, so I'm not entirely sure 
what you're trying to accomplish, but the above seems wrong.

First off, you're defining two top-level tokens that can match the 
character sequence 'SEQUENCE': the SEQUENCE rule, which will 
assign it the token type SEQUENCE, and the ID rule, which will 
assign it the token type SQ (which you are not actually using 
anywhere in your parser rules, so if it ever gets produced your 
parser will have mismatched token errors).

If you want the ID rule to be the only source of matching for the 
input text "SEQUENCE", then you need to make the SEQUENCE rule a 
fragment and either change your parser rules to look for SQ 
instead or to make the ID rule set the token type to SEQUENCE.

Alternatively (provided there aren't weird issues with lookahead) 
you ought to be able to remove the SEQUENCE-matching clause from 
ID entirely.  The input text "SEQUENCE" should always generate a 
SEQUENCE token without that clause in ID being present, so that 
clause is just introducing ambiguity (though it does guarantee 
full lookahead).

>How can I tell ANTLR that 'SEQUENCE' means a list when followed 
>by '{', otherwise it is an ID?

This sounds like an entirely different problem than what the above 
clause is trying to accomplish :)

There are two schools of thought on how to treat out-of-place 
keywords as if they are just normal identifiers, and they're both 
covered here:
   <http://www.antlr.org/wiki/pages/viewpage.action?pageId=1741>http://www.antlr.org/wiki/pages/viewpage.action?pageId=1741