[antlr-interest] Token Mismatch

Gavin Lambert antlr at mirality.co.nz
Thu Jul 24 14:53:07 PDT 2008


At 05:33 25/07/2008, Brisard, Fred D wrote:
>It seems like any literal specified in the parser rules becomes 
>an implied token.

In a combined grammar, yes.  Personally I dislike this behaviour 
since it leads to specifying the same literal more than once 
(making it open to typos), erodes the logical separation of lexer 
and parser, and gives the token an annoying typename.  But some 
people seem to like them.

>/** Match things like "call foo;" */
>
>r : 'call' ID ';' {System.out.println("invoke "+$ID.text);} ;
>
>ID: 'a'..'z'+ ;
>
>WS: (' '|'\n'|'\r')+ {$channel=HIDDEN;} ;
>
>I get a MismatchedTokenException I run this in the ANTLRWorks 
>debugger for the input
>
>call call;
>
>I would like the second call to be identified as a token of type 
>ID.

You cannot do this directly, but you can specify that the literal 
'call' can be recognised in the context of an identifier:

r : 'call' id ';';
id : ID | 'call';
...

If you're generating an AST, you can also change the type of AST 
node emitted so that the tree walker will indeed see an ID token 
there:

id : ID | a='call' -> ID[$a];

(I'm not 100% certain about this last one's syntax, but I think it 
goes something like that.)



More information about the antlr-interest mailing list