[antlr-interest] common tokens

Robert Soule robert.soule at gmail.com
Thu Feb 26 08:51:08 PST 2009


Hi,

I am converting from one language to another using tree re-write rules.
I am getting an error when both languages define the same token. For
example:

>From A.g:
grammar A;
options { output = AST; }
tokens { PROGRAM; A; }
accept : program EOF      ->   ^(PROGRAM program);
program : ',' 'a'         -> ^(A ',' 'a');
WHITE_SPACE      : (' '|'\t'|'\r'|'\n')+                     {
$channel = HIDDEN; };

>From B.g:
grammar B;
options { output = AST; }
tokens { BPROGRAM; B; }
accept : program EOF      ->   ^(BPROGRAM program);
program : 'b' ','             -> ^(B 'b' ',');
WHITE_SPACE      : (' '|'\t'|'\r'|'\n')+                     {
$channel = HIDDEN; };

>From AtoB.g:
tree grammar AtoB;
options { tokenVocab=A; tokenVocab=B;  ASTLabelType=CommonTree; output=AST; }
accept : ^(PROGRAM program)  -> ^(PROGRAM program);
program : ^(A ',' 'a')           -> ^(B 'b' ',');


The error seems to occur because both A.g and B.g use a ',' literal, assigned
different values in their respective .tokens files. When I reference
the ',' literal
in AtoB.g, I'm guessing that there is no way to tell if it is A's token or B's.

Is there either a way to scope tokens (A::';') or is there way to
define a common
set of tokens that are shared by both grammars, and then import them?

thank you,
Robert


More information about the antlr-interest mailing list