[antlr-interest] common tokens
Robert Soule
robert.soule at gmail.com
Thu Feb 26 13:14:24 PST 2009
Hi,
I just found this web page:
http://www.antlr.org/wiki/display/ANTLR3/Composite+Grammars
and I think this is close to what I want. I can create a lexer grammar
that is imported by my two language parsers/lexers, so they can share the
common lexer rule :
SEMI : ';' ;
However, I would prefer to share the literals. Is this possible? This
is preferable, for example, in 'for' loops, where I have something like:
forStmt : 'for' condition '{' body '}' ;
I would like to share a common definition of the 'for' literal, without
having to create a lexer rule:
FOR : 'for' ;
forStmt : FOR condition '{' body '}' ;
Is this possible with composite grammars?
thanks,
Robert
On Thu, Feb 26, 2009 at 11:51 AM, Robert Soule <robert.soule at gmail.com> wrote:
> Hi,
>
> I am converting from one language to another using tree re-write rules.
> I am getting an error when both languages define the same token. For
> example:
>
> From A.g:
> grammar A;
> options { output = AST; }
> tokens { PROGRAM; A; }
> accept : program EOF -> ^(PROGRAM program);
> program : ',' 'a' -> ^(A ',' 'a');
> WHITE_SPACE : (' '|'\t'|'\r'|'\n')+ {
> $channel = HIDDEN; };
>
> From B.g:
> grammar B;
> options { output = AST; }
> tokens { BPROGRAM; B; }
> accept : program EOF -> ^(BPROGRAM program);
> program : 'b' ',' -> ^(B 'b' ',');
> WHITE_SPACE : (' '|'\t'|'\r'|'\n')+ {
> $channel = HIDDEN; };
>
> From AtoB.g:
> tree grammar AtoB;
> options { tokenVocab=A; tokenVocab=B; ASTLabelType=CommonTree; output=AST; }
> accept : ^(PROGRAM program) -> ^(PROGRAM program);
> program : ^(A ',' 'a') -> ^(B 'b' ',');
>
>
> The error seems to occur because both A.g and B.g use a ',' literal, assigned
> different values in their respective .tokens files. When I reference
> the ',' literal
> in AtoB.g, I'm guessing that there is no way to tell if it is A's token or B's.
>
> Is there either a way to scope tokens (A::';') or is there way to
> define a common
> set of tokens that are shared by both grammars, and then import them?
>
> thank you,
> Robert
>
More information about the antlr-interest
mailing list