[antlr-interest] combine tokens in rewrite rule
Curtis Clauson
NOSPAM at TheSnakePitDev.com
Fri Nov 9 13:56:23 PST 2007
I must admit, I am somewhat confused by the answer given in the
referenced thread. Doesn't the use of fragment lexer rules solve this?
For example, the grammar below will parse this input
<<
name
name.subName1
name.subName1.subName2.subName3
>>
into a tree that has three ID nodes under the root nil node, each with
the complete qualified ID text as a single token. Is this what you mean,
or have I missed something?
(tested with AntLR v3.0.1 and ANTLRWorks v1.1.4)
----------
grammar ABer1;
options {
output = AST;
}
unit: ID+;
ID: UnqualifiedID ('.' UnqualifiedID)*;
WS: (' ' | '\t' | '\r' | '\n' | '\f')+ {$channel = HIDDEN;};
fragment UnqualifiedID : UnqualifiedIDFirst (UnqualifiedIDRest)*;
fragment UnqualifiedIDFirst: 'a'..'z' | 'A'..'Z' | '_';
fragment UnqualifiedIDRest : 'a'..'z' | 'A'..'Z' | '_' | '0'..'9';
----------
I hope that helps.
-- Curtis
Adrian Ber wrote:
> Hi all,
>
> I want to find a way to combine multiple tokens in a single one.
> I've searched the archive and found this thread: http://www.antlr.org/pipermail/antlr-interest/2007-January/019161.html.
> Does any of you have a short sample code on how to do it?
More information about the antlr-interest
mailing list