[antlr-interest] Resolving ambiguities in Lexer rules
Achint Mehta
achintmehta at gmail.com
Sun Aug 16 10:06:31 PDT 2009
Hi David/Sarah,
Thanks for your response.
So If I have to include all the tokens in SPECIAL_STRING, then I assume that
SPECIAL_STRING has to be changed from lexer to parser rule.
Based on your and Joe's suggestion I have modified the grammar as following:
----------------------------------------------------------
grammar sample_parser;
options
{
language=C;
}
tokens{
T_VERSION='VERSION';
T_COUNT='COUNT';
}
requestline : T_VERSION '=' special_string (NEWLINE)?;
special_string:WORD | tokenset;
tokenset: T_VERSION | T_COUNT;
fragment
CHAR: (('a'..'z')|('A'..'Z'));
fragment
DIGIT: ('0'..'9');
WORD: (CHAR|DIGIT)+;
NEWLINE:('\r')? '\n';
------------------------------------------------------------
This grammar seems to work.
Do you think this grammar is fine or can it be improved further ?
Thanks.
Regards,
Achint
On Sun, Aug 16, 2009 at 10:45 AM, David-Sarah Hopwood <
david-sarah at jacaranda.org> wrote:
> Achint Mehta wrote:
> > 2. The second option is that all the tokens have to given as alternate
> > rules/token with SPECIAL_STRING. Again, in a big/complicated parser, all
> the
> > tokens in the whole parser have to be repeated where-ever I intend to use
> > the SPECIAL_STRING. This can be simplified if I give the tokens in the
> > definition of SPECIAL_STRING iteself. But still in a parser which could
> use
> > tens or hundreds of tokens, it would seem to be impractical to repeat all
> > the tokens in SPECIAL_STRING rule and other similar rules (intended for
> > collecting the generic string).
>
> You only need one rule that collects all of the tokens, which can then be
> included as an alternative in SPECIAL_STRING and similar rules. This is
> the approach I use for ECMAScript 5 (which has some contexts in which
> keywords are treated like identifiers). The duplication is mildly
> irritating, but it still works quite well in practice.
>
> --
> David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090816/a2ff10c3/attachment.html
More information about the antlr-interest
mailing list