[antlr-interest] Resolving ambiguities in Lexer rules

[David-Sarah Hopwood]

Achint Mehta wrote:
> Hi David/Sarah,
> 
> Thanks for your response.
> 
> So If I have to include all the tokens in SPECIAL_STRING, then I assume that
> SPECIAL_STRING has to be changed from lexer to parser rule.

Yes, exactly.

You may find as you extend the grammar that you also need non-alphanumeric
tokens, in which case only alphanumeric tokens (called "reserved words" in
most computer languages) should go in what is now 'tokenset'. So I would
call it something like 'reservedWord' instead.

> Based on your and Joe's suggestion I have modified the grammar as following:
> ----------------------------------------------------------
> grammar sample_parser;
> 
> options
> {
>    language=C;
> }
> tokens{
> T_VERSION='VERSION';
> T_COUNT='COUNT';
> }
> 
> requestline : T_VERSION '=' special_string (NEWLINE)?;
> 
> special_string:WORD | tokenset;
> tokenset: T_VERSION | T_COUNT;
> fragment
> CHAR: (('a'..'z')|('A'..'Z'));
> fragment
> DIGIT: ('0'..'9');
> WORD: (CHAR|DIGIT)+;
> NEWLINE:('\r')? '\n';
> ------------------------------------------------------------

-- 
David-Sarah Hopwood  ⚥  http://davidsarah.livejournal.com