[antlr-interest] Extracting a string whose value clashes with token value

[David-Sarah Hopwood]

Benoit Fouletier wrote:
> I think special_string should be a lexer rule, not a parser rule: rename it
> to SPECIAL_STRING. Also, the lexer depends on the order with which you define
> tokens, so make sure you put ANTLRTOKEN  above SPECIAL_STRING.

This will work, but it will generate ambiguity warnings (because "ANTLR"
can be lexed either as an ANTLRTOKEN or a SPECIAL_STRING). To eliminate
the warning and make the intent clear, put ANTLRTOKEN in a 'tokens' block,
i.e.:

  grammar sample_parser;

  options {
    language=C;
  }

  tokens {
    ANTLRTOKEN = 'ANTLR';
  }

  requestline : ANTLRTOKEN WHITESPACE special_string ;

  SPECIAL_STRING : (CHAR | '=' | '.' | '-' | '@' )+ ;
  WHITESPACE  : ( '\t' | ' ' | '\u000C' )+;
  NEWLINE: ('\r')? '\n';
  fragment CHAR: (('a'..'z')|('A'..'Z'));


Note that it is necessary to change CHAR to a fragment rule if
SPECIAL_STRING is a lexer rule (which I agree it probably should be).

-- 
David-Sarah Hopwood  ⚥  http://davidsarah.livejournal.com