[antlr-interest] please help on simple parser antlr v3

Thomas Brandon tbrandonau at gmail.com
Wed Sep 27 06:18:57 PDT 2006


I'm not sure quite sure why that is not giving any warnings, it looks
like it should give a few to me. As far as I can see, there should
still be an ambiguity in the closure in WS between alt 2 (CRLF) and
exit (leading to alt 3). Additionally it looks like there should be
ambiguities between WS and LF\CRLF\TAB, as LR, CRLF and TAB are not
declared as fragment.

Is this a bug, at least in error reporting? Or at least an
undocumented feature, it looks like it should still work and it avoids
the warnings so in this case it's rather useful.

Tom.
On 9/27/06, Michiel Vermandel <Michiel_Vermandel at axi.be> wrote:
>
> Hi Tom,
>
> Thanks for your explanation.
> I made some extra lexer rules and I succeeded in eliminating the warnings.
> Though I'm not sure my grammar is still correct and I don't understand why this solution does not generate the warnings.
>
> Do you have any idea?
>
> Thanks!
>
> -------------------------grammar with no warnings-----------------------
>
> grammar TestParser;
> options {k=2; backtrack=true; memoize=true;}
>
>
> statement: identifier | quoted_string ;
>
> identifier:         IDENTIFIER;
>
> quoted_string: QUOTED_STRING;
>
>
> fragment
> AZ_CHAR: 'a' .. 'z' | 'ç' | 'é' | 'è'| 'ê'  | 'à';
>
> fragment
> SPECIAL_CHAR: '_' | '$' | '#' ;
>
> fragment
> ANY_CHAR: AZ_CHAR | SPECIAL_CHAR;
>
> IDENTIFIER:  ( AZ_CHAR | (N)+)  ANY_CHAR  (ANY_CHAR | N)*;
>
> N:  ('0' .. '9')+;
>
> LF        :        '\n' { channel=99; };
>
> CRLF        :        '\r' ('\n')? { channel=99; };
>
> TAB        :        '\t' { channel=99; };
>
> WS        :        (' '
>                 | TAB
>                 | CRLF
>                 | LF
>                 )+
>         { channel=99; };
>
> QUOTED_STRING: '\'' (  ~('\'') | '\'' '\'' )* '\'';
>
> DOUBLE_QUOTED_STRING: '"' ( '""' | ~('"') )* '"';
>
>
>
>
>
>
>
>
>
>
> I don't think Antlr v3 yet allows you to suppress warnings so these are unavoidable. You want to check that Antlr is behaving correctly and ignoring the correct alternatives. As you have no actions it does not matter which alternative Antlr takes, and it is correctly handling your grammar so you can ignore the warnings.
>  I don't think there is any way to refactor out the remaining ambiguity. Removing the WS rule was correct. It is not needed as it is duplicated by the ~('\'') alternative, but the ambiguity between '\n' and '\r'('\n')? is unavoidable, though ideally you would supress the warning.
>
>  Tom.
> On 9/27/06, Michiel Vermandel <Michiel_Vermandel at axi.be> wrote:
>
>  Sorry,
>
>  It would have been better to test the grammar first without the WS rule in the QOUTES_STRING rule since I still get warnings (less though) and now I can't
>  figure out how to solve this.
>  Can someone please tell me how to prevent the multiple alternatives warnings?
>
>  ------------------------new grammar---------------------------
>
>  grammar TestParser;
>  options {k=2; backtrack=true; memoize=true;}
>
>
>  statement: identifier | quoted_string ;
>
>  identifier:         IDENTIFIER;
>
>  quoted_string: QUOTED_STRING;
>
>
>  fragment
>  AZ_CHAR: 'a' .. 'z' | 'ç' | 'é' | 'è'| 'ê'  | 'à';
>
>  fragment
>  SPECIAL_CHAR: '_' | '$' | '#' ;
>
>  fragment
>  ANY_CHAR: AZ_CHAR | SPECIAL_CHAR;
>
>  IDENTIFIER:  ( AZ_CHAR | (N)+)  ANY_CHAR  (ANY_CHAR | N)*;
>
>  N:  ('0' .. '9')+;
>
>  WS        :        (' '
>                  |        '\t'
>                  |        ('\n'|'\r'('\n')?)
>                  )+
>          { channel=99; };
>
>  QUOTED_STRING: '\'' (  ~('\'') | '\'' '\'' )* '\'';
>
>  --------------------remaining warnings---------------------------------
>
>  [14:26:53] test1.g:28:15: Decision can match input such as "'\n'" using multiple alternatives: 1, 2
>  As a result, alternative(s) 2 were disabled for that input
>  [14:26:53] C:\Documents and Settings\mvrm\My Documents\Antlr\grammars\test1.g:28:15: Decision can match input such as "'\n'" using multiple alternatives: 1, 2
>
>  As a result, alternative(s) 2 were disabled for that input
>
>
>
>
>
>
>
>
>  Hi,
>
>  I started converting my v2 grammars to v3 and ran against some warnings when generating code for a very simple test grammar.
>  Below, you find the parser, followed by the warnings.
>  This grammar is a part of a grammar in v2 which I am converting.
>
>  In v2 the rule QUOTED_STRING was:
>
>  QUOTED_STRING: '\'' ( WS | '\'' '\'' | ~('\'') )* '\'';
>
>  Now in v3 I get this list of warnings (see below).
>  In v2 the WS was needed in the QUOTED_STRING rule in order to increment the line counter properly.
>  I think I can solve the warnings by leaving out the WS rule since in Antlr v3 I do not need to bother about incrementing the line counter.
>
>  So, Can please someone confirm that I am right in just leaving out the WS rule from the QUOTED_STRING rule?
>
>  Thanks!
>
>
>  ---------------------------------------------------------------------
>
>
>  grammar TestParser;
>  options {k=2; backtrack=true; memoize=true;}
>
>
>  statement: identifier | quoted_string ;
>
>  identifier:         IDENTIFIER;
>
>  quoted_string: QUOTED_STRING;
>
>
>  fragment
>  AZ_CHAR: 'a' .. 'z' | 'ç' | 'é' | 'è'| 'ê'  | 'à';
>
>  fragment
>  SPECIAL_CHAR: '_' | '$' | '#' ;
>
>  fragment
>  ANY_CHAR: AZ_CHAR | SPECIAL_CHAR;
>
>  IDENTIFIER:  ( AZ_CHAR | (N)+)  ANY_CHAR  (ANY_CHAR | N)*;
>
>  N:  ('0' .. '9')+;
>
>  WS        :        (' '
>                 |        '\t'
>                 |        ('\n'|'\r'('\n')?)
>                 )+
>         { channel=99; };
>
>  QUOTED_STRING: '\'' ( WS | '\'' '\'' | ~('\'') )* '\'';
>
>  -------------------------------------------------------------------------------
>
>  [14:09:37] test1.g:28:15: Decision can match input such as "'\n'" using multiple alternatives: 1, 2
>  As a result, alternative(s) 2 were disabled for that input
>  [14:09:37] test1.g:32:48: Decision can match input such as "' '" using multiple alternatives: 1, 3
>  As a result, alternative(s) 3 were disabled for that input
>  [14:09:37] test1.g:32:48: Decision can match input such as "'\t'" using multiple alternatives: 1, 3
>  As a result, alternative(s) 3 were disabled for that input
>  [14:09:37] test1.g:32:48: Decision can match input such as "'\r'" using multiple alternatives: 1, 3
>  As a result, alternative(s) 3 were disabled for that input
>  [14:09:37] test1.g:32:48: Decision can match input such as "'\n'" using multiple alternatives: 1, 3
>  As a result, alternative(s) 3 were disabled for that input
>  [14:09:37] C:\Documents and Settings\mvrm\My Documents\Antlr\grammars\test1.g:28:15: Decision can match input such as "'\n'" using multiple alternatives: 1, 2
>  As a result, alternative(s) 2 were disabled for that input
>  [14:09:37] C:\Documents and Settings\mvrm\My Documents\Antlr\grammars\test1.g:32:48: Decision can match input such as "' '" using multiple alternatives: 1, 3
>  As a result, alternative(s) 3 were disabled for that input
>  [14:09:37] C:\Documents and Settings\mvrm\My Documents\Antlr\grammars\test1.g:32:48: Decision can match input such as "'\t'" using multiple alternatives: 1, 3
>  As a result, alternative(s) 3 were disabled for that input
>  [14:09:37] C:\Documents and Settings\mvrm\My Documents\Antlr\grammars\test1.g:32:48: Decision can match input such as "'\r'" using multiple alternatives: 1, 3
>  As a result, alternative(s) 3 were disabled for that input
>  [14:09:37] C:\Documents and Settings\mvrm\My Documents\Antlr\grammars\test1.g:32:48: Decision can match input such as "'\n'" using multiple alternatives: 1, 3
>  As a result, alternative(s) 3 were disabled for that input
>
>
>


More information about the antlr-interest mailing list