[antlr-interest] should lexical rules with identical content be treated equally?

Claude Quézel cquezel at mechatools.com
Fri Oct 12 14:03:21 PDT 2007


There is a typo in the example a submited. The original code was:

   protected Variable : '{' ID '.' ID '}' ;  // **** this is the line that
puzzles me! ****

and when I posted I replaced ID by DVCONTENT intead of DVCONTEXT

The problem is still there. Thank You Austin for pointing this out.

Claude

2007/10/12, Claude Quézel <cquezel at mechatools.com>:
>
> I'm very new to antlr and this may be a trivial error of mine but here it
> goes:
>
> Given this grammar (cut down for this post):
>
>
> paragraph : Non_variable;
>
> protected Variable : '{' DVCONTENT '.' ID '}' ;  // **** this is the line
> that puzzles me! ****
>
> protected Non_variable : (ESCAPED_SEQUENCE | CHARACTERS)*;
>
> fragment CHARACTERS : ~SPECIAL_CHARACTERS;
>
> fragment ESCAPED_SEQUENCE : '\\' SPECIAL_CHARACTERS;
>
> fragment SPECIAL_CHARACTERS : ('{' | '}' | '\\' );
> fragment DVCONTEXT : ID;
> fragment ID : IDLETTER (IDLETTER | DIGIT)*;
> fragment IDLETTER : 'a'..'z'|'A'..'Z';
> fragment DIGIT    : '0'..'9';
>
>
> if I execute this with the following input: "a\nb", I get a "line 1:1
> mismatched character '\n' expecting set null" warning.
>
> if I change the "puzzling line" to:
>
> protected Variable : '{' ID '.' ID '}' ;  // **** this is the line that
> puzzles me! ****
>
> then I do not get the warning. If I compare the generated lexer code,
> there is one line that differs:
>
>
>     public final void mCHARACTERS() throws RecognitionException {
>         try {
>             // D:test.g:25:21: (~ SPECIAL_CHARACTERS )
>             // D:test.g:25:23: ~ SPECIAL_CHARACTERS
>             {
> /*differ*/            if ( (input.LA(1)>='\u0000' && input.LA(1)<='\t')||(
> input.LA(1)>='\u000B' && input.LA(1)<='\uFFFE') ) { // first case (skips
> \n)
> /*differ*/            if ( ( input.LA(1)>='\u0000' && input.LA
> (1)<='\b')||(input.LA(1)>='\n' && input.LA(1)<='\uFFFE') ) { // second
> case (skips \t)
>
>                 input.consume();
>
>             }
>             else {
>                 MismatchedSetException mse =
>                     new MismatchedSetException(null,input);
>                 recover(mse);    throw mse;
>             }
>
>
>             }
>
>         }
>         finally {
>         }
>     }
>
> Can anybody explain what my problem is?
>
> Thank you
>
> Claude
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20071012/9a570bc2/attachment.html 


More information about the antlr-interest mailing list