[antlr-interest] Bug report: Composite grammar with all tokens defined in Lexer

[George S. Cowan]

Ummm...

Terence, I'm not clear what you tried. I submitted working code that
contains the work-around, and in order to get the error you will need to
uncomment the PUBLIC rule before running. When I do that, the file
Debug_DebugLex.java is quietly not generated.

(Once I think about it, it does seem odd that I submitted working code in a
bug report.)

George


-----Ter's reply-----
Subject: Re: [antlr-interest] Bug report: Composite grammar with all tokens
defined in Lexer


what error?
Ter


---------- Original Message ---------------
Using ANTLR 3.1.1 on Windows XP, I was unable to split Yang Jiang's java.g
(http://openjdk.java.net/projects/compiler-grammar/antlrworks/Java.g) into
separate parser and lexer grammars for a composite grammar. The following
grammar distilled from that one works, but not if the PUBLIC rule is
uncommented. The file Debug_DebugLex.java is not generated; however, there
is no warning or error message.
 
The pattern seems to be that the grammar fails when all the tokens used in
the parse grammar have a lex grammar rule defined for them. 
 
------------ begin Debug.g -----------
 grammar Debug;
 
options { 
          backtrack=true;
          memoize=true;
        }
 
import DebugParse
     , DebugLex
     ;
 
interfaceHeader 
    :   modifiers 'interface' IDENTIFIER
    ;

------------- end Debug.g ------------
 
------------ begin DebugLex.g -----------
lexer grammar DebugLex;
       
INTERFACE
    :   'interface'
    ;
 
// PUBLIC
//     :   'public'
//     ;
 
              
IDENTIFIER
    :   IdentifierStart IdentifierPart*
    ;

fragment
IdentifierStart
    :   'a'..'z' | 'A'..'Z'
    ;                
                       
fragment 
IdentifierPart
    :   'a'..'z' | 'A'..'Z' | '0'..'9'
    ;
 
WS  
    :   (
             ' '
        |    '\r'
        |    '\t'
        |    '\u000C'
        |    '\n'
        ) 
            {
                skip();
            }          
    ;

------------- end DebugLex.g ------------
 
------------ begin DebugParse.g -----------
parser grammar DebugParse;
 
modifiers  
    :
    (   'public'
    )*
    ;
 
interfaceHeader 
    :   modifiers 'interface' IDENTIFIER
    ;

------------- end DebugParse.g ------------