[antlr-interest] Composite grammar issues

[Jim Idle]

Kumar Rangarajan wrote:
> Hi,
>
> First things first. Thanks for creating such a wonderful tool. We use 
> Antlr for our project work, and recently we switched to using 
> composite grammars, as we are a distributed team, and multiple people 
> need to modify various portions of the grammar file.
>
> While most things just work out of the box or could be trivially 
> worked around, one breaking issue is related to the way the lexer is 
> generated, if the composite grammar defines any lexer rules. I am 
> using Java code generation, and we see that the contents of the 
> "header" defined either in the composite file or in the central lexer 
> file does not get copied to the generated lexer code.
>
> This breaks our packaging structure, as the package statement is 
> normally mentioned in the "header" section of either the central or 
> composite grammars, and because it does not propagate to the generated 
> lexer code, it breaks compilation. 
>
> I see that the same issue was reported last year and a bug has been 
> filed for the same - http://www.antlr.org/jira/browse/ANTLR-370
>
> It would be great if anyone can provide a workaround for this or can 
> provide any updates on the potential fix.
>
> Thanks again for creating this (I mean the tool, not the bug :)).
Kumar,

It all works much better if you separate the lexer grammar into its own 
'lexer grammar' file then import the vocab. The lexer is then standalone 
and can be in any package. Ther eis no particular reason you need to 
import the grammar rather than the vocab in my experience. That said, 
there is an issue at the moment that if you import the vocab then 
accidentally use a token that you have not yet defined in the lexer, you 
do not get told about it, so watch for that one.

I don't know the status of that JIRA, but maybe Terence will comment 
later - he is in China right now.

Jim