[antlr-interest] Why is Filter thwarting me?

[Gavin Lambert]

At 20:54 6/03/2008, alan brown wrote:
>Hmmmm.  I was unaware that grammars and lexers are frequently 
>split into different files.  It seems that putting the lexer in 
>its own file is no problem, but when I put the parser in its own 
>file it complains (in red in intellij) that it can't find any of 
>the tokens my rule is referring to.
>
>Is there some sort of #include that enables my parser to know 
>where it's looking for the tokens or should I consider the red 
>squiggles under all my token names as decoration?  :)

lexer grammar foo;
options { filter=true; }
...

parser grammar bar;
options { tokenVocab=foo; }
...

See: <http://www.antlr.org/wiki/display/ANTLR3/Grammar+options>

>Also, I'm confused about your explanation (though it does seem to 
>help).  If filter=true is only a lexer rule then why should it be 
>a problem defining it in a hybrid (lexer/parser) grammar 
>file?  Surely, as a lexer rule it should only affect the lexer.

Because the point of hybrid grammars is to let you introduce 
additional tokens in parser rules (which get autogenerated into 
lexer rules), and one of the points of filter mode is to have a 
specific author-defined ordering of lexer rules.  The two goals 
conflict.