[antlr-interest] Explicit whitespace handling in W3C grammars

[Gavin Lambert]

At 02:32 23/10/2007, Andreas Ravnestad wrote:
 >Consider this example:
 >A : B C;
 >
 >With explicit whitespace handling, there can NOT be whitespace
 >between B and C in this production.
 >
 >How can this be implemented with Antlr?

Write it as a lexer rule.  Since whitespace is typically converted 
into a hidden token (or discarded outright) as another (lower 
priority) lexer rule, all other lexer rules that support internal 
whitespace must explicitly state where it is allowed to occur.

(The same would be true of parser rules as well, if whitespace 
wasn't skipped or hidden during lexing.)