[antlr-interest] Whitespace: More than meets the eye?

[Gerald Rosenberg]

At 07:00 PM 8/5/2009, Graham Wideman wrote:
>Hi Loring
>
>Thanks for your comments. So when you say that "ANTLR makes no 
>assumptions", I take that to mean ANTLR makes these assumptions:

No assumptions means exactly that.  ANTLR itself has no concept of 
whitespace.  An input text is just a sequence of tokens and it 
entirely up to the lexer rules to group the tokens in some meaningful 
fashion.  ANTLR does have a concept of completely tossing (or hiding) 
tokens if, once recognized, they convey no further meaning in 
recognizing the input text.  That is, the only information they carry 
is that they separate other groups of tokens.

ANTLR allows any token to be skipped or hidden -- no presumption is 
made about the type or value.

So|if|your||input|text|was|||structured|in}some|strange|way.

The so-called white space rule could be:

BARS:  ( '|' | '}' | '.' ) {$channel=HIDDEN;} ;
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090805/42b669b6/attachment.html