[antlr-interest] help requested for selective whitespace

[Andy Tripp]

I, too, need to keep whitespace (and newlines and comments). I do this 
so that I
can determine what code the comments "seem to go with" by analyzing the 
spacing and indentation.
I have the scanner keep all whitespace, and then do a little processing 
with the whitespace tokens
in the mix, and then remove all whitespace,newline, and comment tokens 
from token stream.
I just wrote Java code to manipulate Lists, didn't do it in ANTLR.

Andy

Scott Amort wrote:

>Hi All,
>
>I'm working away at trying to understand ANTLR and compiler generation
>in general.  The current project I am using to help in my learning is
>the creation of scanner/parser for a textual musical description
>language.  Things are going quite well, except that I have run into a
>problem with whitespace.  Currently, my scanner is set to ignore
>whitespace (with $setType(Token.SKIP);), so the parser has no knowledge
>of any of the whitespace (or comments) contained in the original
>document.  However, the description language requires whitespace in a
>few instances, for example between musical note descriptions (musical
>notes are described by note name, i.e. c, d, e, f..., etc. followed by
>other additional data to specificy accidentals, octave and duration).
>So, something like this:
>
>a b c
>
>is valid, while:
>
>abc
>
>is not.  My current scanner treats both those examples as equivalent,
>since it is discarding whitespace.  Now, if I instead allow the scanner
>to tokenize whitespace, the parser gets very confused as I cannot easily
>or adequately address all possible locations of whitespace in the
>original document.  So, what I am hoping is that there is a method to
>selectively test for whitespace in certain cases, while ignoring all
>other instances.  Would someone be able to point me in the right
>direction?  Perhaps with additional reading or a simple example to get
>me started?  Thanks very much for any help!
>
>Best,
>Scott
>
>  
>