[antlr-interest] help requested for selective whitespace

Scott Amort jsamort at sympatico.ca
Sun Jan 29 17:44:05 PST 2006


Hi All,

I'm working away at trying to understand ANTLR and compiler generation
in general.  The current project I am using to help in my learning is
the creation of scanner/parser for a textual musical description
language.  Things are going quite well, except that I have run into a
problem with whitespace.  Currently, my scanner is set to ignore
whitespace (with $setType(Token.SKIP);), so the parser has no knowledge
of any of the whitespace (or comments) contained in the original
document.  However, the description language requires whitespace in a
few instances, for example between musical note descriptions (musical
notes are described by note name, i.e. c, d, e, f..., etc. followed by
other additional data to specificy accidentals, octave and duration).
So, something like this:

a b c

is valid, while:

abc

is not.  My current scanner treats both those examples as equivalent,
since it is discarding whitespace.  Now, if I instead allow the scanner
to tokenize whitespace, the parser gets very confused as I cannot easily
or adequately address all possible locations of whitespace in the
original document.  So, what I am hoping is that there is a method to
selectively test for whitespace in certain cases, while ignoring all
other instances.  Would someone be able to point me in the right
direction?  Perhaps with additional reading or a simple example to get
me started?  Thanks very much for any help!

Best,
Scott



More information about the antlr-interest mailing list