[antlr-interest] detecting transitions in stanza-based files

[Jon.Rand at thomson.com]

Chris Black wrote:
>> Excel likes to add enough delimiters at the end of every 
>> line so that all lines have an equal number of columns, 
>> this leads to lots of rules in my grammars that end 
>> in "(DELIM)* NEWLINE" which I understand can be 
>> inefficient and also lead to some nondeterminism 
>> difficulties. I am going to have my TokenFilter remove 
>> these. 

Could you use a regular expression against the input string to remove
the extraneous delimiters before you pass the input to ANTLR?  Then you
wouldn't have to clutter your grammar with stuff that essentially does
exactly the same thing.  It's not a pure ANTLR solution, but it might
make your grammar a lot easier to deal with.

Jon