[antlr-interest] Handling explicit continuation characters

[Sohail Somani]

Johannes Luber wrote:
>> MINUS: '-' ('\r'? '\n' { skip(); })?;
>> PLUS: '+' ('\r'? '\n' { skip(); })?;
>>
>> (Or instead of skipping, you could change the $type to 
>> CONTINUATION and the $channel to HIDDEN.)
>>
>> You will also need to ensure that '-' and '+' won't be consumed by 
>> other lexer rules (or at least that if they are, it's in contexts 
>> where a continuation isn't permitted).  And your parser will have 
>> to be able to cope with getting MINUS and PLUS tokens between 
>> other tokens.
> 
> Wouldn't it be easier to create an own StringStream (dreived from ANTLRStringStream) which silently swallows the + and - as well the following newline? Then both lexer and parser are cleaner.

I would prefer swallowing it in the lexer as written above. I once did
this for a language that had similar line continuations. By documenting
the full language in the grammar file, I could unambiguously know
exactly what subset (or superset) of the language was supported, atleast
syntactically.

Going Java file spelunking is probably the least fun you could have when
trying to figure out why a syntactically valid file is not parsing with
ANTLR.

Just my thoughts on the subject!

-- 
Sohail Somani
Hassle-free time tracking for JIRA

Website: http://worklogassistant.com
Blog   : http://blog.worklogassistant.com
Twitter: http://twitter.com/somanisoftware