[antlr-interest] Handling explicit continuation characters

[Johannes Luber]

> Johannes Luber wrote:
> >> MINUS: '-' ('\r'? '\n' { skip(); })?;
> >> PLUS: '+' ('\r'? '\n' { skip(); })?;
> >>
> >> (Or instead of skipping, you could change the $type to 
> >> CONTINUATION and the $channel to HIDDEN.)
> >>
> >> You will also need to ensure that '-' and '+' won't be consumed by 
> >> other lexer rules (or at least that if they are, it's in contexts 
> >> where a continuation isn't permitted).  And your parser will have 
> >> to be able to cope with getting MINUS and PLUS tokens between 
> >> other tokens.
> > 
> > Wouldn't it be easier to create an own StringStream (dreived from
> ANTLRStringStream) which silently swallows the + and - as well the following
> newline? Then both lexer and parser are cleaner.
> 
> I would prefer swallowing it in the lexer as written above. I once did
> this for a language that had similar line continuations. By documenting
> the full language in the grammar file, I could unambiguously know
> exactly what subset (or superset) of the language was supported, atleast
> syntactically.

Nothing prevents you to add a comment explaining the differences from the grammar described to the input which is actually accepted. Doing everything in the lexer will make both lexer and parser more complicated and you might miss a corner case, which won't be fun to debug.
> 
> Going Java file spelunking is probably the least fun you could have when
> trying to figure out why a syntactically valid file is not parsing with
> ANTLR.

Use unit tests to check the code (won't be so difficult that you miss a possible case). Or you could create a separate preprocessor program, which strips the continuations. Then you can compare the clean file against your modifications and you will know, if your modifies stream is the culprit or the grammar.

Johannes
> 
> Just my thoughts on the subject!
> 
> -- 
> Sohail Somani
> Hassle-free time tracking for JIRA
> 
> Website: http://worklogassistant.com
> Blog   : http://blog.worklogassistant.com
> Twitter: http://twitter.com/somanisoftware
> 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger