[antlr-interest] Handling explicit continuation characters
Johannes Luber
JALuber at gmx.de
Tue Jan 13 00:05:07 PST 2009
> At 06:53 13/01/2009, Brisard, Fred D wrote:
> >I am parsing a grammar that uses minus or plus at the end of line
> >to indicate a continuation.
> [...]
> >I would like to just absorb them into the hidden stream so that
> >the input appears to be on a single line.
> >
> >If I wanted to make a first pass on the input and just absorb
> >'-\n' and '+\n' then all would be well. That seems pretty
> >wasteful to make a pass to just do that. It seems like I should
> >be able to do it during the lexical pass.
>
> You can; you just need to give the lexer enough hinting so that it
> can cope with both cases properly:
>
> MINUS: '-' ('\r'? '\n' { skip(); })?;
> PLUS: '+' ('\r'? '\n' { skip(); })?;
>
> (Or instead of skipping, you could change the $type to
> CONTINUATION and the $channel to HIDDEN.)
>
> You will also need to ensure that '-' and '+' won't be consumed by
> other lexer rules (or at least that if they are, it's in contexts
> where a continuation isn't permitted). And your parser will have
> to be able to cope with getting MINUS and PLUS tokens between
> other tokens.
Wouldn't it be easier to create an own StringStream (dreived from ANTLRStringStream) which silently swallows the + and - as well the following newline? Then both lexer and parser are cleaner.
Johannes
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger
More information about the antlr-interest
mailing list