[antlr-interest] Handling explicit continuation characters

Johannes Luber JALuber at gmx.de
Tue Jan 13 00:05:07 PST 2009


> At 06:53 13/01/2009, Brisard, Fred D wrote:
> >I am parsing a grammar that uses minus or plus at the end of line 
> >to indicate a continuation.
> [...]
> >I would like to just absorb them into the hidden stream so that 
> >the input appears to be on a single line.
> >
> >If I wanted to make a first pass on the input and just absorb 
> >'-\n' and '+\n' then all would be well.  That seems pretty 
> >wasteful to make a pass to just do that.  It seems like I should 
> >be able to do it during the lexical pass.
> 
> You can; you just need to give the lexer enough hinting so that it 
> can cope with both cases properly:
> 
> MINUS: '-' ('\r'? '\n' { skip(); })?;
> PLUS: '+' ('\r'? '\n' { skip(); })?;
> 
> (Or instead of skipping, you could change the $type to 
> CONTINUATION and the $channel to HIDDEN.)
> 
> You will also need to ensure that '-' and '+' won't be consumed by 
> other lexer rules (or at least that if they are, it's in contexts 
> where a continuation isn't permitted).  And your parser will have 
> to be able to cope with getting MINUS and PLUS tokens between 
> other tokens.

Wouldn't it be easier to create an own StringStream (dreived from ANTLRStringStream) which silently swallows the + and - as well the following newline? Then both lexer and parser are cleaner.

Johannes
> 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger


More information about the antlr-interest mailing list