[antlr-interest] Whitespace is significant sometimes

Loring Craymer lgcraymer at yahoo.com
Tue Sep 28 00:43:37 PDT 2010


This is the sort of problem that can be solved with lexer modes.  ANTLR 3 does 
not support lexer modes, however, so the usual approach is two phase parsing, 
where the first pass would parse text outside of [ ] but recognizes [ ... ] 
constructs as single tokens.  The second pass processes the text inside of 
braces.  Not pretty, but it works.  Trying to implement something which requires 
feedback between parser and lexer is a bad idea, even if you can get it to work: 
 the resulting grammars will be very fragile.

--Loring



----- Original Message ----
> From: Daniel Lidström <daniel.lidstrom at gpsgate.com>
> To: Antlr <antlr-interest at antlr.org>
> Sent: Tue, September 28, 2010 12:03:09 AM
> Subject: Re: [antlr-interest] Whitespace is significant sometimes
> 
> From: "Daniel Lidström" <daniel.lidstrom at gpsgate.com>
> [...]
> >  Is there a way to capture
> > the part outside [] as-is,
> > with  whitespace exactly like written, and the part inside [] is relaxed
> > where  whitespace to go hidden
> > channel?
> >
> > Daniel
> >
> >  Here's my grammar, btw:
> >
> > parse
> > : (IDENT | variable)+  EOF
> > ;
> >
> > variable
> > : '[' mult ']'
> >  ;
> >
> > // the multiplication is optional
> > mult
> > : IDENT  ('*' INTEGER)?
> > ;
> >
> > fragment DIGIT : '0'..'9' ;
> >  INTEGER : DIGIT+ ;
> >
> > fragment LETTER : 'a'..'z' |  'A'..'Z';
> > IDENT : LETTER+ ;
> >
> > WS : ' '+ { $channel =  HIDDEN; } ;
> 
> I will try to answer myself. After a good nights sleep I  think I have found 
> a way to solve
> this problem, using actions. I am going  to build an abstract syntax tree 
> from this grammar.
> Then I figured I can  keep a context variable within my parser. Whenever I am 
> parsing  a
> variable (rule above) I set the variableContext to true. Then I capture 
> whitespace within
> the WS rule but only when the variableContext is set to  false. I haven't 
> worked out the
> details yet but I am positive it should  work. Is there anyone who knows of 
> an online
> grammar example that does  something like this?
> 
> Daniel 
> 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: 
>http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> 


      



More information about the antlr-interest mailing list