[antlr-interest] Whitespace is significant sometimes
Loring Craymer
lgcraymer at yahoo.com
Tue Sep 28 00:43:37 PDT 2010
This is the sort of problem that can be solved with lexer modes. ANTLR 3 does
not support lexer modes, however, so the usual approach is two phase parsing,
where the first pass would parse text outside of [ ] but recognizes [ ... ]
constructs as single tokens. The second pass processes the text inside of
braces. Not pretty, but it works. Trying to implement something which requires
feedback between parser and lexer is a bad idea, even if you can get it to work:
the resulting grammars will be very fragile.
--Loring
----- Original Message ----
> From: Daniel Lidström <daniel.lidstrom at gpsgate.com>
> To: Antlr <antlr-interest at antlr.org>
> Sent: Tue, September 28, 2010 12:03:09 AM
> Subject: Re: [antlr-interest] Whitespace is significant sometimes
>
> From: "Daniel Lidström" <daniel.lidstrom at gpsgate.com>
> [...]
> > Is there a way to capture
> > the part outside [] as-is,
> > with whitespace exactly like written, and the part inside [] is relaxed
> > where whitespace to go hidden
> > channel?
> >
> > Daniel
> >
> > Here's my grammar, btw:
> >
> > parse
> > : (IDENT | variable)+ EOF
> > ;
> >
> > variable
> > : '[' mult ']'
> > ;
> >
> > // the multiplication is optional
> > mult
> > : IDENT ('*' INTEGER)?
> > ;
> >
> > fragment DIGIT : '0'..'9' ;
> > INTEGER : DIGIT+ ;
> >
> > fragment LETTER : 'a'..'z' | 'A'..'Z';
> > IDENT : LETTER+ ;
> >
> > WS : ' '+ { $channel = HIDDEN; } ;
>
> I will try to answer myself. After a good nights sleep I think I have found
> a way to solve
> this problem, using actions. I am going to build an abstract syntax tree
> from this grammar.
> Then I figured I can keep a context variable within my parser. Whenever I am
> parsing a
> variable (rule above) I set the variableContext to true. Then I capture
> whitespace within
> the WS rule but only when the variableContext is set to false. I haven't
> worked out the
> details yet but I am positive it should work. Is there anyone who knows of
> an online
> grammar example that does something like this?
>
> Daniel
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
>http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
More information about the antlr-interest
mailing list