[antlr-interest] Parsing expressions inside a double quoted string?

Matthew Ford Matthew.Ford at forward.com.au
Fri Jul 11 21:41:32 PDT 2003


I my case the problem was inside [  ]   (array indexing)
The language I was parsing suddenly became whitespace sensitive when inside
[ ]
So I used the following in the lexer to control the return of whitespace
You might be able to do something similar inside "  " to return different
tokens in these cases for '${' and  '}'
and drive your parser that way.

matthew

LBRACK  options { paraphrase = "'['";}
  : '[' {booleanStack.push(true); } // return white space
  ;
RBRACK  options { paraphrase = "']'";}
 : ']' {
        booleanStack.pop();
         if (booleanStack.empty()) {
          throw new RecognitionException("Unmatched ']'",getFilename(),
getLine());
         }
        } // restore last state

// Whitespace -- ignored ,   well mostly except inside  [ ]
WS options { paraphrase = "'white-space'";}
   : ( ' '
  | '\t'
  | '\f' {$setText(" "); } // replace with space. I use to delimit [] in
printer output
  | '\b' {$setText(" "); } // replace with space. I use to delimit [] in
printer output
  // handle newlines
  | ( '\r'('\n')?    // Macintosh or Dos
   | '\n'    // Unix (the right way according to Ter)
   )
   { newline(); }
  )
  {
   if (!booleanStack.peek()) {   // skip if not inside  [ ]
     _ttype = Token.SKIP;
   }
  }
 ;

----- Original Message -----
From: "Andrew Deren" <andrew at adersoftware.com>
To: <antlr-interest at yahoogroups.com>
Sent: Saturday, July 12, 2003 4:20 AM
Subject: RE: [antlr-interest] Parsing expressions inside a double quoted
string?


> I had similar problem a while ago. I was doing lexer/parser for cold
fusion
> which has expressions inside of # ("Hello #foo# #2+2#")
> What I did was to use predicates. It would be easier to do with rule based
> parser generator.
> Just have a variable that holds current lexer "mode".
> Default mode, InString and expressionInString. Initially I wanted to use 2
> lexers/parsers, but couldn't find enough docs on that. My current solution
> works as expected, but I don't like it.
> Andrew
>
> -----Original Message-----
> From: Rodrigo B. de Oliveira [mailto:rodrigobamboo at hotmail.com]
> Sent: Friday, July 11, 2003 12:26 PM
> To: antlr-interest at yahoogroups.com
>
> Oh, I see.
>
> I didn't think it was possible... I've just went through all the docs and
> it's clear how it works now, thanks.
>
> I still haven't been able to figure it all out. For a start I want the
> lexers to transform this:
>
> "Hello ${foo}, ${2 + 2}"
>
> into this:
>
> "Hello" + (foo) + ", " + (2 + 2)
>
> I'll give it some thought again later tonite.
>
> Thanks again,
> Rodrigo
>
> ----- Original Message -----
> From: "Pete Forman" <pete.forman at westerngeco.com>
> To: <antlr-interest at yahoogroups.com>; <antlr-interest at yahoogroups.com>
> Sent: Wednesday, July 09, 2003 11:19 AM
> Subject: Re: [antlr-interest] Parsing expressions inside a double quoted
> string?
>
>
> > At 2003-07-09 07:43 -0300, Rodrigo Oliveira wrote:
> > >You mean by using a filter?
> >
> > Not a filter but a second lexer.  The main lexer should switch to the
> > second when it finds the opening double quote.  One example is in the
Java
> > parser for javadoc comments IIRC.
> >
> > --
> > Pete Forman                -./\.-  Disclaimer: This post is originated
> > WesternGeco                  -./\.-   by myself and does not represent
> > pete.forman at westerngeco.com    -./\.-   opinion of Schlumberger, Baker
> > http://petef.port5.com           -./\.-   Hughes or their divisions.
> >
> >
> >
> >
> > Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/
> >
> >
> >
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>
>
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 




More information about the antlr-interest mailing list