[antlr-interest] Re: Getting at the token text in a parser

lgcraymer lgc at mail1.jpl.nasa.gov
Sun Feb 29 20:03:47 PST 2004


This one's easy--note the use of "SKIP" for whitespace in the
examples.  Basically you want something like

ENDL :
      '\n'
      ( (' ' | '\t')+ { _ttype = SKIP; } | )
     ;

which looks for whitespace after a newline; if present, mark the token
as one to be skipped (you might need Token.SKIP instead of SKIP) so
that the parser does not see it.

ANTLR lexers can be extraordinarily capable--if you can write a lexer
by hand, you can write an ANTLR lexer for the same problem and much
more quickly.  The Python grammar that Ter just posted to the ANTLR
web site is a good example.

--Loring

--- In antlr-interest at yahoogroups.com, "offline314159" <offline at s...>
wrote:
> Halleleujah!
> 
> Thank you for the help.  Much, MUCH better now.
> 
> So, in a not-quite related note, since i'm asking anyway, here's an
> interesting one:
> 
> The input for my parser may consist of "folded" text - basically, some
> lines that are beyond a reasonable length will be wrapped with leading
> whitespace indicating that their values are to be unfolded and treated
> as all one piece.
> 
> So how do i look for that?  I was originally handling it by filtering
> the input, but that was messing with my linefeeds and CRs, so i'd like
> to handle it in the parser.  Essentially what i want is to return a
> single token, call it VALUE, that takes everything from the first ":"
> on a line to the first ENDL character that is not followed by some
> whitespace.
> 
> Can this even be done?
> 
> --- In antlr-interest at yahoogroups.com, "lgcraymer" <lgc at m...> wrote:
> > $getText works in the lexer, not in the parser--in the lexer, there is
> > text associated with a rule, but not in the parser..  What you want to
> > do is label the AST (or Token) nodes that you want to extract text
> > from--say
> > v:value { #v.getText(); }



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list