[antlr-interest] Re: Folding Text

lgcraymer lgc at mail1.jpl.nasa.gov
Sun Feb 29 23:28:34 PST 2004


Try something like

second
    :
    ( ('\n'! ( ' ' | '\t')+ )=> '\n'! ( ' ' | '\t' )+
    | ~'\n'
    )+
    ;

That matches everything to end of line, including continuation lines
but removing the newline on continuation lines ('\n'!).

--Loring


--- In antlr-interest at yahoogroups.com, "offline314159" <offline at s...>
wrote:
> The reason i want it as a single piece is because there are so many
> different forms of data that can come in that field
> 
> They run the gamut from semi-colon-separated lists of (~';')* to
> base-64 or quoted-printable encoded text.  The latter are the ones
> that give me serious fits, since i really don't know how to parse
> those.  But i figure worst case, i get the whole thing in as one
> element and use other parsing methods on it as needed.
> 
> The other kicker is, the type of encoding is specified by a token that
> precedes that value, by some arbitrary (albeit bounded by the start of
> the line) number of tokens.
> 
> Can you suggest the syntactic predicate and (i assume) text rewriting
> that i would need to do this?  I'm at a bit of a dead end here.
> 
> Thanks for the help, and if someone *does* see a better solution than
> the one i've got, please feel free to tell me!
> 
> --
> C
> 
> --- In antlr-interest at yahoogroups.com, "lgcraymer" <lgc at m...> wrote:
> > Yes, it is possible--it will require a syntactic predicate to skip
> > over the line continuation if you do the processing directly.  It
> > would be easier to do it as a token filter.
> > 
> > However, this looks like you are trying to do too much in the lexer (a
> > pretty common early mistake--part of the learning process is where to
> > divide processing passes).  Ask yourself why you want "value ... line
> > 2" as a single token.  If you really want it as one string for post
> > processing, then you can re-consolidate it from multiple tokens in the
> > parser; that way, you can guarantee fixed spacing under your control.
> >   If you are going to do further processing on the contents, there is
> > a good chance you want to keep these as separate tokens in a syntax
> tree.
> > 
> > --Loring



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list