[antlr-interest] Re: Folding Text
lgc at mail1.jpl.nasa.gov
Sun Feb 29 23:28:34 PST 2004
Try something like
( ('\n'! ( ' ' | '\t')+ )=> '\n'! ( ' ' | '\t' )+
That matches everything to end of line, including continuation lines
but removing the newline on continuation lines ('\n'!).
--- In antlr-interest at yahoogroups.com, "offline314159" <offline at s...>
> The reason i want it as a single piece is because there are so many
> different forms of data that can come in that field
> They run the gamut from semi-colon-separated lists of (~';')* to
> base-64 or quoted-printable encoded text. The latter are the ones
> that give me serious fits, since i really don't know how to parse
> those. But i figure worst case, i get the whole thing in as one
> element and use other parsing methods on it as needed.
> The other kicker is, the type of encoding is specified by a token that
> precedes that value, by some arbitrary (albeit bounded by the start of
> the line) number of tokens.
> Can you suggest the syntactic predicate and (i assume) text rewriting
> that i would need to do this? I'm at a bit of a dead end here.
> Thanks for the help, and if someone *does* see a better solution than
> the one i've got, please feel free to tell me!
> --- In antlr-interest at yahoogroups.com, "lgcraymer" <lgc at m...> wrote:
> > Yes, it is possible--it will require a syntactic predicate to skip
> > over the line continuation if you do the processing directly. It
> > would be easier to do it as a token filter.
> > However, this looks like you are trying to do too much in the lexer (a
> > pretty common early mistake--part of the learning process is where to
> > divide processing passes). Ask yourself why you want "value ... line
> > 2" as a single token. If you really want it as one string for post
> > processing, then you can re-consolidate it from multiple tokens in the
> > parser; that way, you can guarantee fixed spacing under your control.
> > If you are going to do further processing on the contents, there is
> > a good chance you want to keep these as separate tokens in a syntax
> > --Loring
Yahoo! Groups Links
<*> To visit your group on the web, go to:
<*> To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
More information about the antlr-interest