[antlr-interest] Re: Folding Text

offline314159 offline at shaw.ca
Sun Feb 29 22:42:05 PST 2004


The reason i want it as a single piece is because there are so many
different forms of data that can come in that field

They run the gamut from semi-colon-separated lists of (~';')* to
base-64 or quoted-printable encoded text.  The latter are the ones
that give me serious fits, since i really don't know how to parse
those.  But i figure worst case, i get the whole thing in as one
element and use other parsing methods on it as needed.

The other kicker is, the type of encoding is specified by a token that
precedes that value, by some arbitrary (albeit bounded by the start of
the line) number of tokens.

Can you suggest the syntactic predicate and (i assume) text rewriting
that i would need to do this?  I'm at a bit of a dead end here.

Thanks for the help, and if someone *does* see a better solution than
the one i've got, please feel free to tell me!

--
C

--- In antlr-interest at yahoogroups.com, "lgcraymer" <lgc at m...> wrote:
> Yes, it is possible--it will require a syntactic predicate to skip
> over the line continuation if you do the processing directly.  It
> would be easier to do it as a token filter.
> 
> However, this looks like you are trying to do too much in the lexer (a
> pretty common early mistake--part of the learning process is where to
> divide processing passes).  Ask yourself why you want "value ... line
> 2" as a single token.  If you really want it as one string for post
> processing, then you can re-consolidate it from multiple tokens in the
> parser; that way, you can guarantee fixed spacing under your control.
>   If you are going to do further processing on the contents, there is
> a good chance you want to keep these as separate tokens in a syntax
tree.
> 
> --Loring 



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list