[antlr-interest] Re: another way to preserve token order in ASTs

thrutchy eric_mahurin at yahoo.com
Sun Aug 1 15:15:49 PDT 2004


--- In antlr-interest at yahoogroups.com, Terence Parr <parrt at c...> wrote:
> 
> On Jul 31, 2004, at 6:42 PM, thrutchy wrote:
> 
> > - changes were minimal (override one method in an AST)
> > - AST nodes need no additional data (but I need one extra node per
> > root - you could argue which is best)
> > - I knew how to also preserve whitespace
> > - it made more sense to me
> 
> Those are good reasons.  I had trouble dealing with tokens not put into 
> the tree.  How can I spit them back out!?

I'm just avoiding the problem!  I'm not going to put in any "!" into
the parser.  I guess you could merge them in with the hiddenTokens,
but I didn't want mix real tokens with whitespace/comments and I
didn't see a nice way to override the "!".  My plan is to make a new
base AST that will have just "right", "down", a simple string for the
"hiddenAfter" (or maybe a byte array or even do some huffman
encoding), so putting real token text in the hiddenAfter strings
probably wouldn't be a good idea.

> > I think the way I really would like to have done this would be to have
> > a "left" children pointer in addition to current children pointer and
> > sibling pointer.  With AST nodes like that, these rules:
> 
> Yeah, double link would be nice.

A bidirectional link list for the children would be another solution -
just point the root to the point in the list where it was removed.  I
was talking about having the root point to 2 unidirectional link lists
of children - left list and right list.




 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list