[antlr-interest] ANTLR 3 & hidden token management?

[David Holroyd]

On Sun, Aug 06, 2006 at 08:17:34PM +0000, David Holroyd wrote:
> Hi!
> 
> On Sun, Aug 06, 2006 at 10:25:11AM -0700, Terence Parr wrote:
> > As for trees, I automatically track start/stop token indexes for all  
> > rule invocations and hence I know the precise range of tokens  
> > associated with each rule tree construction.  This includes all  
> > hidden channels simply because they are in range.  Now, it does *not*  
> > track whitespace etc... before/after the rule...those will presumably  
> > be encompassed by other rule's AST construction.
> 
> I can see how that would give access to {channel=99} tokens produced by
> lexing and parsing.  However, I want to be able to insert 'synthetic'
> nodes into the AST / token stream, as I'm building an API for
> programmatic code generation.  For instance, addComment() with ANTLR 2:
> 
> http://svn.badgers-in-foil.co.uk/metaas/trunk/src/java/uk/co/badgersinfoil/metaas/impl/ASTStatementList.java
> 
> Having to potentially update startIndex/endIndex values in the whole
> tree on insertion/deletion sounds a bit hairy.  Does this leave me
> needing custom, linked-list Tree/Token implementations?

Actually, does anyone already have such an implementation to hand?

Google knows of only one reference to org.antlr.runtime.Token at the
moment ;)


many thanks,
dave

-- 
http://david.holroyd.me.uk/