[antlr-interest] ANTLR 3 & hidden token management?

[Terence Parr]

On Aug 6, 2006, at 1:17 PM, David Holroyd wrote:

> Hi!
>
> On Sun, Aug 06, 2006 at 10:25:11AM -0700, Terence Parr wrote:
>> As for trees, I automatically track start/stop token indexes for all
>> rule invocations and hence I know the precise range of tokens
>> associated with each rule tree construction.  This includes all
>> hidden channels simply because they are in range.  Now, it does *not*
>> track whitespace etc... before/after the rule...those will presumably
>> be encompassed by other rule's AST construction.
>
> I can see how that would give access to {channel=99} tokens  
> produced by
> lexing and parsing.  However, I want to be able to insert 'synthetic'
> nodes into the AST / token stream, as I'm building an API for
> programmatic code generation.

Ah!  okay, hmm....  I guess there is no input token number associated  
with these nodes.  If it is the root node, then ANTLR will  
automatically set the start/stop indexes.  If the imaginary node is  
in the middle of the tree somewhere then yes you would have to update  
those indexes yourself.  Are you asking about inserting nodes after  
the fact of tree construction? if so, I simple possibly recursive  
function will handle the token update per my article on the website  
about tracking token indices with version 2.

Ter