[antlr-interest] Advice sought on constructing "tree of token lists"

[micheal_jor <open.zone at virgin.net>]

I am attempting to use an ANTLR parser to build an AST in which all 
the nodes are essentially anchors for imposing some structure on the 
input list of Tokens.

The parser is itself a TokenStream and is being used for pre-
validation and pre-simplification of the input stream. All the input 
tokens can potentially pass through or a few may be dropped or 
altered.

I am seeking advice on how to persuade an ANTLR parser to allow 
access to all the original tokens as matched in each rule. I am 
proceeding on the assumption that buildAST must be false and that I 
have to do all the treebuilding manually since ANTLR parsers would 
normally "convert" input tokens into corresponding AST objects.

To clarify:

1. An AST node in the tree is a mechanism for storing a list of 
tokens matched by a rule. The tokens corresponds to the grammar rule 
that constructed the node.

2. An AST node may have one or more such lists of tokens - the list 
of tokens may be partitioned (using action code) depending on the 
rule.

Cheers,

Micheal



 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/