[antlr-interest] ANTLR 3 & hidden token management?

[Prashant Deva]

The channel stuff is nice, but I dont seem to like the syntax 'channel=99'.

Why do you want to use a no? Why not something friendlier and easier to
remember?
A newcomer looking at a grammar would get really confused when he sees
channel=99.
even something like channel=hidden would convey more information.

I have also noticed in quite a few places that people (esp newcomers) seem
to find the antlr syntax too confusing.
Maybe we should reservere a week just to discuss and try to make the syntax
of antlr 3 more friendly.


Prashant

On 8/6/06, Terence Parr <parrt at cs.usfca.edu> wrote:
>
>
> On Aug 6, 2006, at 10:10 AM, David Holroyd wrote:
>
> > ANTLR 2 provides CommonHiddenStreamToken et al, which I try to use for
> > layout-preserving code rewriting.  I've just been looking at v3 to see
> > if manipulating ASTs containing hidden tokens has become any
> > easier, but
> > I don't see equivalent classes.
> >
> > Is this because there's a radically different way to do this which
> > I've
> > not spotted, or because it's just not implemented yet?
>
> Hi Dave,
>
> ANTLR v3 in a sense sends all tokens to the parser, with the hidden
> tokens on different "channels".  this is done with a simple channel
> number in each token object.  The commonTokenStream objct filters for
> only those tokens of interest to the parser, but keeps all tokens
> around.  At any point, you can walk forwards/backwards in the buffer
> looking for stuff of interest on other channels.
>
> As for trees, I automatically track start/stop token indexes for all
> rule invocations and hence I know the precise range of tokens
> associated with each rule tree construction.  This includes all
> hidden channels simply because they are in range.  Now, it does *not*
> track whitespace etc... before/after the rule...those will presumably
> be encompassed by other rule's AST construction.
>
> Just added to faq:
>
> http://www.antlr.org/wiki/pages/viewpage.action?pageId=1057
>
> Ter
>
>


-- 
Prashant Deva
Creator, ANTLR Studio
Founder, Placid Systems, www.placidsystems.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20060807/cc563e31/attachment.html