[antlr-interest] ANTLR 3 & hidden token management?
kroepke at classdump.org
Mon Aug 7 02:39:42 PDT 2006
On 7. Aug 2006, at 11:24 Uhr, Loring Craymer wrote:
> From what I've seen, Martin tends to complain about design
> missteps, and most of his complaints are accurate. The complaint
> here is that numbered channels are non-intuitive and it would be
> better to support symbolic names. I have to agree--and Ter
> probably will too when he thinks about it. One solution would be
> to add a lexer option, "channels", which takes a string with comma-
> separated names or some such. ( channels = "HIDDEN, COMMENTS"; )
> Or have a "channels" section in the lexer.
Loring and *,
I didn't want to sound unappreciative, so if I did come across that
way, I'm sorry for that. I was just a bit bewildered by the
discuss syntax changes this late in the game (knowing how such
discussions go, most of the time...).
As for the channel handling, I agree with you that named channels are
more descriptive and more intuitive than an arbitrarily chosen
number, e.g. 99.
I was just trying to make the point that this naming scheme shouldn't
be handled in the default runtime, because there it could lead to
misunderstandings as to how things work with channels, but that's
essentially what you say, too.
Effectively, for the standard out-of-the-box CommonTokenStream class,
all tokens not on the default channel (== 0) are hidden, so I'm a bit
to call channel 99 the "hidden channel", because every other channel,
save one, are also "the" hidden channel. It really only makes things
Now, if I plan to use multiple channels, I will hopefully have the
guts to #define me a descriptive name for that and not use raw
over the place. But that's more the responsibility of the grammar
Maybe it's more useful to be able to specify channelOverrideMap
(which maps tokentypes to channels in the CommonTokenStream) somehow
in the grammar, so that lives close to the definition of the tokens
themselves, the lexer rules.
But I still think that there are more important things, like shipping
3.0 final and when the ANTLR grammars are rewritten to be self-
Ter can incorporate the outcome of such discussions as this into ANTLR.
Kay Röpke <kroepke at classdump.org>
More information about the antlr-interest