[antlr-interest] C Runtime problem with $channel=HIDDEN and SKIP()

[Jim Idle]

On Sun, 2008-10-26 at 21:41 -0400, Buğra Gedik wrote:

> Adding the line "LEXSTATE->tokenStartCharIndex += $text->size
> -1;" after the SKIP() fixes the SKIP bug. 
> 
> 
> 
> But then, the behavior of SKIP and $channel=HIDDEN become the same
> with respect to my original problem, that is when I print the rule
> text using $text->chars, I do not get the hidden or skipped tokens.
> This seems to be the expected behavior with skip() in Java runtime.
> However, with $channel=hidden in Java runtime, I get the hidden chars
> in the text of the rule, using $text, which is inconsistent with the C
> runtime.



Though generally the text for the whole rule isn't that useful, I can
see that it might be in some cases. I will review the Java code and see
if this was deliberately changed to do this or is just a side effect. I
can see arguments for both, but when you get the start and end span in
an AST node, then it includes the HIDDEN tokens, so it probably should
do so in the rule text.

Jim

> 
> 
> Cheers,
> -bugra
> 
> 
> 
> 
> 
> On Sun, Oct 26, 2008 at 8:52 PM, Buğra Gedik <bgedik at gmail.com> wrote:
> 
>         Hi (Jim),
>         
>         
>         
>         With the C runtime when using $channel=HIDDEN the hidden
>         tokens DO NOT appear when printing a rule with $text->chars,
>         whereas with the Java runtime they appear (which I assume is
>         the expected behaviour). The problem can be solved by using
>         SKIP() instead with the C runtime, but the SKIP function has a
>         bug (I saw it the official list of bugs), it messes up the
>         next token (includes the spaces).
>         
>         
>         I'm using 3.1 (not 3.1.1). Any plans on fixing these? Or, are
>         they fixed in 3.1.1?
>         
>         
>         Cheers,
>         -bugra
>         
>         
> 
> 
> 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20081027/72ff1b23/attachment.html