[antlr-interest] [BUG] 3.0b4 no complaint on parser reference to lexical fragment

[Kay Roepke]

On 13. Nov 2006, at 2:44 , John B. Brodie wrote:

>
> i must be really stupid (but hopefully we can fix that ;-)

i doubt that. the first part, i mean ;)

> in the above rules, i see that the lexer will recognize a 'c' in  
> the input
> stream ---- but never pass that fact up to the parser! is that real?

yeah, totally contrived example and not useful. sorry 'bout that.

> perhaps you meant something like:
>
> FOO : 'x' C 'y' ;
> fragment C : 'c' ; // reference to C in parser is legal
> B : 'b' ( 'a' { $type = C; } )? ;

consider the input:
xcxbba

the lexer would return the following tokens (when you look at their  
type):
FOO B C

> and i am suggesting:
>
> tokens { C; } // reference to C in parser is legal
> FOO : 'x' X 'y' ;
> fragment X : 'c' ; // reference to X in parser is illegal
> B : 'b' ( 'a' { $type = C; } )? ;

this works, but token types C and X will be different.

well, ANTLR treats fragment rules specially, but not *that* specially.
fragment X creates a valid token type even though rule X cannot and  
does not
return a token by itself. OTOH another rule can use that token type  
and set it
manually, thus passing X tokens to the parser.

-k

-- 
Kay Röpke
http://classdump.org/