[antlr-interest] Lexer alt does not generate corresponding token?

[Gerald Rosenberg]

Interesting.  This appears to generate correctly with COMMENT_BLOCK and 
COMMENT_LINE declared as tokens.

COMMENT_LINE_XX
     : SL_COMMENT (SL_COMMENT)+ -> channel(HIDDEN), type(COMMENT_BLOCK)
     | SL_COMMENT -> channel(HIDDEN), type(COMMENT_LINE)
     ;

Curious to know if it runs afoul of the same issue.

On 10/21/2012 4:09 PM, Terence Parr wrote:
> yep, we did that on purpose. If you ever set a token type with a lexer command, is very risky to reference the surrounding lexer rule in a parser.
>
> @Sam: can remember exactly how this a bit us?
>
> Ter
> On Oct 21, 2012, at 2:23 PM, Gerald Rosenberg wrote:
>
>> When generating a lexer containing the rule
>>
>> COMMENT_LINE
>>     : SL_COMMENT (SL_COMMENT)+ -> channel(HIDDEN), type(COMMENT_BLOCK)
>>     | SL_COMMENT -> channel(HIDDEN)
>>     ;
>>
>> no 'COMMENT_LINE' token is produced in the associated token vocab. Splitting the alts into separate rules produces the expected set of tokens.  Bug or am I missing something?
>>
>> Using v4b2.  Test lexer attached.
>> <TestLexer.g4>
>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>