[antlr-interest] Lexer alt does not generate corresponding token?

Terence Parr parrt at cs.usfca.edu
Sun Oct 21 16:09:39 PDT 2012


yep, we did that on purpose. If you ever set a token type with a lexer command, is very risky to reference the surrounding lexer rule in a parser.  

@Sam: can remember exactly how this a bit us?

Ter
On Oct 21, 2012, at 2:23 PM, Gerald Rosenberg wrote:

> When generating a lexer containing the rule
> 
> COMMENT_LINE
>    : SL_COMMENT (SL_COMMENT)+ -> channel(HIDDEN), type(COMMENT_BLOCK)
>    | SL_COMMENT -> channel(HIDDEN)
>    ;
> 
> no 'COMMENT_LINE' token is produced in the associated token vocab. Splitting the alts into separate rules produces the expected set of tokens.  Bug or am I missing something?
> 
> Using v4b2.  Test lexer attached.
> <TestLexer.g4>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address



More information about the antlr-interest mailing list