[antlr-interest] Lexer alt does not generate corresponding token?

Gerald Rosenberg gerald at certiv.net
Sun Oct 21 14:23:08 PDT 2012


When generating a lexer containing the rule

COMMENT_LINE
     : SL_COMMENT (SL_COMMENT)+ -> channel(HIDDEN), type(COMMENT_BLOCK)
     | SL_COMMENT -> channel(HIDDEN)
     ;

no 'COMMENT_LINE' token is produced in the associated token vocab. 
Splitting the alts into separate rules produces the expected set of 
tokens.  Bug or am I missing something?

Using v4b2.  Test lexer attached.
-------------- next part --------------
lexer grammar TestLexer;

tokens {
	COMMENT_BLOCK
}

@lexer::header {
	package net.certiv.test;
}

// match anything between /* and */
COMMENT
    :   '/*' .* '*/'  -> channel(HIDDEN) 
    ;

COMMENT_LINE
	: SL_COMMENT (SL_COMMENT)+ -> channel(HIDDEN), type(COMMENT_BLOCK)
	| SL_COMMENT -> channel(HIDDEN)
	;

/*
COMMENT_BLOCK
	: SL_COMMENT (SL_COMMENT)+ -> channel(HIDDEN)
	;

COMMENT_LINE
	: SL_COMMENT -> channel(HIDDEN)
	;
*/

fragment 
SL_COMMENT
    : '//' ~[\r\n]* '\r'? '\n'
    ;

// white space
WS :  [ \t\r\n\u000C]+ -> channel(HIDDEN)
    ;


More information about the antlr-interest mailing list