[antlr-interest] Lexer consumes input but doesn't emit all tokens

Glenn McGregor glenn at fenris.net
Mon Aug 6 13:22:11 PDT 2012


Given the partial grammar from a much larger...


tokens { COLON = ':' }

fragment
ALPHA_NUM
     :    'A'..'Z' | 'a'..'z' | '0'..'9';

NAME_LITERAL
     :    '\\'? ALPHA_NUM ( ( ':' | '_' | '-' | ALPHA_NUM )* ALPHA_NUM )? ;

ANY    :    . ;



I would like the input

test:ack:

to arrive as two tokens, a NAME_LITERAL of 'test:ack', and a COLON.

Instead, this input disappears entirely, but parses successfully.

Any suggestions?

Glenn McGregor



More information about the antlr-interest mailing list