[antlr-interest] Lexer errors when looking for wrong token

Kevin J. Cummings cummings at kjchome.homeip.net
Sun Oct 10 16:49:45 PDT 2010


On 10/10/2010 06:31 PM, A Z wrote:
> Hello,
> 
> I have a lexer with the following rules:
> 
> 
> LBMINUSGT                  : '[->';
> LBASRB                     : '[*]';
> LBAST                      : '[*';
> LBEQUALS                   : '[=';
> LBPLUSRB                   : '[+]';
> LBRACE                     : '{';
> LBRACKET                   : '[';
> MINUS                      : '-';
> 
> The lexer fails(with an error message) when any string of '[-' or '[*' is
> detected. I'm confused why ANTLR cannot tokenize '[-' correctly as LBRACKET
> MINUS. It also discards two characters after the failed token. I do not have
> a static k defined and ANTLR generates no warnings when compiling. I'm still
> debugging but it's slow figuring out how the antlr3dfapredict() function
> works. Any help is appreciated.

Does the following help you out?

fragment
LBMINUSGT: // "[->"
	;

fragment
LBASRB:	// "[*]"
	;

fragment
LBAST:	// "[*"
	;

fragment
LBEQUALS: // "[="
	;

fragment
LBPLUSRB: // "[+]"
	;

LBRACKET: '[' ( ('-' '>')=> '-' '>' { $token = LBMINUSGT; }
              | ('*' ( ']' { $token = LBASRB; }
                     | { $token = LBAST; } )
	      | ('+' ']')=> '+' ']' { $token = LBPLUSRB; }
	      | '=' { token = LBEQUALS; }
	      | )
        ;

-- 
Kevin J. Cummings
kjchome at rcn.com
cummings at kjchome.homeip.net
cummings at kjc386.framingham.ma.us
Registered Linux User #1232 (http://counter.li.org)


More information about the antlr-interest mailing list