[antlr-interest] Lexer not backtracking properly?
Stevenson, Todd (GE Healthcare)
Todd.T.Stevenson at ge.com
Fri May 21 08:31:13 PDT 2010
I have been working on a lexer grammar that appears to try to process
one rule but,
when it fails, does not back track and find the correct rule.
Consider this grammar:
start
: LBRACE option RBRACE EOF
;
option
: DELETE
| IMPORT
| ALL
;
DELETE : 'delete';
ALL : 'all';
IMPORT : 'import';
LBRACE : '{';
RBRACE : '}';
GUID : LBRACE XDIGIT+ RBRACE;
HEXNUMBER : XDIGIT+;
WS : ( ' ' | '\t' | '\r' | '\n')+ {$channel=HIDDEN;};
fragment
XDIGIT : '0' .. '9' | 'a' .. 'f' | 'A' .. 'F';
------------------------
When I use the following input '{all}' it fails because it appears to
try to process the 'GUID'
rule but when it fails, cannot backtrack and process the 'LBRACE' rule.
It works correctly
when I processing the input '{ all}'. Is this correct behavior?
I know that the GUID rule could probably be a parser rule but I wanted
to not allow embedded
whitespace so I left it as a lexer rule.
More information about the antlr-interest
mailing list