[antlr-interest] Solving lexer ambiguities

Jose Juan Tapia jjtapia at gmail.com
Tue Sep 11 17:45:32 PDT 2012


So I was gaving a problem with my lexer recognition where my double token
is defined as follows.

FLOAT:
  (DIGIT)+ '.' (DIGIT)* EXPONENT?
| '.' (DIGIT)+ EXPONENT?
| (DIGIT)+ EXPONENT;


However additional to that I have certain structures where the following
syntax:

1structure.2structure .35

Should be recognized by the following grammar

STRING (DOT STRING)? FLOAT

The problem being of course, that my lexer is recognizing the .2 token as a
FLOAT and I'm not sure how can I make it so that it choses the alternative
solution. (I've tried using  backtracking to no avail. Maybe I'm doing it
wrong but my current assumption is that since the ambiguity is at the lexer
rather than at the parser level the parser can't do much to solve the
conflict).
-- 
José Juan Tapia Valenzuela
Research Associate
University of Pittsburgh
3076.1 Biological Sciences Tower 3
Pittsburgh, Pa 15260


More information about the antlr-interest mailing list