[antlr-interest] lexing tips needed... (or, could I use predicate in lexer rule?)

Lloyd Dupont ld at galador.net
Sat Jul 14 18:02:18 PDT 2007


For my parser I reaped of the lexer rules from the Java exemple.
Particularly you've got this lexer token:
NUM_FLOAT
 : DIGITS '.' (DIGITS)? (EXPONENT_PART)? (FLOAT_TYPE_SUFFIX)?
 | ............ other match
 ;

Now there is a problem with that!....
in the language I target everything is an object.

so:
"2."  is NUM_FLOAT["2."]
"2.1"  is NUM_FLOAT["2.1"]
"2.ToString()" is NUM_INT["2"] DOT["."] ID["ToString"] LPAREN["("] RPAREN[")]

So I was trying to disambiguate the lexer with a construct like that (trying predicate in the lexer):
NUM_FLOAT
 : (
   ( DIGITS '.' EXPONENT_PART )=> DIGITS '.' (EXPONENT_PART) (FLOAT_TYPE_SUFFIX)?
  | ( DIGITS '.' ID )=> DIGITS
  | DIGITS '.' (DIGITS)? (EXPONENT_PART)? (FLOAT_TYPE_SUFFIX)?
  )
  |    ..................... other match
;

but this doesn't seem to work,
1. it told me the NUM_INT rule is now not accessible anymore (I guess the NUM_FLOAT rule absorbs it with my second alternative)
2. but more importantly an input such as "2.a" return "2." followed by MismatchTokenException.

can I use predicate in lexer rule?
how could I disambiguate the NUM_FLOAT lexer rule?
I.e. being both able to read "2.3" (NUM_FLOAT) and "2.ToString()" (NUM_INT DOT ID LPAREN RPAREN)?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20070715/d718ba1b/attachment-0001.html 


More information about the antlr-interest mailing list