[antlr-interest] lexing tips needed... (or, could I use predicate in lexer rule?)

[Lloyd Dupont]

MessageThe problem is: the result really depends of what's after.

1. is ambiguous

1.e is ambiguous
1.e3 is a float
1.echo is INT DOT ID

  ----- Original Message ----- 
  From: Micheal J 
  To: antlr-interest at antlr.org 
  Sent: Sunday, July 15, 2007 11:36 AM
  Subject: Re: [antlr-interest] lexing tips needed... (or,could I use predicate in lexer rule?)


  Hi, Forgot to ask:

  What should the lexer return for these values (note the spaces)?
  ".1"
  "1 .1"
  "1 ."

  Micheal

  -----------------------
  The best way to contact me is via the list/forum. My time is very limited. 

    -----Original Message-----
    From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Lloyd Dupont
    Sent: 15 July 2007 02:02
    To: antlr-interest at antlr.org
    Subject: [antlr-interest] lexing tips needed... (or,could I use predicate in lexer rule?)


    For my parser I reaped of the lexer rules from the Java exemple.
    Particularly you've got this lexer token:
    NUM_FLOAT
     : DIGITS '.' (DIGITS)? (EXPONENT_PART)? (FLOAT_TYPE_SUFFIX)?
     | ............ other match
     ;

    Now there is a problem with that!....
    in the language I target everything is an object.

    so:
    "2."  is NUM_FLOAT["2."] 
    "2.1"  is NUM_FLOAT["2.1"]
    "2.ToString()" is NUM_INT["2"] DOT["."] ID["ToString"] LPAREN["("] RPAREN[")]

    So I was trying to disambiguate the lexer with a construct like that (trying predicate in the lexer):
    NUM_FLOAT
     : (
       ( DIGITS '.' EXPONENT_PART )=> DIGITS '.' (EXPONENT_PART) (FLOAT_TYPE_SUFFIX)?
      | ( DIGITS '.' ID )=> DIGITS
      | DIGITS '.' (DIGITS)? (EXPONENT_PART)? (FLOAT_TYPE_SUFFIX)?
      )
      |    ..................... other match
    ;

    but this doesn't seem to work,
    1. it told me the NUM_INT rule is now not accessible anymore (I guess the NUM_FLOAT rule absorbs it with my second alternative)
    2. but more importantly an input such as "2.a" return "2." followed by MismatchTokenException.

    can I use predicate in lexer rule?
    how could I disambiguate the NUM_FLOAT lexer rule?
    I.e. being both able to read "2.3" (NUM_FLOAT) and "2.ToString()" (NUM_INT DOT ID LPAREN RPAREN)?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20070715/a292ba11/attachment-0001.html