[antlr-interest] lexing tips needed... (or, could I use predicate in lexer rule?)
Micheal J
open.zone at virgin.net
Sat Jul 14 18:29:03 PDT 2007
You may be trying to do too much in the lexer Lloyd. Seems you can simply
change the lexer to emit say NUM_FLOAT_DOT for floats that end with the
decimal point. The lexer seems to handle your other NUM_FLOAT examples just
fine.
You can then deal with the NUM_FLOAT_DOT ID sequence in the parser.
NUM_INT
: DIGITS
(
'.' { $type = NUM_FLOAT_DOT; }
( fraction_digits=DIGITS { $type = NUM_FLOAT; } )?
( exponent_part=EXPONENT_PART { $type = NUM_FLOAT; } )?
( float_suffix=FLOAT_TYPE_SUFFIX { $type = NUM_FLOAT; } )?
)?
| ..................... other match
;
Micheal
-----------------------
The best way to contact me is via the list/forum. My time is very limited.
-----Original Message-----
From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Lloyd Dupont
Sent: 15 July 2007 02:02
To: antlr-interest at antlr.org
Subject: [antlr-interest] lexing tips needed... (or,could I use predicate in
lexer rule?)
For my parser I reaped of the lexer rules from the Java exemple.
Particularly you've got this lexer token:
NUM_FLOAT
: DIGITS '.' (DIGITS)? (EXPONENT_PART)? (FLOAT_TYPE_SUFFIX)?
| ............ other match
;
Now there is a problem with that!....
in the language I target everything is an object.
so:
"2." is NUM_FLOAT["2."]
"2.1" is NUM_FLOAT["2.1"]
"2.ToString()" is NUM_INT["2"] DOT["."] ID["ToString"] LPAREN["("]
RPAREN[")]
So I was trying to disambiguate the lexer with a construct like that (trying
predicate in the lexer):
NUM_FLOAT
: (
( DIGITS '.' EXPONENT_PART )=> DIGITS '.' (EXPONENT_PART)
(FLOAT_TYPE_SUFFIX)?
| ( DIGITS '.' ID )=> DIGITS
| DIGITS '.' (DIGITS)? (EXPONENT_PART)? (FLOAT_TYPE_SUFFIX)?
)
| ..................... other match
;
but this doesn't seem to work,
1. it told me the NUM_INT rule is now not accessible anymore (I guess the
NUM_FLOAT rule absorbs it with my second alternative)
2. but more importantly an input such as "2.a" return "2." followed by
MismatchTokenException.
can I use predicate in lexer rule?
how could I disambiguate the NUM_FLOAT lexer rule?
I.e. being both able to read "2.3" (NUM_FLOAT) and "2.ToString()" (NUM_INT
DOT ID LPAREN RPAREN)?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20070715/71c5a316/attachment.html
More information about the antlr-interest
mailing list