[antlr-interest] Problem with lexing: EOF and syntactic predicate.

[Taco H. in den Bosch]

Hi everyone,

 

I have been using Antlr for some time now and have done various things with
it. Now I have two (related) problems, however which I seem to be unable to
address.

 

I have the following lexer productions:

 

EOS : '.' 

      ( '\t' | ' ' | '\n' )

      ;

 

Number : 

      ('0'..'9' )+

      (

            ( '.' '0'..'9' ) =>

            '.'

            ('0'..'9')+

      )?

      ;

 

(1) I want EOS to recognize a period followed by whitespace or the end of
the file. I tried

 

EOS : '.' 

      ( '\t' | ' ' | '\n' | EOF )

 

But that does not work as EOF can only be used in parsers, not lexers.
However I do not know another way to do this.

 

(2) I want the period of the production "Number" to be recognized only when
followed by digits (to form floating point numbers). Otherwise the lexer
should stop recognizing the production at the period, so the period can be
part of the EOS production. Thus:

 

123.456 => [Number="123.456"]

123.<space> => [Number="123"][EOS]

 

I thought at first that the syntactic predicate in the number production
should do the trick, but I now think syntactic predicates only work within a
production, i.e.:

 

Number : 

      ('0'..'9' )+

      (

            ( '.' '0'..'9' ) =>

            '.'

            ('0'..'9')+

      |

            '.'

            // Period with no following digits matched here

      )

      ;

 

Can anyone help me out?

 

Thanks in advance,

Taco.

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20050630/ce4a4450/attachment-0001.html