[antlr-interest] missing tokens and strange behaviour regarding some chars

[Kevin J. Cummings]

On 08/03/2010 03:37 AM, Nieves.Salor.Moral at esa.int wrote:

> addition_operator:  ADDITION_OPERATOR
>         ;
> 
> ADDITION_OPERATOR
>         :       '+'|'-'
>         ;

Why the 2 rules?  Why not just:

ADDITION_OPERATOR
	  :	'+' | '-'
	  ;

>  UNSIGNED_INTEGER
>         :       DIGIT+
>         ;
> 
> simple_factor
>         :       addition_operator simple_factor

          :       ADDITION_OPERATOR simple_factor

>         |       NEGATION_BOOLEAN_OPERATOR simple_factor
>         |       constant
>         |       '('expression ')'
>         |       function
>         |       object_property_request
>         |       OBJECT_TYPE partial_path
>         |       'ask user' '(' expression ('default' expression)? ')' 
> ('expect' predefined_type)?
>         ; 
> 
> constant:       BOOLEAN_CONSTANT
>         |       UNSIGNED_INTEGER ( numeric_constant| 
> RELATIVE_TIME_CONSTANT)
>         |       RELATIVE_TIME_CONSTANT
>         |       string_constant
>         |       HEXADECIMAL_CONSTANT
>         ;
> real_constant
>         :       ('.' UNSIGNED_INTEGER)? ('e' addition_operator? 
> UNSIGNED_INTEGER)? 
>         ;

Unless it is legal for your real constants to contain whitespace:

REAL_CONSTANT
          :       ('.' UNSIGNED_INTEGER)? ('e' ADDITION_OPERATOR?
UNSIGNED_INTEGER)?
          ;

In fact, your real_constant could in fact be empty!  This *must* be
creating headaches for you, yes?

> numeric_constant
>         :        real_constant engineering_units?
>         ;

numeric_constant
          :        REAL_CONSTANT engineering_units?
          ;

I don't see a definition of engineering_units.  Could this too be
changed to a LEXER rule and somehow change your:

	UNSIGNED_INTEGER ( numeric_constant | RELATIVE_TIME_CONSTANT )

into another LEXER rule (this would eliminate the possibility of you
being able to lex the string "123 . 567" as a single real_constant....)

Do these changes help you?

-- 
Kevin J. Cummings
kjchome at rcn.com
cummings at kjchome.homeip.net
cummings at kjc386.framingham.ma.us
Registered Linux User #1232 (http://counter.li.org)