[antlr-interest] Basic question : howto differantiate value/pair combination

[Ric Klaren]

On Thu, Jul 29, 2004 at 09:20:30AM -0000, Bruno Delzant wrote:
> <!--
> class TestParser extends Parser;
> 
> startRule        : arg (SEPARATOR arg)* ;
> arg              : ID EQ VALUE ;
> 
> class TestLexer extends Lexer;
> 
> protected UPPER  : ('A'..'Z');
> protected LOWER  : ('a'..'z');
> protected LETTER : (LOWER | UPPER);
> protected DIGIT  : ('0'..'9');
> 
> ID     : UPPER (LETTER | DIGIT)* ;
> VALUE  : (LETTER | DIGIT)+ ;
> 
> protected ID_OR_VALUE : (ID EQ) => ID { $setType(ID); }
>                       | (ID) => VALUE { $setType(VALUE); };
> 
> EQ          : '=';
> SEPARATOR   : ','|';'|' ';
> 
> -->
> 
> I always get the error :
> 
> <!--
> ANTLR Parser Generator   Version 2.7.3   1989-2004 jGuru.com
> test.g: warning:lexical nondeterminism between rules ID and VALUE upon
> test.g:     k==1:'A'..'Z'
> -->

Non protected lexer rules get 'or'-ed together in one nextToken method in
the generated lexer. So ID and VALUE indeed are nondeterministic.

Note that protected rules will never pass a value to the parser. So making
ID_OR_VALUE protected will not work.

Making ID and VALUE protected and ID_OR_VALUE not protected will probably
fix the problem.

You could also only have one rule for ID and VALUE and check the token in
the parser for the upper case start letter. (e.g. move the check into the
parser (this will give slightly better performing code))

Usually with this sort of things it helps to read the generated code to see
what antlr made of it (sometimes the warning can be ignored). When things
go wrong don't forget running antlr with -traceParser / -traceLexer /
-traceTreeParser to get trace output from the generated parsers.

Cheers,

Ric
-- 
-----+++++*****************************************************+++++++++-------
    ---- Ric Klaren ----- j.klaren at utwente.nl ----- +31 53 4893755  ----
-----+++++*****************************************************+++++++++-------
  Chaos always defeats order because it is better organized.
  --- Terry Pratchet



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/