[antlr-interest] ANTLR Problem When a Token Name is a Prefix of Another Token Name

Sameh W. Zaky sameh.wz at gmail.com
Mon May 17 16:00:38 PDT 2010


Hey all,

In the following simple grammar:

*start* : *ANIMAL* ('or' *ANIMAL*)* 'and' *SERVICE EOF* ;

*ANIMAL* : ('dog' | 'cat' | 'horse') ;
*SERVICE* : ('dog hardware' | 'software') ;

NOTICE: 'dog' is a proper prefix of 'dog hardware'..
======================================================

*When I run this grammar by giving an input sentence, something goes wrong
whenever I use the token 'dog'..*
"dog and software" --> "dog and" disappears in the input box, and also in
the tree
"dog or cat or software"  --> "dog or" disappears in the input box, and also
in the tree
"cat or dog and software" --> "dog and" disappears..

*While there is no problem with the token 'dog hardware'*
"cat and dog hardware" --> works fine..

I know the reason.. It's because the grammar is confused when one token is a
proper prefix of another token.. So the token with the bigger length works
fine while the other one doesn't..

Any solution to this problem? (Other than changing the name of the token
because in my real grammar I really need the token names to stay as they
are)

Many Thanks for your time :-)

-- 
Sameh W. Zaky


More information about the antlr-interest mailing list