[antlr-interest] Problem with simple tokens

[Markus Stoeger]

Hello,

I'm new to ANTLR and I'm trying to get an a-few-hundred-lines-grammar 
working (fun! ;-). I've been chewing on a problem for quite a while now 
and was finally able to locate the cause but I don't understand it even 
after reading several online docs (haven't got the book yet).

It's probably very easy to explain for someone with a bit of 
experience... here's a short piece of grammar to reproduce the problem:

--- CUT ---
grammar DotProblem;

rule1: Foo ('.' | '!');

Foo: 'foo';
Identifier: 'a'..'z'+ ('.' 'a'..'z'+)*;
--- CUT ---

When running that in the debugger it matches "foo!" but not "foo.", 
which causes a MismatchedTokenException.

Why doesn't it match "foo."?

It has something to do with the Identifier token (which contains a dot) 
but I don't understand why.. note that to match as Identifier the dot 
would have to be followed by at least one letter, which isn't the case 
with "foo.". Also the token Foo should have precedence over the token 
Identifier as it is defined earlier.

thanks, Max