[antlr-interest] Unexpected Token problem

[Bryan Ewbank]

The change you made eats whitespace and letters as a single token. 
The problem is still in IDENT, because the change you made does not
see each identifier separately.

Go back to your original IDENT rule, remove the reference to WS, and
add a trailing "*"

> IDENT   :       ('a'..'z'|'A'..'Z'|'0'..'9'|'_'|'\'')(WS)* ;

IDENT   :       ('a'..'z'|'A'..'Z'|'0'..'9'|'_'|'\'')*;

In the WS rule, change the code so that that token is skipped; it's
different for the different target languages, so look for SKIP in the
examples that were packaged with ANTLR.

B-)