[antlr-interest] Bug in ANTLRWorks interpreter with repeated alternations

[Ben Karel]

Hi all,

Here's a small example for which the interpreter in ANTLRWorks 1.4.2
incorrectly gives MismatchedTokenExceptions:

grammar test;

sa : a EOF;
sb : b EOF;
sc : c EOF;

p : '('   ')';

x :  p | '^' ;

a : (p | '^')*; // does not match ^  does match ()
b : ('^' | p)*; // does match ^    no match for ()
c :     x    *; // does match ^      does match ()

This bug also applies if the p rule is inlined into the others, but only if
p is a two-token rule. If p matches a single token, then the bug goes away
if p is inlined.

Cheers from a happy ANTLR user,
-- Ben