[antlr-interest] Lexer bug? (with test cases!)

[Gavin Lambert]

Mere moments ago, I wrote:
 >Then classify the rule like so:
[...]

Couple of afterthoughts:

First, any (...)* loop is treated as if it were ((...)+)? for 
purposes of this classification algorithm (since that reduces the 
complexity of combinations a bit while still retaining correct 
behaviour).

Second, there's an extra rule:
7. Any required element (nonzero lookahead) following an optional 
subclause
   'foo'? 'bar'
   (('0'..'9')+)? 'bar'
   Required lookahead = up to (lookahead of optional clause) + 
(lookahead of trailing clause)

This is a bit of a special case because as soon as any part of the 
optional clause would fail to match then the clause as a whole is 
abandoned and the lookahead just becomes the trailing 
portion.  But if the optional clause is actually present in the 
input then it needs to be scanned completely through so the tail 
clause can be tested before the rule as a whole can be valid.