[antlr-interest] duplicate token type when collapsing subrule into set, etc.

Gavin Lambert antlr at mirality.co.nz
Sat Mar 24 13:45:55 PDT 2007

At 05:52 25/03/2007, David Holroyd wrote:
 >I just grabbed antlr-03-22-2007.18.tar.gz to see if the
 >static-lexer-init change gives a speedup when I create large 
 >rules (for unicode identifiers).
 >Running against my existing grammar, which was processed without
 >warnings by an ANTLR from source control around the start of 
Jan, I
 >now get these errors (among many others):
 >  error(204): org/asdt/core/internal/antlr/AS3.g3:1526:6: 
 >token type '\\' when collapsing subrule into set
 >  error(204): org/asdt/core/internal/antlr/AS3.g3:1527:6: 
 >token type '\\' when collapsing subrule into set

This is probably related to the bug I've been discussing with Ter 
for the past couple of versions -- while ANTLR now will try to 
merge sets together so you can reference one lexer rule from 
another when trying to match individual characters (which is an 
improvement since it leads to less duplication), unfortunately at 
the moment it seems to get confused about which rules are sets and 
which rules are sequences.  (I think it uses the rule where it is 
*used* rather than the rule where it is *defined*, but that's just 
a guess.)

For the moment, you'll have to use an earlier version.  Hopefully 
Ter will fix this one soon, though, but I haven't heard from him 
for a few days now.

More information about the antlr-interest mailing list