[antlr-interest] ANTLR enforces LL(1) beyond about 300 tokens

Kenneth Domino kenneth.domino at domemtech.com
Sun Aug 22 10:54:41 PDT 2010


> allows a clean compile but the ANTLR book indicates ANTLR should try to
> match in the order listed.

Hard to say without looking at the full grammar file, but you could try to 
separate
the lexer and parser grammars into two files, then use "option 
{filter=true;}" in
the lexer grammar.  (The option seems to only work if you make two 
grammars.)
That generates a lexer that uses a rule order priority, which is what you 
mention.
By default, Antlr does not generate this kind of lexer since the filter-type 
lexer
has its problems (See The Definitive ANTLR Reference).  Also, when 
separating
the parser and lexer grammars, it seems Antlr has a problem keeping the two 
token lists
in synch.  You can fix this with a hack: declare all tokens, with the same 
exact order,
in the two grammars.  Not sure why,

Ken

 



More information about the antlr-interest mailing list